Recent forum and wiki downtime 2019-05-01...2019-05-03

User avatar
celeron55
Administrator
Posts: 532
Joined: Tue Apr 19, 2011 10:10
GitHub: celeron55
IRC: celeron55

Recent forum and wiki downtime 2019-05-01...2019-05-03

by celeron55 » Post

The forum and wikis were down from 2019-05-01 to 2019-05-03 due to a VPS crash and disk corruption.

During downtime I restored the forum and the wikis to my latest off-site backup dated 2019-03-13 and upgraded the base system to OpenBSD 6.3. Much of the data left on the server was unusable, like the forum database.

Unfortunately that means the latest 6 weeks of data was lost. I'll be looking into implementing more frequent off-site backups to other servers ran by trusted community members.

ContentDB has recorded the missing mod and game titles and the list can be found here: https://gist.github.com/rubenwardy/1cd6 ... 1ffdcbdc4d

I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it. Feel free to re-post any of your lost content.

User avatar
Joseph16
Member
Posts: 310
Joined: Tue Dec 06, 2016 05:35
In-game: smb3

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Joseph16 » Post

Glad to see it's been restored.
Testin' mines since 1989

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Wuzzy » Post

Ouch. That hurts. But it could have been worse. Thankfully, there was a backup. Phew. :-)

Obviously, some of my forum threads for my countless mods became outdated as well. So I had to manually restore them to point to the latest mod version.
@modders: If you released a new version in the past 6 months or so, this means you must go through your mod and game forum threads and re-update them to the latest version.

User avatar
Festus1965
Member
Posts: 4181
Joined: Sun Jan 03, 2016 11:58
GitHub: Festus1965
In-game: Festus1965 Thomas Thailand Explorer
Location: Thailand ChiangMai
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Festus1965 » Post

I had much more time without the Forum to play ... I am not so sad.

add:
Maybe a good warning about what is sure not needed, off topic, video, music and other threads making a lot of work - bad for the forum itself.
The less the better, easier, less mod work ... also look forward to new rules of ... EU
Last edited by Festus1965 on Fri May 03, 2019 22:44, edited 1 time in total.
Human has no future (climate change)
If urgend, you find me in Roblox (as CNXThomas)

User avatar
b3u
Member
Posts: 12
Joined: Mon Jun 25, 2018 01:41
GitHub: b3u
IRC: Fairion
Location: Maryland, USA
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by b3u » Post

Thanks to all who helped get the forums back online.

User avatar
RajMahal
Member
Posts: 10
Joined: Wed Jan 16, 2019 20:56

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by RajMahal » Post

Image
Introducing the #t.backup.#t.borem.#t. recovery system
Thank You for the forum recovery. I suggest FEWER backups, NOT more.
inprocess

ingredients:
#t.backup.&.borem. recovery.system
#t.flog



sources:
#t.forum.minetest
#t.forum.member...celeron55
#t.forum.member...sorcerykid
_ _ _....more to be added
Spoiler





closed
I.suggest.fewer.backups.NOT.more
my.suggestion
_ _ _I suggest FEWER
#t.mtforum...
#v.backup
_ _ _NOT more


***#v.goto.#t.post...25***





closed
YES to security...NO to cloud backups.
my.suggestion
_ _ _YES to security...NO to cloud backups.





***#v.goto.#t.post...26***





inprocess
Introducing.my.system.of.forum.posting
my.observation
_ _ _Introducing the
#t.backup.&.borem. recovery.system
_ _ _Or for those seeking violent alternatives,
#t.flog
for forum/blog.

***#v.goto.#t.post...27***










***#t.post...25***
I.suggest.fewer.backups.NOT.more
my.suggestion
_ _ _I suggest FEWER
#t.mtforum...
#v.backup
_ _ _NOT more


_ _ _my.link

_ _ _more off site backups...Recent forum and wiki downtime 2019-05-01...2019-05-03
viewtopic.php?f=18&t=22343

other.observation
Celeron55 ...says:
....."I'll be looking into implementing more frequent off-site backups to other servers ran by trusted community members."…
my.suggestion

Code: Select all

_ _ _
I suggest FEWER Backups, NOT More
_____Backups are a bitter sweet endeavor.  I do have suggestions, despite the appearance I may be taking advantage of a tragedy (but so what if I am?).
I design and compile my offline spreadsheet into phbb code for EVERY post.  Yes its time consuming.  But I still have records of conversations to refer to.
The shortcoming of “MY” system is that it fails to read chronologically.  That is, if I edit my post, it does not appear in a the order of forum conversation.  I would like to get ideas on a chronological  display of a single users single post within the context of a GROUP chat.
Losing 6 weeks of work in any personal sense is unnacceptable.  Multply that by 1000 people and that makes for a big loss, even if its “only” a forum.
I have seen complaints about “MY” system (I use “MY” to distinguish from the norm, not as ownership).  Say what you will, but if my posts are deleted, I have backups.  As it is I havent posted recently so it has minor relevance.
A single post by user to handle the context of group topic discussions de-clutters the forum and provides for a primitive means of user backup.  If fails chronological integration, but I hope something can remedy that.
I could have called it a "FLOG", the forum & blog.  But that would make this a beginners guide to advance phbb flogging!
Will call this system "Borum".  A forum and a BLog....and because it can bore 'em. (be boring)
Total system becomes the "Back_em_Up 'n' Borem" system.












my.conclusion

Code: Select all

_ _ _#t.date...20191204
I propose the initiative to promote user level backups of posted data.  Although large forum recoveries are obviously still necessary, the forums would not be “AS” dependant on mass backups.

The “Back_em_up_n_Borem” system is here for any to copy/paste/utilyze in a spreadsheet format.

I am not advocating everyone should, that would be absurd.  I am very appreciative of the work many put into the forums, it has been an indispensable resource.















***#t.post...26***
YES to security...NO to cloud backups.
my.suggestion
_ _ _YES to security...NO to cloud backups.





_ _ _my.link

_ _ _privacy...Recent forum and wiki downtime 2019-05-01...2019-05-03
_ _ _viewtopic.php?f=18&t=22343

other.observation
SorceryKid ...says:
"It is my understanding that the forum database contains private messages, hidden subforums, hashed passwords, and IP addresses. Even if fully encrypted, that's a lot of sensitive, personal information that is just begging for a privacy breach to happen. We've all seen the news of major companies and even government agencies in the line-of-fire for not properly protecting confidential data ;) In my view, nobody else except the core developers and moderators should be trusted with access to the forum database."
my.observation

Code: Select all

_ _ _I suggest FEWER Backups, NOT More
Am I speaking a foreign language?
It's as if solutions to this problem consist ONLY of more frequent backups.  It seems clear FEWER forum backups are desirable!!!


Exactly, I couldn’t agree more.  Security MUST come first and foremost before convenience and lost time.
And so the solution that is suggested includes “cloud” backups?  You have to be  X!#$%**@*  kidding me.
Losing 6 weeks of work in any personal sense is unnacceptable.  Potentially exposing user data for exploitation….do I need to ellaborate?
If individuals care about the work they post, they need to back it up THEMSELVES.  That way moderators and contributors can focus on other projects, not forum backups.
A single post by user to handle the context of group topic discussions de-clutters the forum and provides for a primitive means of user backup.  If fails chronological integration, but I hope something can remedy that.
It appears I am attempting to create a forum in which each USer has ONE post per topic.  ANd then within this ONE post, conversation is Biased to the USer.  IT does not work well considering content is hidden within Spoilers.  But Spoilers allow excessive amount of information to be minimized and avoid clutter.
Some smart people have identified the problems with backups and security.  But they fail to identify a valid solution and instead suggest MORE backups. 



















***#t.post...27***
Introducing.my.system.of.forum.posting
my.observation
_ _ _Introducing the #t.backup.#t.borem.#t. recovery system
_ _ _Or for those seeking violent alternatives, FLOG, for forum/blog.

other.observation

Code: Select all

#t.date...20191204
Its a primitive yet effective approach to create bbcode templates.
Simply copy N paste your custom template into Minetest forum for an organized Single post alternative.
I am going to attempt to QUOTE my post in a different post.  If it works as intended, that means anyone is welcome to copy and paste my post into a .xls spreadsheet.
my.process

Code: Select all

_ _ _Borum.csv.header.row1
[HIDEE] or [spoilerr],[quotee],[codee],[color=#],[I],[b],[s],Code Text,[url],[img],[list],[attachment=0],Who,Says:,TEXT,...,Reference,...,[/attachment],[/list],[/img],[/url],[/s],[/b],[/I],[/color],[/codee],[/quotee],[/HIDEE] or [/spoilerr]







Attachments
ScreanShotOfSpreadsheet_revg
ScreanShotOfSpreadsheet_revg
25.1.png (167.02 KiB) Viewed 7784 times
Last edited by RajMahal on Fri May 01, 2020 17:32, edited 32 times in total.

User avatar
Desour
Member
Posts: 1469
Joined: Thu Jun 19, 2014 19:49
GitHub: Desour
IRC: Desour
In-game: DS
Location: I'm scared that if this is too exact, I will be unable to use my keyboard.

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Desour » Post

celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.
How exactly does this work?
I am not really experienced in using google.
he/him; Codeberg; GitHub; ContentDB; public personal TODO list; "DS" is preferred (but often too short)

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Linuxdirk » Post

DS-minetest wrote:
celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.
How exactly does this work?
I am not really experienced in using google.
Just search for your mod’s forum post URL in your browsing history and copy the URL into Googles search field. You’re either lucky and it finds the post (and then click "from cache") or it does not find your post.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Post

Thanks for all your efforts celeron55. I'm relieved that you were able to recover most of the forum posts.

However, I can't seem to get the Google cache to work. I've tried several forum URLs, and they all result in this message:

Image

Yet in the search results, it shows these pages are indexed and that cached versions are available.

User avatar
Codesound
Member
Posts: 365
Joined: Thu Jun 09, 2016 14:56

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Codesound » Post

Thanks @Celeron55,

for ALL, ALL, ALL, ALL your works!!!

R

User avatar
DELTA_FORCE
Member
Posts: 165
Joined: Tue Oct 30, 2018 01:26
IRC: DELTA_FORCE
In-game: DELTA_FORCE
Location: ee

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by DELTA_FORCE » Post

Great to see that things can be fixed fast here. Good job :)
check me out nowhere because i never do anything

User avatar
Desour
Member
Posts: 1469
Joined: Thu Jun 19, 2014 19:49
GitHub: Desour
IRC: Desour
In-game: DS
Location: I'm scared that if this is too exact, I will be unable to use my keyboard.

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Desour » Post

Linuxdirk wrote:
DS-minetest wrote:
celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.
How exactly does this work?
I am not really experienced in using google.
Just search for your mod’s forum post URL in your browsing history and copy the URL into Googles search field. You’re either lucky and it finds the post (and then click "from cache") or it does not find your post.
I see, I have to use that green arrow.

However, I get the same results as sorcerykid.
the internet never forgets
Hehe, I guess that rule was broken.
he/him; Codeberg; GitHub; ContentDB; public personal TODO list; "DS" is preferred (but often too short)

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by VanessaE » Post

Yeah, neither Wayback Machine nor Google Cache have recent enough snapshots of the Forum to be useful. Bummer, that. :-/
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

WayBack Machine
New member
Posts: 6
Joined: Fri May 03, 2019 19:42

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by WayBack Machine » Post

Attachments
WaybackButtons.png
WaybackButtons.png (10.29 KiB) Viewed 7784 times

User avatar
Skamiz Kazzarch
Member
Posts: 613
Joined: Fri Mar 09, 2018 20:34
GitHub: Skamiz
In-game: Skamiz
Location: la lojbaugag.

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Skamiz Kazzarch » Post

I just realized probably why I couldn't log in. I asume the passwords were also returned?
Thank you for your hard work in making the forum run again.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Post

To aid in recovery efforts, I generated snapshots of the forum index using Bing's cache (see, Microsoft isn't so bad after all ;)

Featured General User-contributed Content Using a custom awk script, I redirected all topic URLs back to Bing search. I can't promise that Bing has cached copies of all such pages, but at least it should be easier to navigate than manually copying and pasting every topic URL into the search engine.

PS. If that doesn't work out and you are still desperate, then give Yandex.com a try. It seems to have archived a good number of recent discussions that haven't even been crawled by Google or Bing.

User avatar
voxelproof
Member
Posts: 1087
Joined: Sat Aug 05, 2017 08:13
Location: Europe

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by voxelproof » Post

It's very frustrating, but luckily at least the communication platform has been restored. Thanks for your work, guys, and for bringing the situation back under control :)
Last edited by voxelproof on Sat May 04, 2019 17:11, edited 1 time in total.
To miss the joy is to miss all. Robert Louis Stevenson

User avatar
homthack
Member
Posts: 105
Joined: Sun Apr 15, 2018 19:11
GitHub: homt8
IRC: homthack
In-game: homthack
Location: Canada
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by homthack » Post

I forgot my thing, I lost my thing that I did last April I published a statement on my server that is scheduled the next day April 26 that I did for tomorrow and then I cancel it before Saturday I'm not there but I'm I went there and saw the news that happened yesterday and then I came to that program and I went down last Sunday, I did not know what else it was that made a backup copies and for that I am happy that minetest forum has been alive step on the line that put it in a backup since March.
Server-Available: [HM-Skyblock] Contact-Discord: [homthack#9917]

User avatar
Fixer
Member
Posts: 904
Joined: Sun Jul 31, 2011 11:23
IRC: Fixer
In-game: Fixer
Location: Ukraine

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Fixer » Post

@celeron55
What's your avg backup size?

User avatar
uwu
Member
Posts: 43
Joined: Sun Jan 21, 2018 15:01

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by uwu » Post

^ I'd like to know this as well.
If it's not too outrageous, I'm willing to provide some storage space with sftp access on one of my VPS.

After this crash, we as a community should work towards increasing our Bus factor, so the Minetest project won't fail if one person suddenly disappears or another catastrophic event like this happens again. I've seen too many projects in complete disarray just because one guy is missing, e.g. Void Linux.

User avatar
Kilarin
Member
Posts: 894
Joined: Mon Mar 10, 2014 00:36
GitHub: Kilarin

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Kilarin » Post

You don't realize how much you really needed something until it's gone.
I missed the minetest forums!

I want to add my voice to the other saying thank you for all the hard work getting the forums back up and keeping them running.

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Linuxdirk » Post

Is it me or is the server noticeably slower?

User avatar
celeron55
Administrator
Posts: 532
Joined: Tue Apr 19, 2011 10:10
GitHub: celeron55
IRC: celeron55

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by celeron55 » Post

@Fixer @uwu

The backup size for the forum and wikis is currently at about 5GB.

If you think the community might trust you with an encrypted backup and you're up for setting up the receiving end (including limited ssh access, the necessary cron jobs to prune old backups on your end and whatever else is needed), send me an email. Once I know our resources, I can figure out what would make most sense.

What I do know is the backups do contain some personal information (which at least in EU matters a lot) so I need to be careful. Also, there's a certain someone who really would like to get his hands on these backups to... ehm... "update" his website and send very useful and not spammy at all emails to people's private addresses.

User avatar
TumeniNodes
Member
Posts: 2941
Joined: Fri Feb 26, 2016 19:49
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes
Location: in the dark recesses of the mind
Contact:

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by TumeniNodes » Post

celeron55 wrote:@Fixer @uwu

The backup size for the forum and wikis is currently at about 5GB.

If you think the community might trust you with an encrypted backup and you're up for setting up the receiving end (including limited ssh access, the necessary cron jobs to prune old backups on your end and whatever else is needed), send me an email. Once I know our resources, I can figure out what would make most sense.

What I do know is the backups do contain some personal information (which at least in EU matters a lot) so I need to be careful. Also, there's a certain someone who really would like to get his hands on these backups to... ehm... "update" his website and send very useful and not spammy at all emails to people's private addresses.
Yeh, it's that 2d bit which scares me more than any EU issues. At least the EU can be reasoned with.
I do not envy your position of needing to make a choice eventually.
A Wonderful World

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Post

I'm not very comfortable with random people being handed the keys to the castle (no offense Fixer, it's nothing personal :)

It is my understanding that the forum database contains private messages, hidden subforums, hashed passwords, and IP addresses. Even if fully encrypted, that's a lot of sensitive, personal information that is just begging for a privacy breach to happen. We've all seen the news of major companies and even government agencies in the line-of-fire for not properly protecting confidential data ;) In my view, nobody else except the core developers and moderators should be trusted with access to the forum database.

Why not look into a cloud hosting solution? I've been using ADrive for automated weekly backups of my server. They support all the industry standard protocols including rsync, sftp and scp. And of course under Linux, you can remote mount the filesystem via SSHFS. The 100 GB storage plan is just 25 USD for an entire year. This way you could maintain a centralized, secured backup that is always online. And only appointed staff would be give the login credentials.

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests