Recent forum and wiki downtime 2019-05-01...2019-05-03

celeron55
Administrator
 
Posts: 452
Joined: Tue Apr 19, 2011 10:10
GitHub: celeron55
IRC: celeron55

Recent forum and wiki downtime 2019-05-01...2019-05-03

by celeron55 » Fri May 03, 2019 11:09

The forum and wikis were down from 2019-05-01 to 2019-05-03 due to a VPS crash and disk corruption.

During downtime I restored the forum and the wikis to my latest off-site backup dated 2019-03-13 and upgraded the base system to OpenBSD 6.3. Much of the data left on the server was unusable, like the forum database.

Unfortunately that means the latest 6 weeks of data was lost. I'll be looking into implementing more frequent off-site backups to other servers ran by trusted community members.

ContentDB has recorded the missing mod and game titles and the list can be found here: https://gist.github.com/rubenwardy/1cd6 ... 1ffdcbdc4d

I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it. Feel free to re-post any of your lost content.
 

User avatar
Joseph16
Member
 
Posts: 308
Joined: Tue Dec 06, 2016 05:35
In-game: Joseph16
 

User avatar
Wuzzy
Member
 
Posts: 3610
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Wuzzy » Fri May 03, 2019 12:47

Ouch. That hurts. But it could have been worse. Thankfully, there was a backup. Phew. :-)

Obviously, some of my forum threads for my countless mods became outdated as well. So I had to manually restore them to point to the latest mod version.
@modders: If you released a new version in the past 6 months or so, this means you must go through your mod and game forum threads and re-update them to the latest version.
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

User avatar
Festus1965
Member
 
Posts: 880
Joined: Sun Jan 03, 2016 11:58
Location: Thailand - Chiang Mai (+5-6h to MEZ)
In-game: Thomas Explorer

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Festus1965 » Fri May 03, 2019 13:23

I had much more time without the Forum to play ... I am not so sad.

add:
Maybe a good warning about what is sure not needed, off topic, video, music and other threads making a lot of work - bad for the forum itself.
The less the better, easier, less mod work ... also look forward to new rules of ... EU
Last edited by Festus1965 on Fri May 03, 2019 22:44, edited 1 time in total.
Server: still up, see some gamer - but there is a new 1st important thread "climate change" for me. Something I know since 1992 - had seen the melting glacier in Switzerland.
 

User avatar
b3u
Member
 
Posts: 12
Joined: Mon Jun 25, 2018 01:41
Location: Maryland, USA
GitHub: b3u
IRC: Fairion
 

User avatar
RajMahal
New member
 
Posts: 7
Joined: Wed Jan 16, 2019 20:56

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by RajMahal » Fri May 03, 2019 14:47

___ Thank You for the forum recovery. I suggest FEWER backups, NOT more.

*in-process*
+ Spoiler
Last edited by RajMahal on Mon Aug 12, 2019 20:34, edited 24 times in total.
 

User avatar
DS-minetest
Member
 
Posts: 1112
Joined: Thu Jun 19, 2014 19:49
Location: I'm scared that if this is too exact, I will be unable to use my keyboard.
GitHub: DS-Minetest
In-game: DS

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by DS-minetest » Fri May 03, 2019 14:48

celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.

How exactly does this work?
I am not really experienced in using google.
Do not call me -minetest.
Call me DS or DS-minetest.
I am German, so you don't have to pm me English if you are also German.
The background is a lie.
 

User avatar
Linuxdirk
Member
 
Posts: 2177
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Linuxdirk » Fri May 03, 2019 15:06

DS-minetest wrote:
celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.

How exactly does this work?
I am not really experienced in using google.

Just search for your mod’s forum post URL in your browsing history and copy the URL into Googles search field. You’re either lucky and it finds the post (and then click "from cache") or it does not find your post.
 

User avatar
sorcerykid
Member
 
Posts: 1061
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Fri May 03, 2019 15:10

Thanks for all your efforts celeron55. I'm relieved that you were able to recover most of the forum posts.

However, I can't seem to get the Google cache to work. I've tried several forum URLs, and they all result in this message:

Image

Yet in the search results, it shows these pages are indexed and that cached versions are available.
 

User avatar
Codesound
Member
 
Posts: 221
Joined: Thu Jun 09, 2016 14:56
 

DELTA_FORCE
Member
 
Posts: 131
Joined: Tue Oct 30, 2018 01:26
Location: Somewhere in the Atlantic
IRC: DELTA_FORCE
In-game: DELTA_FORCE

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by DELTA_FORCE » Fri May 03, 2019 17:30

Great to see that things can be fixed fast here. Good job :)
If you don't like something I say, you are too easily offended...
 

User avatar
DS-minetest
Member
 
Posts: 1112
Joined: Thu Jun 19, 2014 19:49
Location: I'm scared that if this is too exact, I will be unable to use my keyboard.
GitHub: DS-Minetest
In-game: DS

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by DS-minetest » Fri May 03, 2019 17:37

Linuxdirk wrote:
DS-minetest wrote:
celeron55 wrote:I recommend searching your lost forum posts by their title on Google and using Google's cache to recover the content, and to do this in time before they delete it.

How exactly does this work?
I am not really experienced in using google.

Just search for your mod’s forum post URL in your browsing history and copy the URL into Googles search field. You’re either lucky and it finds the post (and then click "from cache") or it does not find your post.

I see, I have to use that green arrow.

However, I get the same results as sorcerykid.

the internet never forgets

Hehe, I guess that rule was broken.
Do not call me -minetest.
Call me DS or DS-minetest.
I am German, so you don't have to pm me English if you are also German.
The background is a lie.
 

User avatar
VanessaE
Moderator
 
Posts: 4462
Joined: Sun Apr 01, 2012 12:38
Location: Waynesville, NC
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by VanessaE » Fri May 03, 2019 18:09

Yeah, neither Wayback Machine nor Google Cache have recent enough snapshots of the Forum to be useful. Bummer, that. :-/
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)
 

WayBack Machine
New member
 
Posts: 6
Joined: Fri May 03, 2019 19:42
 

Skamiz Kazzarch
Member
 
Posts: 80
Joined: Fri Mar 09, 2018 20:34
Location: la lojbaugag.
GitHub: Skamiz
In-game: Skamiz

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Skamiz Kazzarch » Fri May 03, 2019 21:16

I just realized probably why I couldn't log in. I asume the passwords were also returned?
Thank you for your hard work in making the forum run again.
 

User avatar
sorcerykid
Member
 
Posts: 1061
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Fri May 03, 2019 22:42

To aid in recovery efforts, I generated snapshots of the forum index using Bing's cache (see, Microsoft isn't so bad after all ;)

Featured
General
User-contributed Content

Using a custom awk script, I redirected all topic URLs back to Bing search. I can't promise that Bing has cached copies of all such pages, but at least it should be easier to navigate than manually copying and pasting every topic URL into the search engine.

PS. If that doesn't work out and you are still desperate, then give Yandex.com a try. It seems to have archived a good number of recent discussions that haven't even been crawled by Google or Bing.
 

User avatar
voxelproof
Member
 
Posts: 887
Joined: Sat Aug 05, 2017 08:13
Location: Europe

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by voxelproof » Sat May 04, 2019 10:54

It's very frustrating, but luckily at least the communication platform has been restored. Thanks for your work, guys, and for bringing the situation back under control :)
Last edited by voxelproof on Sat May 04, 2019 17:11, edited 1 time in total.
Game is the essence of life; therefore every good game makes life better.
 

User avatar
homthack
Member
 
Posts: 34
Joined: Sun Apr 15, 2018 19:11
Location: Caracas
GitHub: homt8
IRC: homthack
In-game: homthack

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by homthack » Sat May 04, 2019 14:43

I forgot my thing, I lost my thing that I did last April I published a statement on my server that is scheduled the next day April 26 that I did for tomorrow and then I cancel it before Saturday I'm not there but I'm I went there and saw the news that happened yesterday and then I came to that program and I went down last Sunday, I did not know what else it was that made a backup copies and for that I am happy that minetest forum has been alive step on the line that put it in a backup since March.
[Server-Online] HM-Server [Discord] homthack#9917
[Website-Account]
Discord HM-Server MineCity MinetestHosting Trello
 

User avatar
Fixer
Member
 
Posts: 890
Joined: Sun Jul 31, 2011 11:23
Location: Ukraine
IRC: Fixer
In-game: Fixer
 

User avatar
uwu
Member
 
Posts: 24
Joined: Sun Jan 21, 2018 15:01

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by uwu » Sun May 05, 2019 11:13

^ I'd like to know this as well.
If it's not too outrageous, I'm willing to provide some storage space with sftp access on one of my VPS.

After this crash, we as a community should work towards increasing our Bus factor, so the Minetest project won't fail if one person suddenly disappears or another catastrophic event like this happens again. I've seen too many projects in complete disarray just because one guy is missing, e.g. Void Linux.
 

Kilarin
Member
 
Posts: 774
Joined: Mon Mar 10, 2014 00:36

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by Kilarin » Mon May 06, 2019 00:56

You don't realize how much you really needed something until it's gone.
I missed the minetest forums!

I want to add my voice to the other saying thank you for all the hard work getting the forums back up and keeping them running.
 

User avatar
Linuxdirk
Member
 
Posts: 2177
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk
 

celeron55
Administrator
 
Posts: 452
Joined: Tue Apr 19, 2011 10:10
GitHub: celeron55
IRC: celeron55

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by celeron55 » Mon May 06, 2019 16:30

@Fixer @uwu

The backup size for the forum and wikis is currently at about 5GB.

If you think the community might trust you with an encrypted backup and you're up for setting up the receiving end (including limited ssh access, the necessary cron jobs to prune old backups on your end and whatever else is needed), send me an email. Once I know our resources, I can figure out what would make most sense.

What I do know is the backups do contain some personal information (which at least in EU matters a lot) so I need to be careful. Also, there's a certain someone who really would like to get his hands on these backups to... ehm... "update" his website and send very useful and not spammy at all emails to people's private addresses.
 

User avatar
TumeniNodes
Member
 
Posts: 2825
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by TumeniNodes » Mon May 06, 2019 16:48

celeron55 wrote:@Fixer @uwu

The backup size for the forum and wikis is currently at about 5GB.

If you think the community might trust you with an encrypted backup and you're up for setting up the receiving end (including limited ssh access, the necessary cron jobs to prune old backups on your end and whatever else is needed), send me an email. Once I know our resources, I can figure out what would make most sense.

What I do know is the backups do contain some personal information (which at least in EU matters a lot) so I need to be careful. Also, there's a certain someone who really would like to get his hands on these backups to... ehm... "update" his website and send very useful and not spammy at all emails to people's private addresses.


Yeh, it's that 2d bit which scares me more than any EU issues. At least the EU can be reasoned with.
I do not envy your position of needing to make a choice eventually.
Ich mag keine grünen Eier und Schinken, ich mag sie nicht Sam I Am
 

User avatar
sorcerykid
Member
 
Posts: 1061
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Re: Recent forum and wiki downtime 2019-05-01...2019-05-03

by sorcerykid » Mon May 06, 2019 17:44

I'm not very comfortable with random people being handed the keys to the castle (no offense Fixer, it's nothing personal :)

It is my understanding that the forum database contains private messages, hidden subforums, hashed passwords, and IP addresses. Even if fully encrypted, that's a lot of sensitive, personal information that is just begging for a privacy breach to happen. We've all seen the news of major companies and even government agencies in the line-of-fire for not properly protecting confidential data ;) In my view, nobody else except the core developers and moderators should be trusted with access to the forum database.

Why not look into a cloud hosting solution? I've been using ADrive for automated weekly backups of my server. They support all the industry standard protocols including rsync, sftp and scp. And of course under Linux, you can remote mount the filesystem via SSHFS. The 100 GB storage plan is just 25 USD for an entire year. This way you could maintain a centralized, secured backup that is always online. And only appointed staff would be give the login credentials.
 

Next

Return to News



Who is online

Users browsing this forum: JTE and 2 guests