Cash's World Server

[angelo1123_]

I am very desperate plz help me:/

Its in the "bin" folder

[Lone_Wolf]

I am very desperate plz help me:/

Its in the "bin" folder

Oh is that what he meant? xD
lol thanks angelo

[PurpleOne]

Server down. Not sure if it was on purpose or accident.

[nuu7]

[843jdc]

On purpose. I am running version 2 of my mapshrink program. Cross your fingers that it works well enough to not destroy too much.

I see that players have placed around 700 chests and/or prot blocks within the past week.

What is happening:
The only other mapshrink program that I know of is lag01's mapshrink program that was written for use on a SQL database file. It only saves map blocks within a set radius of spawn. We have buildings all over the map. I had to write a program to search the map for 'desired items' to save and to work with Redis database files. While conversion between database file types is possible, I thought it best and faster to use Redis directly.

The world map consists of 'map blocks'. Each map block contains a 16 x16 x16 block(meter) area. The program is searching for map blocks that contain protection blocks, protection logos, locked chests, and protected chests. Each of those map blocks will be saved. Map blocks in a 2 map block radius of those map blocks will also be saved. Everything else will be deleted aka destroyed >:)

For each map block that will be saved, there are 125 map blocks that might be saved. Multiply 125 by 26,429 map blocks found so far and you will see just how big that is. Each of those get checked and then saved to a list of map blocks to save.

Checks are being done to ensure that the map blocks being saved actually exist in the database and that they are not beyond the map boundaries. It is those checks that eat up most of the time.

I read in WorldEdit's tutorial that mapgen can effect areas 7 map blocks away from any particular area. My program is only saving 2 map blocks away. We will see how much damage is caused.

We might have to do some cleaning up when the server restarts because if the damage isn't too bad, we are keeping the map. Saving a 2 map block radius might not be enough but I hate to think how long the program will need to run if I have to save a 7 map block radius. The server might be down for a week :/

The program is being written with the intention of it being able to do much more than just a mapshrink. But only the mapshrink function is functional now. If the program works good enough, I will post it to github and hope that at least one person that is good at math functions, mutiple threads, etc will help develop it.

Ok. this is long enough. I gtg. I got a cat that just seriously stunk up the place. God almighty JB, will you ever learn to cover that stinky stuff with litter? /me gags and hunts for his carbon filter gasmask and air deodorizer.

[nuu7]

I see that players have placed around 700 chests and/or prot blocks within the past week.

O.O That is nuts.

Thanks for the detailed write up. Redis is interesting

[PurpleOne]

B e a u t i f u l


I have this nagging feeling that this will lead to underlying problems in the future (But what do I know). ¯\_(ツ)_/¯

[843jdc]

"I have this nagging feeling that this will lead to underlying problems in the future (But what do I know). ¯\_(ツ)_/¯

What, the colored text?

There is a bad block on the map that has existed for too long. I need to go there, see what's there. Then delete it

Mapshrink is taking a long time. Server might be down for hours more yet. Sorry.

1.5 hours for a 28 MB file. Now it is working on a 5.9 GB file. Oh boy. Hmm. Might be time for me to see what needs cleaning around the place. Might even fix the bad coax cable connector to my TV. Or remodel my bathroom and still be waiting for the program to finish haha. :(

[Beast24]

how much longer do u think server will be down 843jdc

[PurpleOne]

What, the colored text?

No this whole process.

[Lone_Wolf]

how much longer do u think server will be down 843jdc

Judging from his comments: A little less than a week?

[843jdc]

OMG. 15 hours and the program is still running! Shesh. There has got to be a way to speed this up. Out of 18 million map blocks in the database file, only about 1 million needs to be saved. 5.5 % or so. There has got to be a way to speed this up.

The program creates a list of map blocks to be saved. It adds to the list every time it finds a map block to add. I think the slowdown is caused as that list gets longer and longer. Because it checks to be sure that the number is not already on the list. Maybe if I make the program work in chunks of maybe 100 map blocks each and restarts the list from scratch each time.

Though doing it that way will cause any particular map block to be saved multiple times, the database is in RAM. Writes take a fraction of a second. Negligible if done in chunks.

While I rewrite the program, I am leaving it running. It doesn't provide constant updates of what it is doing. It hasn't written anything to the screen in about 14 hours. It could be locked up, somewhere in the middle of finding map blocks to save, or a few minutes from being finished. If the latter and I shut it down, that's a lot of hours wasted for nothing.

[Beast24]

how much longer do u think server will be down 843jdc

Judging from his comments: A little less than a week?

it would take a week only if he ran 7 map block area but he is doing 2 map block area so it should only take few hours

[ManElevation]

how much longer do u think server will be down 843jdc

Judging from his comments: A little less than a week?

it would take a week only if he ran 7 map block area but he is doing 2 map block area so it should only take few hours

what hapened to it?

[Lone_Wolf]

how much longer do u think server will be down 843jdc

Judging from his comments: A little less than a week?

it would take a week only if he ran 7 map block area but he is doing 2 map block area so it should only take few hours

OMG. 15 hours and the program is still running! Shesh.

Nothing usually ever happens the way you want it to go lol

[Beast24]

Now it has been over 24 hours how are things going hope server will be up soon

[843jdc]

Good news: I found where the program comes to nearly a complete halt.
Bad news: Server won't be up for several more hours yet.

I need to try a different method of saving the map blocks that what I was using. I'm pretty sure that will work. I just got to write the code to do that. Which will be sometime after I figure out how :) Redis has an interesting ability: I can have more than one database in the same database file. I'll read from one, create and write to another, transparently switch them, delete the original, do a save during server shutdown and it's done. Piece of cake. Nothing to it. /me knocks on wood. :)

[843jdc]

Strange. Using a small database file, after doing a search, I ended up with a larger database file :/ Though there is a lot of the word 'None' in the file that is not in the original. IDK what they are for.

Eh. As long as the main world map is reduced some, I'll win. I won't know that for at least an hour. Mapshrink started and I'm not sitting here waiting for it. Good night.

PS
I'm taking bets if the mapshrunked map ends up being in the terabyte range when I wake up. Any takers? xD

[ManElevation]

Strange. Using a small database file, after doing a search, I ended up with a larger database file :/ Though there is a lot of the word 'None' in the file that is not in the original. IDK what they are for.

Eh. As long as the main world map is reduced some, I'll win. I won't know that for at least an hour. Mapshrink started and I'm not sitting here waiting for it. Good night.

PS
I'm taking bets if the mapshrunked map ends up being in the terabyte range when I wake up. Any takers? xD

Good Luck!

[Beast24]

holy shit 843jdc is it bigger than a terabyte already that is 1,024 GB that is quit a bit

[843jdc]

I lost my bet. The map was down to about 240 MB. But the map was messed up by mapgen as I figured it would be. Program runtime of about 1.5 hours.

Trying again. This time with a block save radius of 7. That should keep mapgen away from protected areas. I'm not sure if the program can handle that large number of blocks. I'll know in about an hour.

[ExeterDad]

Wow 843jdc ! You is crazy! lol
Good luck, and hope you win.

[843jdc]

Bah. In the new map version, spawn was almost totally destroyed and hanging in the air far above newly generated terrain. 2.5 hours wasted. I think it is a problem with reading the data from the original map file.

[angelo1123_]

Just reset the map,but can I build the spawn ,I will redownload minetest just for that xD

[sorcerykid]

843jdc, why not ask lag for some pointers? He could probably even modify the script for your needs. Honestly, if knew enough about Python, I'd give it a shot. But my area of expertise is Perl and JavaScript. So can't really offer much help, I'm afraid :)