Is it safe to delete individual mapblocks?

User avatar
sorcerykid
Member
 
Posts: 1182
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Is it safe to delete individual mapblocks?

by sorcerykid » Fri Jan 10, 2020 16:38

I'm working on a database maintenance command line tool for Minetest worlds, and I was wondering if it's safe to delete individual mapblocks from the database? Or would that adversely affect the mapgens, since those work on entire mapchunks?
 

Sokomine
Member
 
Posts: 3956
Joined: Sun Sep 09, 2012 17:31
GitHub: Sokomine

Re: Is it safe to delete individual mapblocks?

by Sokomine » Fri Jan 10, 2020 20:46

I'm sure paramat will be able to tell you more about it. To my knowledge, deleting individual mapblocks (those 16x16x16 nodes things) in the database will lead to mapgen beeing called for the mapchunk again. There is no way to generate only one mapblock at a time. Even if you'd set chunksize to 1, the outer shell (1 mapblock wide in each direction) would still be affected and receive new caves and mudflow.

I still wish for the database structure to be extended so that date of creation, date of last change and perhaps sum total of nodes placed and removed by players could be stored in the database itself. That'd help a lot with backups and reducing disk space of long-running servers.
A list of my mods can be found here.
 

sofar
Developer
 
Posts: 2112
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar
 

User avatar
sorcerykid
Member
 
Posts: 1182
Joined: Fri Aug 26, 2016 15:36
Location: Illinois, USA
GitHub: sorcerykid
In-game: Nemo

Re: Is it safe to delete individual mapblocks?

by sorcerykid » Fri Jan 10, 2020 23:44

Sokomine wrote:I'm sure paramat will be able to tell you more about it. To my knowledge, deleting individual mapblocks (those 16x16x16 nodes things) in the database will lead to mapgen beeing called for the mapchunk again. There is no way to generate only one mapblock at a time. Even if you'd set chunksize to 1, the outer shell (1 mapblock wide in each direction) would still be affected and receive new caves and mudflow.

I still wish for the database structure to be extended so that date of creation, date of last change and perhaps sum total of nodes placed and removed by players could be stored in the database itself. That'd help a lot with backups and reducing disk space of long-running servers.


That is actually the impetus for the question :) I've been working on a mod that computes the checksums of newly generated mapblocks and exports the results to a text file. These checksums can then later be compared with an offline copy of theh map database to determine which mapblocks are untouched from their original state.

I suspect that the majority of generated mapblocks on large servers are still fresh (esp. in underground regions where players often dig linearly and likewise there are no plant growth ABMs, flowing water, etc.). So in theory this could significantly reduce the size of map.sqlite files on large servers, once such mapblocks can be readily identified.

Entirely safe. `mtmapprune` does this exactly.


Sorry, I should have clarified that I'm hoping to know whether deleting a single mapblock will result in either
  1. all mapblocks in the same mapchunk being regenerated once the player re-enters the area,
  2. only the deleted mapblock being regenerated once the player re-enters the area, or
  3. no regeneration of any deleted mapblocks within partially complete mapchunks
This is a huge concern, since there is potential for unanticipated griefing if the mapgen is allowed to go wild recreating caves and rivers and lakes and trees through previously built areas only because one nearby mapblock was deleted. I suspect this is what caused so many builds to destroyed on the just test server after lag ran his "map unexplore" script. I really want to avoid such a disastrous outcome.

https://forum.minetest.net/viewtopic.php?p=297261#p297261

My best guess, based on Sokomine's response and my rudimentary understanding of the mapgen, is the only "safe" way to delete mapblocks is within a uniform 5x5x5 grid.
 

User avatar
paramat
Developer
 
Posts: 3561
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat

Re: Is it safe to delete individual mapblocks?

by paramat » Sat Jan 11, 2020 01:51

> To my knowledge, deleting individual mapblocks (those 16x16x16 nodes things) in the database will lead to mapgen beeing called for the mapchunk again. There is no way to generate only one mapblock at a time. Even if you'd set chunksize to 1, the outer shell (1 mapblock wide in each direction) would still be affected and receive new caves and mudflow.

Correct as far as i know.
So for default mapchunk size (5 mapblocks), the 5x5x5 mapchunk will be regenerated, but also, cave and dungeon generation, and decorations, can extend out by another mapblock into the mapchunk 'shell', so 7x7x7 mapblocks can be affected.
 

Sokomine
Member
 
Posts: 3956
Joined: Sun Sep 09, 2012 17:31
GitHub: Sokomine

Re: Is it safe to delete individual mapblocks?

by Sokomine » Sat Jan 11, 2020 15:51

sorcerykid wrote:That is actually the impetus for the question :) I've been working on a mod that computes the checksums of newly generated mapblocks and exports the results to a text file. These checksums can then later be compared with an offline copy of theh map database to determine which mapblocks are untouched from their original state.

I wonder if this can't be done on database level. Perhaps not with sqlite...but other databases have triggers and the like. A trigger on insert/update of mapblock entries would notice new blocks beeing created and could at least store creation time easily. Changes could also be detected - though only on the level of "the data is diffrent this time". I'm not sure if the change detection would be fast enough.

sorcerykid wrote:I suspect that the majority of generated mapblocks on large servers are still fresh (esp. in underground regions where players often dig linearly and likewise there are no plant growth ABMs, flowing water, etc.). So in theory this could significantly reduce the size of map.sqlite files on large servers, once such mapblocks can be readily identified.

I suspect the same. Just think about the ~150 mb I saved from Redcrab's map - and that included cities and far-away structures built by many players over many years! A lot of mapblocks are created while players look for ressources and most likely never re-visited.
A list of my mods can be found here.
 

User avatar
BuckarooBanzay
Member
 
Posts: 353
Joined: Tue Apr 24, 2018 05:58
GitHub: BuckarooBanzay
IRC: BuckarooBanzai
In-game: BuckarooBanzai

Re: Is it safe to delete individual mapblocks?

by BuckarooBanzay » Tue Jan 14, 2020 18:25

fyi: i implemented a pure-lua (mod) solution for map cleaning that does exactly that:
https://github.com/thomasrudin-mt/mapcleaner

This checks every map chunk and its neighbours for protected nodes and players in the area.
If the chunk and the surroundings aren't protected then all the mapblocks (5^3) are removed from that chunk.

I'm currently testing this on a copy of my main server:
https://monitoring.minetest.land/d/j5hR ... dorabox.io

The upside of the lua-solution:
  • Database independent
  • No downtime, can be run while playing

I managed to reduce the total mapblock count from around 95 million mapblocks to about 30 million, which spares a few valuable gigabytes of disk-space :)
 


Return to General Discussion



Who is online

Users browsing this forum: No registered users and 18 guests