Page 1 of 1
[Tool] mtmapprune [mtmapprune] Prune your map.sqlite fast!
Posted: Mon Oct 02, 2017 20:17
by sofar
Project website:
https://github.com/minetest-tools/mtmapprune
License: LGPL-2.1+ MIT
Binaries: Linux X86_64:
https://github.com/minetest-tools/mtmapprune/releases
Latest version:
https://github.com/minetest-tools/mtmap ... x86_64.zip
Windows builds are available for some releases. Check the github releases to see which ones are.
Prunes your map.sqlite and deletes blocks outside a specified range.
Usage:
Code: Select all
mtmapprune map.sqlite max_x [max_y [max_z [min_x min_y min_z]]]
If omitted, `max_y` and `max_z` default to the value of the `max_x` limit. The `min_*` limits will default to the negative values of the `max_*` limits. The limits are node positions, not "block positions".
Example: to prune all nodes with x and z > 1000, and x and z < -1000 and y > 200, and y < 200, use:
mtmapprune map.sqlite 1000 200
mtmapprune works really fast by marking all the blocks outside the reserve region as deleted, and then vacuuming the sqlite db. This makes sqlite do all the work and removes all the empty unused space from your sqlite db file.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Mon Jan 01, 2018 06:13
by sofar
v2 adds full support of pruning to any arbitrary (xyz)-(xyz) box coordinate set, while maintaining backwards compatibility with the way that v1 uses the parameters. So if you now have a box that's in all negative coordinates, or doesn't include the origin (0,0,0), it still works.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Thu Feb 08, 2018 22:23
by Fixer
Windows build please!
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Thu Feb 08, 2018 22:36
by twoelk
ooh,
please expand this to a mt-map-database manupilating tool.
stuff like deleting certain nodes, moving map parts around or even merging maps would be cool
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Feb 09, 2018 00:32
by sofar
Fixerol wrote:Windows build please!
Completely untested, but, give this a try:
https://github.com/minetest-tools/mtmap ... -win64.zip
[edit: fixed link]
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Feb 09, 2018 00:34
by sofar
twoelk wrote:ooh,
please expand this to a mt-map-database manupilating tool.
stuff like deleting certain nodes, moving map parts around or even merging maps would be cool
Deleting specific nodes? Probably not, since this code avoids entirely looking at the content. However, moving blocks around would actually be possible. Not sure if I want to go there, though, but it's certainly possible to do just that.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Feb 09, 2018 13:57
by Fixer
>mtmapprune-v3-x86_64.zip
This one is for linux, ELF file.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Feb 09, 2018 18:05
by sofar
Fixerol wrote:>mtmapprune-v3-x86_64.zip
This one is for linux, ELF file.
whoops, fixed link for ya.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Sat Feb 10, 2018 22:37
by Fixer
Gives me error at launch when trying to prune:
Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub
Win7sp1
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Sun Feb 11, 2018 00:56
by sofar
Fixerol wrote:Gives me error at launch when trying to prune:
Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub
Win7sp1
Looks like the latest version fixes that issue as well.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Thu Aug 23, 2018 17:16
by Gael de Sailly
It doesn't work for me
(using v3 binary on Ubuntu 16.04 and Minetest 0.4.16)
The script runs, but nothing is deleted.
First time I run it:
Code: Select all
map.sqlite: removed 47368 of 47548 blocks (limits: [330, 0, -160]-[460, 50, -90])
Second time:
Code: Select all
map.sqlite: removed 0 of 180 blocks (limits: [330, 0, -160]-[460, 50, -90])
The deletion seems to be taken into account, but the file size is not reduced.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Aug 24, 2018 05:36
by sofar
Gael de Sailly wrote:It doesn't work for me
(using v3 binary on Ubuntu 16.04 and Minetest 0.4.16)
The script runs, but nothing is deleted.
First time I run it:
Code: Select all
map.sqlite: removed 47368 of 47548 blocks (limits: [330, 0, -160]-[460, 50, -90])
Second time:
Code: Select all
map.sqlite: removed 0 of 180 blocks (limits: [330, 0, -160]-[460, 50, -90])
The deletion seems to be taken into account, but the file size is not reduced.
Can you post the exact command line used in each run?
What is the size of the file before you ran it the first time, and after?
It's not a script, it's an actual binary.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Aug 24, 2018 07:30
by Gael de Sailly
sofar wrote:Can you post the exact command line used in each run?
I was in the world directory and I ran this twice:
Code: Select all
~/Téléchargements/mtmapprune-v3/mtmapprune map.sqlite 460 50 -90 330 0 -160
sofar wrote:Gael de Sailly wrote:What is the size of the file before you ran it the first time, and after?
20.5 MB before, 20.5 MB after.
The modification date is not even updated.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Aug 24, 2018 17:59
by sofar
Gael de Sailly wrote:sofar wrote:Can you post the exact command line used in each run?
I was in the world directory and I ran this twice:
Code: Select all
~/Téléchargements/mtmapprune-v3/mtmapprune map.sqlite 460 50 -90 330 0 -160
sofar wrote:Gael de Sailly wrote:What is the size of the file before you ran it the first time, and after?
20.5 MB before, 20.5 MB after.
The modification date is not even updated.
*shrug*
maybe can you upload the map.sqlite (gzip it first) to this thread, so I can figure out the issue?
Note that mtmapprune only deletes *blocks* (80x80x80!) and you are cutting down to a really small area, so it may just be that the pruning algorithm leaves a ton of blocks around - your border limits may just be causing some confusion. But, I'd like to see your map to make sure that is the case.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Fri Sep 28, 2018 05:44
by sofar
v4 posted. There is a windows build as well. Please test.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Thu Nov 28, 2019 05:34
by SiliconPenguin
Thank you so much for this tool, sofar! It took me a loooooooonnnngggg time to find it. As a matter of fact, it was your comment on someone else's post regarding resetting a world where you provided a link to this thread, yeah, that's how I found this wonderful tool.
mtmapprune worked excellently for me. It pruned a 1.6 GB map.sqlite map down to less then 200MB. It wasn't the file size that was important to me. I had added some mods that added new ores and trees, and I had also explored a very large part of the map. I wanted to "unexplore" the entire map, other than a limited space, so that the new ores and trees would spawn in in nearby regions.
mtmapprune was a little confusing to use at first because the example was too abbreviated. Also, when I use F5, I get those 3 coordinates: East/West - Height - North/South. I got a little confused because when I look at a map, I can see an X and a Y, and my Z becomes my depth. So I had to run mtmapprune twice to get the results I desired, but that was me trying to be too smart I guess. ;)
Anyway, the only thing I'm disappointed about in regards to this tool is how hard it was to find it!
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Tue Dec 10, 2019 00:34
by Grigor
Hey everybody! My first post.
I've got this small map, 2000 x 2000 / mapgen v6, which I've made into an island with a combination of sfan5's nuke mod (modified) and worldedit, working in creative mode. The question:
Is there a way to extend the edges of the map with default:water_source blocks? It'd be nice to move the horizon beyond the haze without blowing up too much more of the landscape!
- map.jpg (757.73 KiB) Viewed 1082 times
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Tue Dec 10, 2019 03:34
by sofar
Grigor wrote:Hey everybody! My first post.
I've got this small map, 2000 x 2000 / mapgen v6, which I've made into an island with a combination of sfan5's nuke mod (modified) and worldedit, working in creative mode. The question:
Is there a way to extend the edges of the map with default:water_source blocks? It'd be nice to move the horizon beyond the haze without blowing up too much more of the landscape!
map.jpg
Yes, just create a mapgen that generates default_water_source in ungenerated mapblocks, or air above 0. This is something a mapgen mod needs to do, not this program. Note: making a mapgen isn't too difficult, but outside of scope of this thread - please open a separate thread or look for existing mapgens that do something similar.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Wed Dec 11, 2019 02:02
by Grigor
Thanks sofar; does this mean I can extend the map I've got, or would I need to start from scratch? Will post this is the relevant section also.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Wed Dec 11, 2019 02:52
by sofar
Grigor wrote:Thanks sofar; does this mean I can extend the map I've got, or would I need to start from scratch? Will post this is the relevant section also.
mapgen can be changed and only ever affects *ungenerated* blocks. This means that any block you preserve with mtmapprune will also be preserved by mapgen.
Re: [Tool] mtmapprune [mtmapprune] Prune your map.sqlite fas
Posted: Sun Dec 15, 2019 00:59
by Grigor
Problem solved by editing map_meta.txt. See
viewtopic.php?f=12&t=23770&p=363389#p363389