Is it possible to reduce the size of an existing map? [SOLVED]

Post Reply
User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Is it possible to reduce the size of an existing map? [SOLVED]

by Mr. Rar » Post

I have a few old Minetest maps that are 100+ MB in size.
My objective is to reduce the file size of these maps by erasing generated parts of the map that I don't care about. There are some structures near spawn that I want to keep. Everything else can go.
Last edited by Mr. Rar on Thu Nov 03, 2022 14:21, edited 1 time in total.
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
littlePrettyCat
Member
Posts: 19
Joined: Fri Oct 28, 2022 15:15
GitHub: alwayshopeless
In-game: littlePrettyCat
Location: Ukraine

Re: Is it possible to reduce the size of an existing map?

by littlePrettyCat » Post

viewtopic.php?t=8682
Duplicate. WorldEdit can help you
cdb_b870072d9f7f

User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Re: Is it possible to reduce the size of an existing map?

by Mr. Rar » Post

littlePrettyCat wrote:
Mon Oct 31, 2022 18:32
viewtopic.php?t=8682
Duplicate. WorldEdit can help you
Indeed that might work. However I am hoping to preserve the original seed and the relative locations of everything in the map. This would be a big hassle to do in WE. I wonder if there is a way to do this using direct database manipulation.
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
littlePrettyCat
Member
Posts: 19
Joined: Fri Oct 28, 2022 15:15
GitHub: alwayshopeless
In-game: littlePrettyCat
Location: Ukraine

Re: Is it possible to reduce the size of an existing map?

by littlePrettyCat » Post

Mr. Rar wrote:
Mon Oct 31, 2022 20:03
littlePrettyCat wrote:
Mon Oct 31, 2022 18:32
viewtopic.php?t=8682
Duplicate. WorldEdit can help you
Indeed that might work. However I am hoping to preserve the original seed and the relative locations of everything in the map. This would be a big hassle to do in WE. I wonder if there is a way to do this using direct database manipulation.
Hmm, I'm not sure, but it seems to me that this API could help determine the places of user activity, to make markup like "XYZ Start / XYZ End", and then export these parts to a new LUA map with a script through WorldEdit.
viewtopic.php?t=23891
cdb_b870072d9f7f

wsor4035
Member
Posts: 182
Joined: Sun Aug 11, 2019 21:23
GitHub: wsor4035
IRC: wsor
In-game: wsor

Re: Is it possible to reduce the size of an existing map?

by wsor4035 » Post

see https://github.com/BuckarooBanzay/mapcleaner/ buckaroo makes some good server/map related tools
j5uBLfc6NxgersvVj5D5dIsiKDkoQb0o

User avatar
BuckarooBanzay
Member
Posts: 435
Joined: Tue Apr 24, 2018 05:58
GitHub: BuckarooBanzay
IRC: BuckarooBanzai
In-game: BuckarooBanzai

Re: Is it possible to reduce the size of an existing map?

by BuckarooBanzay » Post

wsor4035 wrote:
Mon Oct 31, 2022 22:11
see https://github.com/BuckarooBanzay/mapcleaner/ buckaroo makes some good server/map related tools
Haven't used/updated that one in a while, not sure if the dependencies still work, use it on a backup-map first
Mr. Rar wrote:
Mon Oct 31, 2022 17:45
I have a few old Minetest maps that are 100+ MB in size.
My objective is to reduce the file size of these maps by erasing generated parts of the map that I don't care about. There are some structures near spawn that I want to keep. Everything else can go.
You best bet is still (as others said) to copy the area to a newly generated map,
you can save your positions if you are using /teleport x,y,z in combination with worldedit //pos1 and write the commands down.

If the area is too big for WE you can use this: https://github.com/blockexchange/blocke ... md#offline
¯\_(ツ)_/¯ Not active here anymore, contact me on the minetest discord, irc, lemmy or github (for programming issues)

ShadMOrdre
Member
Posts: 1118
Joined: Mon Dec 29, 2014 08:07
Location: USA

Re: Is it possible to reduce the size of an existing map?

by ShadMOrdre » Post

You can safely preserve the map_meta.txt file and the world.mt file from the world folder. You can create a new world folder with a new name, only needing to update the world.mt file to use the new "world name" which is the world folder name.

Worldedit can help you get the buildings into schematics. Save their coordinates in a text file somewhere.

If you are willing to regenerate the world, you can safely import the schematics back where they came from.


Shad

jackmortain
New member
Posts: 1
Joined: Thu Nov 03, 2022 07:26

Re: Is it possible to reduce the size of an existing map?

by jackmortain » Post

Mr. Rar wrote:
Mon Oct 31, 2022 17:45
I have a few old Minetest maps that are 100+ MB in size.
My objective is to reduce the file size of these maps by erasing generated parts of the map that I don't care about. There are some structures near spawn that I want to keep. Everything else can go.
May be you reduce your Maps file Size by a file size compressor software, but you can't reduce existing maps size.
Jack.Mortain

User avatar
TenPlus1
Member
Posts: 3714
Joined: Mon Jul 29, 2013 13:38
In-game: TenPlus1
Contact:

Idea

by TenPlus1 » Post

I still think that Minetest itself should contain a way to backup or compress map files within the client. This would be handy for servers where the map files are growing huge, as well as personal world backups.

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: Is it possible to reduce the size of an existing map?

by Blockhead » Post

Mapblocks are already compressed with zstd (previously by zlib), and the database can't be compressed less granularly than that without putting the whole file in an archive, which would probably have minimal gains. It might help to have a flag about whether a mapblock has ever been modified though, to make selecting candidates for cleanup tools easier.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Re: Idea

by Mr. Rar » Post

I was able to solve my problem with mapeditr: https://github.com/random-geek/MapEditr

mapeditr [path to map.sqlite file] deleteblocks --p1 x1 y1 z1 --p2 x2 y2 z2 --invert
mapeditr [path to map.sqlite file] vacuum

My world went from 800MB to 30MB!

I had an idea to create a mod that would allow selecting an area of the map and then produce a database quarey. Then you can simple open the map database in https://sqlitebrowser.org/ and run the query. This is unnecessary because of mapeditr.

Thank you everyone for all your suggestions!
TenPlus1 wrote:
Thu Nov 03, 2022 12:19
I still think that Minetest itself should contain a way to backup or compress map files within the client. This would be handy for servers where the map files are growing huge, as well as personal world backups.
I wish Minetest supported limiting the size of worlds and pruning existing worlds.
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
Mantar
Member
Posts: 584
Joined: Thu Oct 05, 2017 18:46
Contact:

Re: Is it possible to reduce the size of an existing map? [SOLVED]

by Mantar » Post

A tool that selectively reran mapgen and compared its results with existing mapblocks, and removed them if they were unchanged would probably be useful. But I'm not about to write it. :)
Lead dev of Exile, git repo: https://codeberg.org/Mantar/Exile

animaux
New member
Posts: 6
Joined: Sat Feb 19, 2022 10:50
GitHub: animaux

Re: Idea

by animaux » Post

Mr. Rar wrote:
Thu Nov 03, 2022 14:20
I was able to solve my problem with mapeditr: https://github.com/random-geek/MapEditr

mapeditr [path to map.sqlite file] deleteblocks --p1 x1 y1 z1 --p2 x2 y2 z2 --invert
mapeditr [path to map.sqlite file] vacuum
That was very useful and worked perfectly, many thanks!

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest