Minetestmapper rewritten into C++
-
- Member
- Posts: 4290
- Joined: Sun Sep 09, 2012 17:31
- GitHub: Sokomine
- IRC: Sokomine
- In-game: Sokomine
With the help of ShadowNinja and VanessaE, I modified a local copy of ShadowNinjas branch of the minetestmapper with the intention of getting a smaller file size where the buildings are easier to see. Of course that map looks far less good than the "normal" one - that is intended. My goal is to get a more abstract, much smaller map that helps players locate roads and towns.
Modifications done:
1. The colors where changed so that grass, dirt and papyrus got the grass-color. Sand, desert_sand, desert_stone, cactus and dry shrub all counts as desert_sand. Stone and gravel retain their color. All other content - possibly player-generated - is drawn in red.
2. The trees turned out to be annoying. Thus, leaves, apples, trees and torches are ignored/treated as air by the mapper.
3. The shading for everythiing but the red content is turned off (the shading indicates height changes on the map).
The result can be seen here: 1.7 MB map of Redcrabs server
It would be more like a real map if the streets where visible as such.
Modifications done:
1. The colors where changed so that grass, dirt and papyrus got the grass-color. Sand, desert_sand, desert_stone, cactus and dry shrub all counts as desert_sand. Stone and gravel retain their color. All other content - possibly player-generated - is drawn in red.
2. The trees turned out to be annoying. Thus, leaves, apples, trees and torches are ignored/treated as air by the mapper.
3. The shading for everythiing but the red content is turned off (the shading indicates height changes on the map).
The result can be seen here: 1.7 MB map of Redcrabs server
It would be more like a real map if the streets where visible as such.
A list of my mods can be found here.
-
- Member
- Posts: 1482
- Joined: Fri Apr 19, 2013 16:19
- GitHub: twoelk
- IRC: twoelk
- In-game: twoelk
- Location: northern Germany
now that looks really interesting!
I have been playing with the mapper for some while now, hoping not to have to change the code but have since come to the conclusion I can not avoid it to get the results I want.
But if somebody else is allready working on it ;-)
For one I wanted to get rid of the default color table that the program keeps defaulting to. For example if I want to make a map that only shows rails I don't know how to convince it to ignore the rest. If I delete everything else in the color.txt it takes the builtin values for grass,stone,etc. Setting them another color, such as white, blocks everything underneath.
So my sollution was either to define an ignore value such as is used for air or make the program use the values described in the external color.txt file only. (as far as I can see there is no ignore value, it is simply hardcoded to ignore air. With Paramat's Moonrealms and such it might get vital to display air in contrast to vacum/void/undefined)
The other feature I was pondering on was a variable for maximum hight. So that one could for example ignore skyblocks over a certain hight or map a certain underground level.
I have been playing with the mapper for some while now, hoping not to have to change the code but have since come to the conclusion I can not avoid it to get the results I want.
But if somebody else is allready working on it ;-)
For one I wanted to get rid of the default color table that the program keeps defaulting to. For example if I want to make a map that only shows rails I don't know how to convince it to ignore the rest. If I delete everything else in the color.txt it takes the builtin values for grass,stone,etc. Setting them another color, such as white, blocks everything underneath.
So my sollution was either to define an ignore value such as is used for air or make the program use the values described in the external color.txt file only. (as far as I can see there is no ignore value, it is simply hardcoded to ignore air. With Paramat's Moonrealms and such it might get vital to display air in contrast to vacum/void/undefined)
The other feature I was pondering on was a variable for maximum hight. So that one could for example ignore skyblocks over a certain hight or map a certain underground level.
-
- Member
- Posts: 4290
- Joined: Sun Sep 09, 2012 17:31
- GitHub: Sokomine
- IRC: Sokomine
- In-game: Sokomine
You can invert the exclusion of leaves/apples/etc and let it exclude anything that is not default:rail. That's pretty easy. I did it for a test with torches. Right now, all these changes are done to the source code and cannot be passed as parameters. But then, those are special maps anyway, and compiling is fast. Much faster than the actual map generation.twoelk wrote: For example if I want to make a map that only shows rails I don't know how to convince it to ignore the rest.
There has to be an internal distinction between air and vacuum. Try to find out the node names!twoelk wrote: With Paramat's Moonrealms and such it might get vital to display air in contrast to vacum/void/undefined)
Oh yes. That sounds very reasonable. All those odd sky plattforms which are not visible from the ground but still block up view have puzzled me a lot. And there have been several players wondering how to get to the space ships visible on the map of Redcrabs.twoelk wrote: The other feature I was pondering on was a variable for maximum hight. So that one could for example ignore skyblocks over a certain hight or map a certain underground level.
[A while later]After experimenting a bit with it, drawing for example one layer at a time (nodes at height 0-15 only, or 16-32), part of it turned out to be intresting. Plotting the areas below ground does not help much as that would require inverting what is drawn - instead of normal nodes, air would suddenly become particulary intresting. Else it's all more or less stone.
I'll attach the colors.txt file and a list of the tiny modifications to ShadowNinjas branch that might help to experiment a bit with it.
- Attachments
-
- mapper-changes.zip
- (58.98 KiB) Downloaded 126 times
Last edited by Sokomine on Fri Jan 31, 2014 20:27, edited 1 time in total.
A list of my mods can be found here.
- Krock
- Developer
- Posts: 4650
- Joined: Thu Oct 03, 2013 07:48
- GitHub: SmallJoker
- Location: Switzerland
- Contact:
*BUMP*
It seems like the mapper can't handle LevelDB-worlds, could someone add the support for it?
Thanks in advance.
It seems like the mapper can't handle LevelDB-worlds, could someone add the support for it?
Thanks in advance.
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
LazyJ wrote:When positive x and/or positive z coords are entered into the geometry option, the cpp minetestmapper generates white space from 0,0 to the specified positive coord and then it renders the area image.
twoelk wrote:This would be a cool feature indeed and usefull for many things. Slicing at a given hight would not only allow something like hight=30 to exclude platforms above that vallue but hight=-130 would give me a layout of all my tunnels at that level (and show me all caves as well of course)
LazyJ wrote:Ok, another feature that would be nice (or perhaps it's a code line that can be tweaked as needed) is to specify a height or altitude to ignore things (kind of like how minetestmapper ignores the clouds).
Sokomine wrote:3. The shading for everythiing but the red content is turned off (the shading indicates height changes on the map).
I have implemented the features that were requestedKrock wrote:It seems like the mapper can't handle LevelDB-worlds, could someone add the support for it?
Here's my fork: https://github.com/sfan5/minetest-mapper-cpp
Last edited by sfan5 on Wed Mar 05, 2014 21:02, edited 1 time in total.
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
I just made a better (automatically generateable) colors.txt for my fork.
Even though the colors.txt was automatically generated using a (pretty primitive) algoritm they still look nice.
Here's a comparison:
Even though the colors.txt was automatically generated using a (pretty primitive) algoritm they still look nice.
Here's a comparison:
I've been trying sfan5's fork of MinetestMapper C++.
The reuslts, so far, have been mixed but encouraging.
--backend sqlite3
I tried generating updates to the overview maps that I post in our Minetest.net Forums', LinuxGaming.us server thread. Our server is using the default, sqlite3 backend.
The four, 1,000x1,000 block, 'section' maps take about 2 minutes each when I have used Mireq's or, at present, VanessaE's MinetestMapper C++ fork. sfan5's version took about 8 minutes to produce each 1,000x1,000 section map.
The 'Main Section' overview map is 3,300x2,400. It usually takes about 5 minutes to render. When I tried to render it with sfan5's fork it took over 3 hours.
--backend leveldb
At first I kept getting 'Unknown map backend: leveldb' error messages.
The included README.rst file instructs to compile the minetestmapper by using:
To get the '--backend leveldb' to work I had to recompile minetestapper with:
A 1,000x1,000, overview map, of a leveldb backend world, took about 5 minutes.
A 2,000x2,000, overview map, of the same leveldb backend world, took about 16 minutes.
Hopefully some of this info will help improve future updates to the minestmapper.
Thanks to Mireq, Sean884, Sokomine, VanessaE and sfan5 for progressing this vital Minetest tool.
The reuslts, so far, have been mixed but encouraging.
--backend sqlite3
I tried generating updates to the overview maps that I post in our Minetest.net Forums', LinuxGaming.us server thread. Our server is using the default, sqlite3 backend.
The four, 1,000x1,000 block, 'section' maps take about 2 minutes each when I have used Mireq's or, at present, VanessaE's MinetestMapper C++ fork. sfan5's version took about 8 minutes to produce each 1,000x1,000 section map.
The 'Main Section' overview map is 3,300x2,400. It usually takes about 5 minutes to render. When I tried to render it with sfan5's fork it took over 3 hours.
--backend leveldb
At first I kept getting 'Unknown map backend: leveldb' error messages.
The included README.rst file instructs to compile the minetestmapper by using:
Code: Select all
cmake .
make
Code: Select all
cmake . -DENABLE_LEVELDB=1
make
A 2,000x2,000, overview map, of the same leveldb backend world, took about 16 minutes.
Hopefully some of this info will help improve future updates to the minestmapper.
Thanks to Mireq, Sean884, Sokomine, VanessaE and sfan5 for progressing this vital Minetest tool.
My blog: LazyJ's Minetest World
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
Could you try reverting https://github.com/sfan5/minetest-mappe ... 807659e056 and then trying again. my mapper should not be slower than mireq's or VanessaE's.LazyJ wrote:sfan5's version took about 8 minutes to produce each 1,000x1,000 section map.
Edit: I have found out why mireq's/VanessaE's mapper is faster:
(It does not generate the full map)
Last edited by sfan5 on Sat Mar 22, 2014 12:30, edited 1 time in total.
- Enke
- Member
- Posts: 469
- Joined: Fri Nov 15, 2013 02:56
- GitHub: NANOsoldierEnke
- IRC: Enke
- In-game: Enke
- Location: The internet
Pretty.....
*drools
*drools
Lush8
ExtraVars for Red Eclipse
<Anarchid> my turn was still the most awesome, yielding all the cripples, two captured paranormals, and death rate of about 30%
<ORCACommander> Anarchid: you need to work harder
<ORCACommander> I am hereby putting you under review until you can increase the casualty rate
ExtraVars for Red Eclipse
<Anarchid> my turn was still the most awesome, yielding all the cripples, two captured paranormals, and death rate of about 30%
<ORCACommander> Anarchid: you need to work harder
<ORCACommander> I am hereby putting you under review until you can increase the casualty rate
I added a few patches on top of sfan5's version:
Miscellaneous info:
I tested my changes with an sqlite database on a system equipped with an SSD.
I'd like to hear about the performance effects when using an HDD.
I haven't gotten around to testing using a leveldb world yet. (feel free to beat me to it :-)
I noticed the code contains two licenses (LGPL 2.1 & BSD-2), without being clear about their respective statuses. That's something to be sorted out / clarified...
- Fixed the computation of block numbers to be included in the map (off-by-one)
- Failure to open output file is now caught (instead of causing SEGV)
- Fixed --min-y and --max-y (was broken)
- Added option --verbose which:
- reports world and map statistics and coordinate limits
- reports database access statistics
- Added option --forcegeometry, which creates a map of the requested geometry, even if the world is smaller
(useful if the map is larger than minetestmapper will handle: generate the map in sections, with --forcegeometry, and glue those together) - Fixed a memory leak in the sqlite database code (which was also a minor performance issue)
- Improved database performance:
- by default, only blocks that are needed, are requested from the database now.
leveldb should benefit greatly (I expect)
as should sqlite when generating partial maps - bulk reading of one world row at a time is still an option for sqlite.
I suspect it might be faster in some cases, depending on the map and on which part of it is being rendered.
- by default, only blocks that are needed, are requested from the database now.
Miscellaneous info:
I tested my changes with an sqlite database on a system equipped with an SSD.
I'd like to hear about the performance effects when using an HDD.
I haven't gotten around to testing using a leveldb world yet. (feel free to beat me to it :-)
I noticed the code contains two licenses (LGPL 2.1 & BSD-2), without being clear about their respective statuses. That's something to be sorted out / clarified...
Last edited by Argos on Wed Mar 26, 2014 09:11, edited 1 time in total.
My mods & tools:
My fork of Minetestmapper - much improved - features, manual, Windows downloads
[MOD] Wrench (rotate:wrench) - improved screwdriver alternative - manual, download
My fork of Minetestmapper - much improved - features, manual, Windows downloads
[MOD] Wrench (rotate:wrench) - improved screwdriver alternative - manual, download
sfan5, I've been trying to use your auto generate colors, how do I use the avgcolors.py? When I run it like python avgcolors.py, it outputs this: Usage: avgcolor.py <input> So I'm presuming that there is something that I need to add after the avgcolors.py? I'm using Ubuntu. And where should the path $(find /minetest/dir -type f -name $2) in the command file lead to?
Thanks!
Thanks!
Thanks.sfan5 wrote:I have merged some of them for now, I'll some more later.
Thanks for your contribution
Please let me know if there is anything I can contribute to getting them merged swiftly and smoothly.
My mods & tools:
My fork of Minetestmapper - much improved - features, manual, Windows downloads
[MOD] Wrench (rotate:wrench) - improved screwdriver alternative - manual, download
My fork of Minetestmapper - much improved - features, manual, Windows downloads
[MOD] Wrench (rotate:wrench) - improved screwdriver alternative - manual, download
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Hello
Are there future plans to add redis backend support?
Thanks,
JPR
Are there future plans to add redis backend support?
Thanks,
JPR
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
Re: Minetestmapper rewritten into C++
I already wanted to it yesterday, but here is it:JPRuehmann wrote:Are there future plans to add redis backend support?
https://github.com/minetest/minetestmap ... be61c84336
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Works
Thanks for the quick Fix.
JPR
Thanks for the quick Fix.
JPR
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Hello
I have a problem with the C minetestmapper
I´ve installed it into /usr/local/bin chown root:root chmod rwx,r-x,r-x.
if I start it from rhe init.lua in the mod Folder theres an normal .png image in the textures Folder.
But if I try to run it from the commandline minetestmapper --backend redis -i (Worldpath) -o (png File) i only get an black image.
what i´m doing wrong?
thanks,
JPR
I have a problem with the C minetestmapper
I´ve installed it into /usr/local/bin chown root:root chmod rwx,r-x,r-x.
if I start it from rhe init.lua in the mod Folder theres an normal .png image in the textures Folder.
But if I try to run it from the commandline minetestmapper --backend redis -i (Worldpath) -o (png File) i only get an black image.
what i´m doing wrong?
thanks,
JPR
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Thanks now it works from the commandline to.
But a teeny wheeny thing there is in the examples the "" for the colors not there.
without it is not working in the help -h it is described correctly.
Thanks,
JPR
But a teeny wheeny thing there is in the examples the "" for the colors not there.
without it is not working in the help -h it is described correctly.
Thanks,
JPR
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
Re: Minetestmapper rewritten into C++
The # not working without a " is the fault of your shell, programs usually do not take this into accout in their help.JPRuehmann wrote:But a teeny wheeny thing there is in the examples the "" for the colors not there.
without it is not working in the help -h it is described correctly.
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Sorry I was to fast.
only the first try worked than all tries produced a black File with 1.4kb in Size.
This Command
minetestmapper --geometry -500:-500+1000+1000 --bgcolor '#000000' --backend redis -i /home/minetest/.minetest/worlds/World/ -o ~/Downloads/World-Map.png
Produced this Output on the Commandline.
And resulted in the File
only the first try worked than all tries produced a black File with 1.4kb in Size.
This Command
minetestmapper --geometry -500:-500+1000+1000 --bgcolor '#000000' --backend redis -i /home/minetest/.minetest/worlds/World/ -o ~/Downloads/World-Map.png
Produced this Output on the Commandline.
Code: Select all
Unknown nodes:
4seasons:desertsand_winter
4seasons:grass_autumn
4seasons:grass_spring
4seasons:ice_flowing
4seasons:ice_source
4seasons:leaves_autumn
4seasons:leaves_spring
4seasons:sand_winter
bedrock:bedrock
bedrock:glass
beer:beer_grape
beer:beer_hops
bitchange:mineninth_in_ground
default:apple
default:cactus
default:clay
default:desert_sand
default:desert_stone
default:dirt
default:dirt_with_grass
default:dry_shrub
default:grass_1
default:grass_2
default:grass_3
default:grass_4
default:grass_5
default:gravel
default:lava_flowing
default:lava_source
default:leaves
default:nyancat
default:nyancat_rainbow
default:papyrus
default:sand
default:stone
default:stone_with_coal
default:stone_with_copper
default:stone_with_gold
default:stone_with_iron
default:stone_with_mese
default:tree
default:water_flowing
default:water_source
farming:pumpkin
farming_plus:banana
farming_plus:banana_leaves
farming_plus:carrot
farming_plus:cocoa
farming_plus:cocoa_leaves
farming_plus:orange
farming_plus:potato
farming_plus:rhubarb
farming_plus:strawberry
farming_plus:tomato
fire_flowers:fireflower
flowers:dandelion_white
flowers:dandelion_yellow
flowers:geranium
flowers:rose
flowers:tulip
flowers:viola
gloopblocks:stone_mossy
glow:shrooms
glow:stone_with_worms
glowblocks:glowblockwhite_on
moreblocks:super_glow_glass
nature:blossom
smooth:dirt_stone
trail:dirt_with_grass_walked
- Attachments
-
- World-Map.png (1.2 KiB) Viewed 1285 times
-
- Moderator
- Posts: 4095
- Joined: Wed Aug 24, 2011 09:44
- GitHub: sfan5
- IRC: sfan5
- Location: Germany
Re: Minetestmapper rewritten into C++
Make sure your colors.txt file is fine, it seems like the color entries for the nodes are not there.
- JPRuehmann
- Member
- Posts: 334
- Joined: Fri Mar 21, 2014 21:40
- Location: Germany
- Contact:
Re: Minetestmapper rewritten into C++
Hello
Sorry I don´t understand what you are talking about.
What is a color.txt File, where must it located and what is its content?
At least where can I get it?
Thanks,
JPR
Sorry I don´t understand what you are talking about.
What is a color.txt File, where must it located and what is its content?
At least where can I get it?
Thanks,
JPR
Who is online
Users browsing this forum: No registered users and 5 guests