numpy-izing minetestmapper

spillz
Member
Posts: 138
Joined: Thu Feb 13, 2014 05:11

Re: numpy-izing minetestmapper

by spillz » Post

Try the latest version. Should work with both python 2 and 3. Thanks for reporting the bugs.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

I made a quick mod for displaying a minetestmapper png in-game. I've put it as a new post to avoid hijacking the numpy post, but it relies on your minetestmapper script, so thanks. viewtopic.php?f=9&t=9219

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

Is there any way of mapping the generated png to minetest world coordinates - ideally from within lua, but if not from the script. This latter could perhaps be acheived having an argument to modify the output name to include location of topL and bottomR coordinates in the world (if that's accessible), and perhaps automatically save the world data in the Comment field of the metadata within the png so it's always available even with name changes. I'm trying to figure out how to place a player location icon correctly on the map while playing the game (for my mapit tool). Cheers

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

I've modified the .py slightly to generate a couple of tEXt Textual data chucks to contain the desired output (I think, I'm not entirely happy with my understanding of how the regions are process, but have made a best guess). Given that I'm not a programmer, it needs some tidying up. I haven't forked your github, as it seemed pointless creating a 40MB fork for a 40kb file. The full edited version (with more debugging code) is on my repo: https://github.com/dgm3333/mapit.
The following code replaces the im.save just before main

Code: Select all

    # region is incorrect if world limits are exceeded. I attempted to correct this, but it didn't work
    # as this code generates incorrect values if the requested region is greater than the world dimensions
    #pngminx = int(args.region[0]/16)*16 if args.region[0] < minx*16 else minx*16
    #pngmaxx = int((args.region[1]+16)/16)*16 if args.region[1] > maxx*16 else maxx*16
    #pngminz = int(args.region[2]/16)*16 if args.region[2] < minz*16 else minz*16
    #pngmaxz = int((args.region[3]+16)/16)*16 if args.region[3] > maxz*16 else maxz*16

    # worldlimits are measured in cubes of 16x16x16
    pngminx = minx*16
    pngmaxx = maxx*16
    pngminz = minz*16
    pngmaxz = maxz*16
    pngregion=[pngminx, pngmaxx, pngminz, pngmaxz]

    print("Saving to: "+ args.output)
    print("PNG Region: ", pngregion)
    print("Pixels PerNode: ", args.pixelspernode)
    print("border: ", border)

    # This saves data in tEXt chunks (non-standard naming tags are allowed according to the PNG specification)
    im.info["pngRegion"] = str(pngregion[0])+ ","+ str(pngregion[1])+ ","+ str(pngregion[2])+ ","+ str(pngregion[3])
    im.info["pngMinX"] = str(pngminx)
    im.info["pngMaxZ"] = str(pngmaxz)
    im.info["border"] = str(border)
    im.info["pixPerNode"] = str(args.pixelspernode)
    pngsave(im, args.output)

    # Now create a square 'thumbnail' for display on square faces (which turns out to benefit from quite high resolution).
    thumbSize = 512
    imSize = im.size
    if imSize[0] > imSize[1]:
      reSize=(thumbSize, int(thumbSize*(int(imSize[1])/imSize[0])))
    else:
      reSize=(int(thumbSize*(float(imSize[0])/imSize[1])), thumbSize)

    thumbBorder=((thumbSize-reSize[0])/2, (thumbSize-reSize[1])/2, thumbSize-(thumbSize-reSize[0])/2, thumbSize-(thumbSize-reSize[1])/2)
    thumbIm = Image.new("RGB", (thumbSize,thumbSize), args.bgcolor)
    thumbIm.paste(im.resize(reSize),thumbBorder)
    thumbIm.save(args.output.replace(".png", "_thumb.png"), "PNG")

#                                                                                                                                      
# wrapper around PIL 1.1.6 Image.save to preserve PNG metadata
#
# public domain, Nick Galbreath                                                                                                        
# http://blog.client9.com/2007/08/28/python-pil-and-png-metadata-take-2.html                                                                 
#                                                                                                                                       
def pngsave(im, file):
    # these can be automatically added to Image.info dict                                                                              
    # they are not user-added metadata
    reserved = ('interlace', 'gamma', 'dpi', 'transparency', 'aspect')

    # undocumented class
    from PIL import PngImagePlugin
    meta = PngImagePlugin.PngInfo()

    # copy metadata into new object
    for k,v in im.info.iteritems():
        if k in reserved: continue
        meta.add_text(k, v, 0)

    # and save
    im.save(file, "PNG", pnginfo=meta)
Last edited by dgm5555 on Tue May 06, 2014 20:39, edited 3 times in total.

spillz
Member
Posts: 138
Joined: Thu Feb 13, 2014 05:11

Re: numpy-izing minetestmapper

by spillz » Post

@dgm5555: that looks useful, so I will probably commit it in the next couple of days. thanks!

If you want more mapper speed you should try the CPP one (it also has the advantage of having fewer dependencies once compiled). But it's a PITA to modify... which is why I'll stick with python myself. There are probably a few things that could be done to eke a bit more speed out of the python version but bear in mind the some features are slow (e.g. caves).

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

@spillz: I may have overthought the calculation of world boundaries, so I've returned to the simple version (edited code above), but comments in PNG are working fine. Speed isn't a great issue for me, and my CPP is quite rusty, so I'll pass on that offer. Worst case I might have to make a cup of tea while waiting for map generation.

BTW: I'm wondering about adding an option to generate a thumbnail with particular dimensions and fill the edges with background. I've made a "map block", but unless the world has been explored in a square, the aspect ratio of the displayed map is off. If I'm right the following would work to enable saving of a 'thumbnail' (but I'm not on the right computer to try it at the moment):-

Code: Select all

size = 128, 128
im.thumbnail(size, Image.ANTIALIAS)
im.save(args.output+ ".thumbnail", "PNG")
EDIT: This doesn't work, the working code has been edited into the snippet above

I'd also vote for water *semi* transparency if you're doing it, and just realised I've possibly done your ToDo #2. GUI. Have you got any other suggestions for it?

spillz
Member
Posts: 138
Joined: Thu Feb 13, 2014 05:11

Re: numpy-izing minetestmapper

by spillz » Post

@dgm555: I added a slightly modified version of your patch (added a command line option for the thumb, and replaced an int() with a float())

https://github.com/spillz/minetest/comm ... 2322e6e5df

spillz
Member
Posts: 138
Joined: Thu Feb 13, 2014 05:11

Re: numpy-izing minetestmapper

by spillz » Post

dgm5555 wrote: I'd also vote for water *semi* transparency if you're doing it, and just realised I've possibly done your ToDo #2. GUI. Have you got any other suggestions for it?
The semi-transparent water is straightforward enough, just more ugly code like the underground stuff. One thing I'm thinking about is that it should be possible to define various blocks as transparent in colors.txt and apply that color with a specified alpha level to blocks beneath. I'll add this feature sometime this week, probably.

Re GUI, the main thing I had in mind was a small app that draws the top view map, then makes it easy to select cross sections to view. Not sure that's all that useful (or easy to implement) for your in-game map.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

spillz wrote:Re GUI, the main thing I had in mind was a small app that draws the top view map, then makes it easy to select cross sections to view. Not sure that's all that useful (or easy to implement) for your in-game map.
In any case, I don't use the cross-section views, but as there seems to be no way of hooking the mouse clicks from lua, I got around it by making a square of 10x10 transparent buttons over the map, but I think (haven't tried it yet) that there is no way of forcing a reload of a texture in minetest (if there was, I would occasionally call the function in the background to update the map).
In any case a frontend gui should be pretty easy with tkinter if you wanted one.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: numpy-izing minetestmapper

by dgm5555 » Post

spillz wrote:One thing I'm thinking about is that it should be possible to define various blocks as transparent in colors.txt and apply that color with a specified alpha level to blocks beneath.
Just a thought. If you do this, could you keep the list separate from the main colors.txt data (eg a separate block at the beginning of the file) - it would be nice to be able to copy and paste it to an updated colors.txt in the future (when/if it gets updated), and if the data is distributed it will be more difficult to extract...
Actually if you were using this system, the same could be done with leaves and glass too, which could be interesting, though probably too small a scale to make much difference.

Dartmouth
Member
Posts: 121
Joined: Sat Dec 06, 2014 14:39

Re: numpy-izing minetestmapper

by Dartmouth » Post

Using VanessaE's colors.txt file, you have to remove the two comments lines starting with "# Random" and "# From", otherwise it will throw errors when trying to call int() on those strings.

Otherwise this script is really good, except that python3 does not work, there is at least one python2 print statement on line 1036 ;)

User avatar
mrob27
Member
Posts: 21
Joined: Sun Sep 27, 2015 13:32
GitHub: mrob27
Contact:

Re: numpy-izing minetestmapper

by mrob27 » Post

@thetoon suggested back on 28th March 2014 (on page 2 of this thread) that Python 2.7 works and they had no need to change colors.txt; and spillz suggested on 29th April 2014 that it "Should work with both python 2 and 3."

But since then things seem to have changed, or maybe this never got tested with SQL databases.

Anyway, I have Python 2.7. To get it to work, I had to make one small change. Find "return BytesIO(r[0])" and change it to "return BytesIO(bytes(r[0]))".

I also removed the back-quotes from the SQL queries (like "SELECT `pos` FROM `blocks`") because I couldn't find anything online to explain why there should be back-quotes in an SQL query within Python. (EDIT: I searched again and found this: using the back-quotes protects against problems if a future version of SQL defines "pos" or "blocks" as a built-in reserved keyword. However, they are not ANSI-SQL compliant. I choose compliance and remove the back-quotes.)

Finally, I added about 80 or 90 lines to my colors.txt which still needs some tweaking (e.g. sandstone is still wrong but I made it better).

Example output (in a spoiler because it is big)
Spoiler
[center] Image [/center]
Yes the terrain is old, because I'm stuck with 0.4.11 on a Mac that is stuck with MacOS 10.6.

- Robert Munafo

User avatar
poikilos
Member
Posts: 67
Joined: Thu Feb 18, 2016 13:45
GitHub: Poikilos
In-game: Poikilos

Re: numpy-izing minetestmapper

by poikilos » Post

You forgot to add 16 (or 15 if inclusive rect) to the boundaries. The code should read:

Code: Select all

    pngminx = minx*16
    pngmaxx = maxx*16+16
    pngminz = minz*16
    pngmaxz = maxz*16+16
    pngregion=[pngminx, pngmaxx, pngminz, pngmaxz]
Using the current code (without +16), [pngregion[1]-pngregion[0], pngregion[4]-pngregion[3]] is a size that 16 fewer blocks than the actual result image.
(I noticed because my WIP livemap http://github.com/expertmm/minetest-chunkymap analyzes the stdout of minetestmapper-numpy.py).

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: numpy-izing minetestmapper

by burli » Post

I want to add more colors, but there are two types of them

80c 183 183 222 # CONTENT_GLASS
default:glass 183 183 222

If I understand the code the second type is ignored. So I need the identifier numbers for the nodes. Where do I get them?

User avatar
BirgitLachner
Member
Posts: 393
Joined: Thu May 05, 2016 10:18
In-game: Bibs

Re: numpy-izing minetestmapper

by BirgitLachner » Post

a bit frustrating ... none of the mappers do work :-/

I got the error:

Code: Select all

Error at (50,23,-1): ord() expected a character, but string of length 0 found
... and some more errors after that but they are may be caused by this first error. Here is the full output:

Code: Select all

Result image (w=304 h=96) will be written to mars1.png
Error at (50,23,-1): ord() expected a character, but string of length 0 found
Traceback (most recent call last):
  File "./minetest-mapper.py", line 719, in generate_map_info
    static_object_version = readU8(f)
  File "./minetest-mapper.py", line 145, in readU8
    return ord(f.read(1))
TypeError: ord() expected a character, but string of length 0 found
Block data: Traceback (most recent call last):
  File "./minetest-mapper.py", line 1070, in <module>
    main()
  File "./minetest-mapper.py", line 1065, in main
    world.generate_map_info(str_to_uid)
  File "./minetest-mapper.py", line 812, in generate_map_info
    for c in r[0]:
NameError: global name 'r' is not defined

User avatar
poikilos
Member
Posts: 67
Joined: Thu Feb 18, 2016 13:45
GitHub: Poikilos
In-game: Poikilos

Re: numpy-izing minetestmapper

by poikilos » Post

BirgitLachner wrote:a bit frustrating ... none of the mappers do work :-/

I got the error:

Code: Select all

Error at (50,23,-1): ord() expected a character, but string of length 0 found
... and some more errors after that but they are may be caused by this first error. Here is the full output:

Code: Select all

Result image (w=304 h=96) will be written to mars1.png
Error at (50,23,-1): ord() expected a character, but string of length 0 found
Traceback (most recent call last):
  File "./minetest-mapper.py", line 719, in generate_map_info
    static_object_version = readU8(f)
  File "./minetest-mapper.py", line 145, in readU8
    return ord(f.read(1))
TypeError: ord() expected a character, but string of length 0 found
Block data: Traceback (most recent call last):
  File "./minetest-mapper.py", line 1070, in <module>
    main()
  File "./minetest-mapper.py", line 1065, in main
    world.generate_map_info(str_to_uid)
  File "./minetest-mapper.py", line 812, in generate_map_info
    for c in r[0]:
NameError: global name 'r' is not defined
Is spillz around anymore or can anyone explain how to fix this error?
I tried wrapping in in a 'try' block, but much of the world (below y=1 or so?) seems to be the underground (I did not set a maxheight so minetestmapper-numpy should use its default which is 300, so maybe even with my workaround the issue still prevents it from going higher than around 0--I will be changing the params to --min-y and --max-y in my fork to conform to the official C++ minetestmapper).
The repo is not easily maintainable since it is only on GitHub as a fork of Minetest and Minetest team no longer has any copy of the python version online, and there is no reason to fork minetest to modify one script file especially if it will decidedly never be pulled (see https://github.com/minetest/minetest/pull/5901).

spillz or anyone willing to help, Please fork from my repo where I added more standard output and fixed the crash: https://github.com/expertmm/minetestmapper-python (I suggest deleting your fork of minetest, if that is the only thing you changed, and work off of my repo for the reasons above and put the new link to that here on the initial post)

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests