[Mod] Converter Minecraft world to Minetest world

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

[Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

EDIT: Nore has made an undocumented, but fully functional python Minecraft to Minetest converter at https://github.com/Novatux/mcimport.
It takes an entire world save folder from minecraft and imports it into minetest with the only restriction being the +/-31k node limit in each direction of minetest (and I haven't found any worlds that big)
Please note this is not the same as the identically named mcimport at: viewtopic.php?f=9&t=4445, which is a lua script for running within a pre-generated world, and is limited to classic minecraft maps and a small import size due to apparent lua memory management glitches.

Since mcimport isn't documented elsewhere, here are some notes:-
mcimport is run from the command line with:-

Code: Select all

python3 mcimport.py ~/.minecraft/saves/mcImportName ~/.minetest/worlds/mtOutputName
The minecraft world can be saved anywhere (eg ~/.minetest/worlds/mcworlds)
NB 'blocks saved' progress refers to sections/chunks of 16x16x16 (= 4096) nodes...

I have created a mod (WIP) which will display all the blocks and will hopefully largely duplicate the function of minecraft blocks. For further documention on this (which supersedes much of the following), see:-
viewtopic.php?f=9&t=11448
Alternatively if you want to use native minetest mods the following will provide a start (but the conversion with this is by no means completed). This mod also includes a generateCommandLine.py script which will (with a little editing on your part) generate the command lines required to automatically convert and copy the converted world.

(Or if you prefer doing it manually and want to convert multiple worlds (eg overnight) on linux put a semi-colon (;) after the command and then repeat it with edits for the next world.)

You will need to manually create the output directory before running the script
You will need to copy the file
world.mt, which contains the text:-

Code: Select all

gameid = minetest
backend = sqlite3
load_mod_moretrees = true
load_mod_nether = true
load_mod_quartz = true
load_mod_plants_lib = true
load_mod_bushes_classic = true
load_mod_flowers_plus = true
load_mod_junglegrass = true
load_mod_poisonivy = true
load_mod_moreblocks = true


The conversion works better if the following mods are enabled:
moretrees (depends: plantlife), nether, quartz
MoreTrees: viewtopic.php?f=11&t=4394&hilit=moretrees; https://github.com/VanessaE/moretrees
PlantLife: viewtopic.php?f=11&t=3898; https://github.com/VanessaE/plantlife_modpack

Nether: viewtopic.php?id=5790; https://github.com/PilzAdam/nether
Quartz: viewtopic.php?id=5682; https://github.com/4Evergreen4/quartz

Other minetest_game mods should be enabled automatically: default, farming, flowers, stairs, wool

There are amazing minecraft worlds to be downloaded from many sites, but for starters you could try
http://www.planetminecraft.com/resources/projects/
http://www.minecraftworldmap.com/browse


This is not part of the core code, but if you want to quickly create a batch script to convert a number of worlds, you can use the following script to generate the command lines to do it (it deletes any old conversion, makes the new folder, copies player priviledges and world settings, then converts and copies the conversion to the main directory (leaving the original as a backup), then repeats for all minecraft worlds in the directory):-

Code: Select all

import os
srcDir='/home/david/.minetest/worlds/mcw'
for x in os.listdir(srcDir):
    if os.path.isfile(x): pass
    else:
        if not(x.find('.zip')>0 or x.find('.rar')>0 or x.find('.txt')>0 or x.find('.mt')>0 or x.find('mcw/convert')>0):
            print('rm -r \"/home/david/.minetest/worlds/mcw/convert/' + x + '\"')
            print('mkdir \"/home/david/.minetest/worlds/mcw/convert/' + x + '\"')
            print('cp \"/home/david/.minetest/worlds/mcw/world.mt\" \"/home/david/.minetest/worlds/mcw/convert/' + x + '\"')
            print('cp \"/home/david/.minetest/worlds/mcw/auth.txt\" \"/home/david/.minetest/worlds/mcw/convert/' + x + '\"')
            print('python3 mcimport.py \"/home/david/.minetest/worlds/mcw/' + x + '\" \"/home/david/.minetest/worlds/mcw/convert/' + x + '\"')
            print('cp -r \"/home/david/.minetest/worlds/mcw/convert/' + x + '\" \"/home/david/.minetest/worlds/' + x + '\"')
            print('')

As mcimport is much better than my attempt, I'm unlikely to work more on the mineblend conversion, but will leave it available for curiosity of others, so...
IGNORE THE FOLLOWING IT IS OUTDATED:-
This is the first stage of a python3 -based minecraft to minetest map converter.
STAGE 1 WORKING: Takes a map from minecraft and produces a list of all the blocks in the map
Because I've basically crippled a blender tool the code is a total mess.
The code is on github, and you would be extremely welcome to work on it. Most of the files are totally unnecessary, but I thought I'd aim to make it available as it is.
https://github.com/dgm3333/mc2mt
https://github.com/dgm3333/mc2mt/archive/master.zip
See my next post below for instructions.
I think you will need to install the demo version of minecraft (I don't know why, but it seems to be necessary)
I'm running it from Liclipse (as below), but you can run the code more simply with

Code: Select all

python3 __init__.py
TODO:-
STAGE 2: convert from minecraft to minetest node type
Based around conversion code from viewtopic.php?f=12&t=6007

STAGE 3: mash data into a minetest map format.
Breakdown here viewtopic.php?f=47&t=11157


Image
Last edited by dgm5555 on Tue Mar 10, 2015 21:16, edited 21 times in total.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: Convert Minecraft worlds to Minetest worlds directly

by dgm5555 » Post

There are various python scripts to convert minecraft worlds to be available to other programs.
One simple one is mineblend, a minecraft - blender conversion, which can read the modern minecraft anvil format and inport it into Blender: http://randomsamples.info/project/mineblend/. The license is GNU GENERAL PUBLIC LICENSE.
With this code already available could someone bootstrap it to create a minetest format world directly from a minecraft world.

Then you could have a simple text-based lookup table to convert from a minecraft block to a minetest one (just as tomtsschem.py viewtopic.php?f=12&t=6007does) and resort the blocks into chunks for minetest.

Finally put it directly into a minetest world sqlite file with any header code.

Even if it took all night for a world to convert it would be worth it as it give minetest users access to the thousands of amazing worlds and objects built by minecraft users...
But the mineblend script can import a reasonable sized world in only a minute or so, which would save a lot of agony compared with mucking around with worldedit.


I think it requires minecraft to be installed, but the free demo version works fine.
To test the conversion under blender.
I registered for and installed the demo version of minecraft
Installed the mineblend script in /home/david/programs/blender-2.72b-linux-glibc211-x86_64/2.72/scripts/addons/io_import_minecraft (only because I don't formally install blender, just extract it to a standalone directory)
Copied a random minecraft map I downloaded from the internet, and extracted the save game directory to /home/david/.minecraft/saves
(the correct directory to have in saves is the one with level.dat and the data subdir in it)
The world wasn't recognised by mineblend until I also copied the "playerdata" folder into that folder from the minecraft demo world (just having the "players" subdir means the world isn't recognised).
I increased the loadradius to 20.
A few seconds later, I have a decent portion of a minetest world imported into blender (admittedly somewhere randomly off screen, so it took me a little while to realise it was there).

Worldedit is totally unstable attempting to import any large volumes. The python version of Minetestmapper-numpy can read an entire minetest world in seconds (or worst case minutes). Apparently the c version is even faster, but python is so much easier for most of us to code and we don't have to recompile for every different computer. tomtsschem.py is so incredibly slow (presumably because it's relying on mcedit to access the file, but also possibly because it doesn't follow the minecraft block order, so extraction is very inefficient.
I'm thinking it would be far better to just bypass mcedit entirely.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: Converter Minecraft map to Minetest map [mod]

by dgm5555 » Post

Here is my initial quick thoughts on which files do what (NB I might well be totally incorrect)
__init__.py, mineregion.py, mcanvilreader.py and nbtreader.py seem the most likely to be useful

Probably one very simple way to start testing would be a minor edit of __init__.py to disable the scene generation in blender and just export all nodes and properties to a text file, or to a minetest .sqlite file. Then use blender as the python interpreter.

blockbuild.py - seems to create the meshes in blender to represent minetest blocks
__init__.py - is 'main()' it defines the main variables, and calls the various functions. From our pov, it's rather cluttered with code defining blender specific functions (eg menus and generation of the scene)
javarandom.py - An implementation of the Java SE 6 random number generator
LICENSE.txt - the GNU license
mcanvilreader.py - Anvil is the current Minecraft 1.x file format http://minecraft.gamepedia.com/Anvil_file_format
mcregionreader.py - mcRegion was the older file format http://minecraft.gamepedia.com/Region_file_format
mineregion.py - seems to do a lot of the grunt work, and contains amonst other things, information about each type of minecraft block
nbtreader.py - the tree/tag structure minecraft store info in http://minecraft.gamepedia.com/NBT_Format
slimes.py - "Javarandom slime Python-version test harness" - huh?
sysutil.py - defines file paths dependent on OS
writeNoodleBuilder.py - I think this creates a 'tree' of objects within a Blender file

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

As above the reader based on mineblend works well and does a basic 'proof of concept' conversion from MC to MT blocks.

This is my current code for converting a minecraft map to a minetest map.
At the moment my notes for conversion are within the output section, but I am considering keeping it relatively separate so the writer is able to be used for other projects where people which to generate their own maps (eg converting from bumpmaps, etc).
It's not finished, but any comments, suggestions or additions would be gratefully appreciated...

Code: Select all

import zlib
import sqlite3
import numpy

zlibCompressLevel = 6
rootDir = '~/user/.minetest/worlds/'

def int64(u):
    while u >= 2**63:
        u -= 2**64
    while u <= -2**63:
        u += 2**64
    return u

def generateMTDB(worldname):
#    mkdir rootDir + worldname

    #create the database file
    db = sqlite3.connect(rootDir + worldname + '/map.sqlite')

    #create the table within the database
    cursor = db.cursor()
    cursor.execute('CREATE TABLE blocks (pos INT NOT NULL PRIMARY KEY,data BLOB)')
    db.commit()
    return db

def saveSectionToMTDB(db,chunkZ,chunkY,chunkX,MCSectionXYZ,secNameIDDic):
    cursor = db.cursor()

    # calculate the primary key ID
    # NB due to variations in MC vs MC axes, this will need to be converted (see below under param0)
    pos = int64(chunkZ*16777216 + chunkY*4096 + chunkX)

    # Generate the map header information
    mapFormatVersion = numpy.uint8(24)

    is_underground = 0x01
    day_night_differs = 0x02
    lighting_expired = 0x04
    generated = 0x08
    flags = numpy.uint8(is_underground | day_night_differs | lighting_expired | generated)


    content_width = numpy.uint8(2)    # Number of bytes in the content (param0) fields of nodes
    params_width = numpy.uint8(2)     # Number of bytes used for parameters per node

    # now convert the node data within the section from MC array to MT array then serialise and compress it
    param0 =  numpy.zeros((16,16,16), dtype=numpy.uint16)
    # In Minecraft, West is +Z (max=∞) and East is -Z (min=-∞), North is -X (min=-∞) and South is +X (max=∞), Up is +Y (max=256) and Down is -Y (min=0), ground level is 63
    # In Minetest,  -X (west) <--- 0 ----> +X (east),           North is +Z and South is -Z,                  Up is +Y and Down is -Y (min=0),           ground level is 0
    #    Minetest dimensions are max 30928 nodes (=1933 chunks) in any direction from origin of 0

    #need to fill the array here
    # Ignore nodes >30928 or <-30928 or ignore chunks >1933 or <-1933

    #Switch axes around
    # NB both
    #    MT  -X   +X   -Z   +Z   Y
    #    MC  +Z   -Z   +X   -X   Y-63


    # serialise and compress data
    p0zlib = zlib.compress(numpy.reshape(param0, 4096), zlibCompressLevel)



    # convert the MC blockdata to MT param1 node data
    param1 =  numpy.zeros((16,16,16), dtype=numpy.uint8)
    #need to fill the array here
    p1zlib = zlib.compress(numpy.reshape(param1, 4096), zlibCompressLevel)



    # I don't think MC has a param2, so just create a dummy array full of zeros
    param2 =  numpy.zeros((16,16,16), dtype=numpy.uint8)
    p2zlib = zlib.compress(numpy.reshape(param2, 4096), zlibCompressLevel)



    #zlib-compressed node metadata list
    metaVersion = numpy.uint16(1)
    metaCount = numpy.uint16(0)

    #Node timers (assume none)
    timerVersion = numpy.uint8(0)

    #Static Objects (assume none - not even sure what they are)
    staticObjVersion = numpy.uint8(0)
    staticObjCount = numpy.uint8(0)

    #Timestamp (mark as invalid/unknown timestamp)
    timeStamp  = numpy.uint32(0xffffffff)

    # Name<->ID Mapping
    NameIDMapVersion = numpy.uint8(0)
    NameIDMapCount = numpy.uint16(len(secNameIDDic))    # fileformat say u16 num_name_id_mappings, but this is located to be a count, and https://github.com/minetest/minetest/blob/master/src/nameidmapping.cpp saves a size
    nameIDs = NameIDMapVersion + NameIDMapCount

    for curNameID in secNameIDDic:
        NameIDMapID = numpy.uint16(curNameID.id)
        NameIDMapLen = numpy.uint16(len(curNameID.name))
        NameIDMapName = numpy.uint8(curNameID.name)
        nameIDs = nameIDs + NameIDMapID + NameIDMapLen + NameIDMapName


    # concatenate the data header
    data = mapFormatVersion + flags + content_width + params_width
    # concatenate the node data
    data = data + p0zlib + p1zlib + p2zlib
    # concatenate the meta data (NB not sure what this is for)
    data = data + metaVersion + metaCount
    # concatenate node timers (NB assuming timers not being converted
    data = data + timerVersion
    # concatenate static objects
    data = data + staticObjVersion + staticObjCount
    # concatenate timeStamp
    data = data + timeStamp
    # concatenate timeStamp
    data = data + nameIDs


    # now write to the database
    cursor.execute('INSERT INTO blocks(pos, data) VALUES(?,?)', (pos,data))
    db.commit()

Nore
Developer
Posts: 501
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: [Mod] Converter Minecraft world to Minetest world

by Nore » Post

I already had a MC -> MT converter here: https://github.com/Novatux/mcimport. It looks similar to yours, but maybe you can improve them with what you're doing (sorry, I have no time to improve mine right now).

User avatar
Calinou
Moderator
Posts: 3169
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

Re: [Mod] Converter Minecraft world to Minetest world

by Calinou » Post

By the way, I've been thinking of creating a mc_compatibility mod. This mod would have two aims:
  • Provide nodes, craftitems and tools with behaviour similar to Minecraft's (eg. mc_compatibility:melon_slice),
  • Provide textures and sounds for these (not taken from Minecraft).

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

That sounds like a great idea, currently I've just got a dictionary to convert between blocks, so anything can be plugged in, but it's likely to get unwieldy with lots of mods, so a compatible mod would make life much easier.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Nore wrote:I already had a MC -> MT converter here: https://github.com/Novatux/mcimport. It looks similar to yours, but maybe you can improve them with what you're doing (sorry, I have no time to improve mine right now).
Why on earth would you not have previously put that somewhere on the forum!! Or at least a readme on git that might be indexed so a google search picked it up!! This would have saved me about 3 weeks and something over 30 hours (and a lot of money if I'd done something more productive). I accept it do with some (very slight) tweaking, but I could have spent that time figuring out something which was made for purpose. And you've had it sitting there for a year since your last edit! AAARRGH!!!
Unfortunately I've now only got today to work on it as I've got to return to all the work I've been putting off, and yours is already better than mine. What a waste (but thanks for telling me, if I get more time, I'll work with yours instead).

EDIT: OK I take it back, I doubt I could do much to optomise it - my python is nowhere near yours...

The tweaks would be to set up the world with the other files and mods enabled, and possibly a little more documentation of the running of the script.

Nore
Developer
Posts: 501
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: [Mod] Converter Minecraft world to Minetest world

by Nore » Post

There are a few more things that could be done, such as converting inventories, converting entities, preserving trunk facedir, etc., and adding more nodes & items (maybe including some from mc mods), but some mod like Calinou suggested would probably be best (even a mod that would just add those items, whose effects would then be added by other mods) could be nice. Also, since it is old, farming, tnt, etc are not converted either.
Oh, and sorry for not having made a forum topic (I just remember RBA mentionned it once in a thread in maps, for a map he had converted that way).

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

I'm just making notes in this post for things to look at should/when I get back to it:
Set default input and output root directory,
Make output directory if doesn't exist
Enhance progress display (% progress and time estimate). If this is too slow to check in advance, perhaps a rough estimate could be made by the number of region files. Ideally this could be directed to std-error so it doesn't scroll.
Display a list of the bounds of world import when finished (in case you can't find them).
Have a 'convert all worlds in dir' which autogenerates names to default output root.
Option to block chunk generation above any chunks of the import (?up to MC y=256) and set them to air
Option to increase y-height of world (so it doesn't subsequently have land generated around it)
Improve block conversion
Print list of unrecognised blocks and data associated with it (there's a very common plant/tree missing from the conversion, and a ?brick (used in house/structures)). for extra marks: Convert this as a special 'unrecognised' block which could be punched to get the minecraft stats on it.
Have a special non-spreading lava and fire, as these seem to cause significant damage when converting as the default minetest versions. (and possibly water)
I wonder if theres a way to stop power off/suspend while the script is running
Move spawn point to the minecraft one
Check error:

Code: Select all

david@zaiqLin:~/Documents/LiClipse Workspace/mcimport$ python3 mcimport.py "/home/david/.minetest/worlds/mcw/StargateAtlantis-Gearmage" "/home/david/.minetest/worlds/mcw/convert/StargateAtlantis-Gearmage"
Saved 0 blocks
Saved 50 blocks
Saved 100 blocks
Traceback (most recent call last):
  File "mcimport.py", line 9, in <module>
    mtmap.save()
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 306, in save
    for block in self.blocks:
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 292, in fromMCMapBlocksIterator
    mtblock.fromMCBlock(mcblock, conversion_table)
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 177, in fromMCBlock
    content[i], param2[i] = conversion_table[blocks[i]][data[i]]
IndexError: list index out of range
Is it using the old MC maxblock of 256? (nope range goes up to 4096). Looks like it might derive back to the table generated from map_content.txt by content.read_content. The error still occurs with a try/except clause.


Also check it wont try to import chunk >1933 or <-1933?

This are some import glitches:
https://dl.dropboxusercontent.com/u/212 ... 6%3A58.png
https://dl.dropboxusercontent.com/u/212 ... 4%3A37.png


Having said that, this is an awesome script. Here's an example which took approx 10mins to convert:-
Image
Last edited by dgm5555 on Mon Feb 16, 2015 07:48, edited 1 time in total.

User avatar
ExeterDad
Member
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
In-game: ExeterDad
Location: New Hampshire U.S.A

Re: [Mod] Converter Minecraft world to Minetest world

by ExeterDad » Post

Whoa! That is very cool!

twoelk
Member
Posts: 1482
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

Re: [Mod] Converter Minecraft world to Minetest world

by twoelk » Post

Oh dear, I desperately need time to get this running on my windows machine ;-P
somehow

regarding the time you invested on this subject, do not consider it wasted. With the insight you have gained you could be able to tweak and add to the code more than most others.

Although I have been fiddling with converting Minecraft map content to Minetest for quite a while I have sadly missed most of youre progress. Maybe too late for this round I do have suggestions for your next burst of activity though.
  • A basic request is to not hardcode absolute paths into the tool. Both my Minecraft and my Minetest installations are portable run-in-place versions that are slightly tweaked and do not use the file paths of normal installations. So consider that the worlds may reside in some remote and portable places and need not be anywhere near a client or server installation.

As these tools of both of you can write to a minetest database they could be usefull for more things than oneway map translations.
  • For one consider there are several mc-map versions so making reading the map source a modular plugin system might be usefull. Besides mc there are also for example maps of minetest forks such as Freeminer and Voxelands (Minetestclassic) that could make use of a converter (more or less). Others could supply more input filters for maybe Terasology or whatever else is related enough.
  • If you include the current Minetest format as input and support all database backends your programm might become usefull to convert between database backends without having to run minetest itself.
  • This might be also usefull for other map manipulations directly in the database. For example one could supply an input file with node names which the programm could replace or delete.
    This could be used for example to delete unknown node relicts from uninstalled mods.
    It could also be used to delete certain regions of an existing map. like deleting everything below a certain depth or outside a defined area. I think OldCoder among others was working on a script to achieve that.
  • Maybe the code could be also used to import content into an existing map like combining two maps (VanessaE and Sokomine worked on a bash script that did that) This would also have to include a possability to move content around like shifting an existing region a defined amount to one side so that the content of both maps is somewhat appart.
  • It might also be usefull to be able load the different versions of schematic files, including those from mc
Last edited by twoelk on Mon Feb 16, 2015 10:08, edited 1 time in total.

Nore
Developer
Posts: 501
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: [Mod] Converter Minecraft world to Minetest world

by Nore » Post

dgm5555 wrote:

Code: Select all

david@zaiqLin:~/Documents/LiClipse Workspace/mcimport$ python3 mcimport.py "/home/david/.minetest/worlds/mcw/StargateAtlantis-Gearmage" "/home/david/.minetest/worlds/mcw/convert/StargateAtlantis-Gearmage"
Saved 0 blocks
Saved 50 blocks
Saved 100 blocks
Traceback (most recent call last):
  File "mcimport.py", line 9, in <module>
    mtmap.save()
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 306, in save
    for block in self.blocks:
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 292, in fromMCMapBlocksIterator
    mtblock.fromMCBlock(mcblock, conversion_table)
  File "/home/david/Documents/LiClipse Workspace/mcimport/block.py", line 177, in fromMCBlock
    content[i], param2[i] = conversion_table[blocks[i]][data[i]]
IndexError: list index out of range
Is it using the old MC maxblock of 256? (nope range goes up to 4096). Looks like it might derive back to the table generated from map_content.txt by content.read_content. The error still occurs with a try/except clause.
I reckon I found out where the bug was: did you use a world with a block id that was > to all known blocks ids? I have made a commit that will hopefully fix this, could you check?

About the import glitches, it looks like a bug in the conversion table; you can look at the table in map_content.txt to check that. Also, you don't *need* moretrees, quartz and nether: in the main script, just remove those you don't want from the list.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Nore wrote:I reckon I found out where the bug was: did you use a world with a block id that was > to all known blocks ids? I have made a commit that will hopefully fix this, could you check?
Brilliant no more glitch.
twoelk wrote:Oh dear, I desperately need time to get this running on my windows machine ;-P
You just need to install python even on Win it should only take 5mins...
The major libs are sqlite and zlib which are pythonic defaults. It doesn't even need numpy.
[It doesn't require minecraft or minetest to be installed]

Paths aren't hardcoded (that was just my little bolt-on batch generation script which generates the command lines to convert a bunch of worlds sequentially)

Your ideas are good, I was wondering about extending that idea with a simple mod within minetest to locate your desired point for import then insert it with mcimport. No more tempramental world-edit. If I understand minetest's handling of the db correctly it should be possible to do this while playing the game.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Nore: Would it be difficult to stop generation of land above an imported area (eg by marking it as generated and filled with air) and/or be able to set the y- import height via command (so a smaller world can be matched to height of land around it)? Having the map generated over/around the import is ruining a significant number of my trialed worlds.


The following node types are converted but not recognised (and I'm not sure if they are an error, or if they were in an old version of the mod):
stairs:stair_nether_brick --------- however in nether mod is ------ replace("stairs:stair_cobble", "nether:brick")
stairs:slab_nether_brick --------- not sure where this might come from...
wool:purple -------------------------- however wool has ---------------- wool:violet
(BTW the plant faults in previous post were due to me not including all of the plant_lib mods)
Unrecognised nodes are currently converted to air

The following mod could be used to stop fire and lava setting fire to things:
https://dl.dropboxusercontent.com/u/21225632/mcsafe.zip

change in mcimport.py:

Code: Select all

nimap, ct = content.read_content(["MORETREES", "NETHER", "QUARTZ", "MCSAFE"])
change in map_content.txt

Code: Select all

#if MCSAFE
	8	mcsafe:water_flowing
	9	mcsafe:water_source
	10	mcsafe:lava_flowing
	11	mcsafe:lava_source
#else
	8	default:water_flowing
	9	default:water_source
	10	default:lava_flowing
	11	default:lava_source
#endif

#if MCSAFE
	51	mcsafe:basic_flame
#else
	51	fire:basic_flame
#endif

I'm not sure if its a related problem, but there seem to be a number of holes in things (walls/posts/etc) (but with trees sprouting everywhere and random hills it's a bit difficult to be sure.
Last edited by dgm5555 on Fri Feb 20, 2015 07:24, edited 1 time in total.

User avatar
Inocudom
Member
Posts: 3121
Joined: Sat Sep 29, 2012 01:14
IRC: Inocudom
In-game: Inocudom

Re: [Mod] Converter Minecraft world to Minetest world

by Inocudom » Post

Impressive work you have done here. Would you be interested in making a script that exports maps to the Cube 2/Tesseract engines? It might be an interesting experiment.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

y-height is easy to manipulate if you are willing to accept chuck sized steps (ie 16 nodes). At least then mapgen doesn't get in the way...
just edit
block.py line 167 with a new YOFFSET
def fromMCBlock(self, mcblock, conversion_table):
self.pos = (mcblock.pos[0], mcblock.pos[1]-4+YOFFSET, mcblock.pos[2])

Also just wondering if it would be easier to modify the mapgen code to adjust it's height in smaller increments (to get finer height control). Obviously this would vary with seed, so anyone wanting to do this could find this was a bit fiddly, but would be easy to write a script to trial height changes and repeatedly overwrite the map.sqlite db file until it was right (overwriting db is equivalent to deleting all the mapgen changes)

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Actually thinking a bit more about moving the import around. It would make more sense to convert it first, without moving it, then copy to a backup, then do the move with a very simple sqlite query like the following (which only takes a few seconds). This means you can quickly tweak the settings and change the import location as you wish.

[EDIT]: THIS SCRIPT MOVES THE ENTIRE WORLD AROUND

Code: Select all

import sqlite3
# set the number of blocks you want to move the world (each block is 16 nodes)
xOffset = 0
yOffset = 0
zOffset = 1

zyxOffset = int(zOffset * 16777216 + yOffset * 4096 + xOffset)
db = sqlite3.connect('/home/david/.minetest/worlds/testWorld/map.sqlite')
cur = db.cursor()

# move the pos values far away, then back so there are never any duplicate pos values
cur.execute('UPDATE blocks SET pos = pos + {xyz}'.format(xyz=zyxOffset+160000))
cur.execute('UPDATE blocks SET pos = pos - 160000')
db.commit()
db.close()

Another possible task would be to merge multiple worlds together:-
http://nahitworkplace.blogspot.co.uk/20 ... n-two.html

Code: Select all

 import sqlite3 as lite  
 import sys  
   
 def join_database(database1Name, database2Name, database3Name):  
  ''' Join two database 'database1Name' and 'database2Name' into database 'database3Name' '''  
    
  con1 = lite.connect(database1Name)      # Connect to Database1  
  con2 = lite.connect(database2Name)      # Connect to Database2  
  con3 = lite.connect(database3Name)      # Connect to Database3  
   
  con1.row_factory = lite.Row         # For reading column names from Database1  
  con2.row_factory = lite.Row         # For reading column names from Database2  
   
  con1_new = lite.connect(database1Name)    # Create new connection with Database1 for reading data  
  con2_new = lite.connect(database2Name)    # Create new connection with Database2 for reading data  
   
  with con1, con2, con3, con1_new, con2_new:  
   cur1 = con1.cursor()            # Get cursor from connection 1 (con1)  
   cur2 = con2.cursor()            # Get cursor from connection 2 (con2)  
   cur3 = con3.cursor()            # Get cursor from connection 3 (con2)  
    
   cur1_new = con1_new.cursor()        # Get connection from connection 1_new (con1_new)  
   cur2_new = con2_new.cursor()        # Get connection from connection 2_new (con2_new)  
     
   cur1.execute("SELECT * FROM your_table_name")  # Select all column names from table your_table_name in Database1 (Why column names see line no. 11)  
   cur2.execute("SELECT * FROM your_table_name")  # Select all column names from table your_table_name in Database2 (Why column names see line no. 12)  
    
   row_1 = cur1.fetchone()      # Fetch column names from database1 table  
   row_2 = cur2.fetchone()      # Fetch column names from database2 table  
    
   columnNames = row_1.keys()    # Read column names from Database1  
     
   for item in row_2.keys():     # Append column names from Database1 with Database2, including duplicate column names only once (See description)  
    if item not in row_1.keys():  
     columnNames.append(item)  
    
   cur3.execute("CREATE TABLE IF NOT EXISTS your_table_name(Id INTEGER PRIMARY KEY, Date TEXT(10), Time TEXT(8))")  # You can change this line according to your needs  
     
   for col in columnNames[3:]:  
    cur3.execute("ALTER TABLE your_table_name ADD {} DOUBLE".format(col)) # You can change the data type(here it is DOUBLE) according to your needs  
    
   cur1_new.execute("SELECT * FROM your_table_name")  # Select complete data at once from table your_table_name in database1  
   cur2_new.execute("SELECT * FROM your_table_name")  # Select complete data at once from table your_table_name in database2  
    
   rows1 = cur1_new.fetchall()   # Read all rows from Database1  
   rows2 = cur2_new.fetchall()   # Read all rows from Database2  
    
   rows1 = [r[1:] for r in rows1]  # Except first row in rows1 read all rows  
   rows2 = [r[3:] for r in rows2]  # Except first three rows in rows2 read all rows  
    
   rows = [r1 + r2 for r1, r2 in zip(rows1, rows2)] #Combine each rows in rows1 and rows2 element wise  
    
   ques = []            # Use by sqlite for inserting into table  
     
   ques = ["?"]*len(columnNames[1:])  # Generate list [?, ?, ?, ?,........till length equals length of columnNames[1:]  
   ques = ",".join(ques)        # Generate string "?,?,?,?,?........"  
     
   columnNames = ",".join(columnNames[1:]) # Generate string "col1, col2, col3............"  
      
   for item in rows:        # Insert combined data into new Database3  
    cur3.execute("INSERT INTO your_table_name({0}) VALUES ({1})".format(columnNames, ques), item)  
      
 if __name__ == '__main__':  
  pass  
Last edited by dgm5555 on Thu Feb 19, 2015 22:02, edited 4 times in total.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

I'm just wondering if it would be possible to create code which took an imported world, and blended the edges to match the mapgen. I've PMed one of the mapgen experts:-
I'm wondering (as you're a mapgen expert) if you would have time and inclination to create a script to smoothly blend the edges of an import into the mapgen landscape so as to avoid the massive cliffs, straight lines, biome jumps, and other texture disparities that usually occur with imports. I'm thinking it would also be useful for you more generally as a landscape smoother at the edges of villages/flats/etc.

twoelk
Member
Posts: 1482
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

Re: [Mod] Converter Minecraft world to Minetest world

by twoelk » Post

regarding moving parts of the map around and merging maps you may find this bash script from VanessaE and Sokomine interesting.
There was a discussion here about the problem of deleting parts of an existing map to make a transfer to a new server easier.
OldCoder did in the end succeed to delete the lower layers of the landrush map but I can"t remember him publishing how exactly he did it.

lag01/AndrejIT wrote this Map unexplore python script that might also include code of interest for you.

About merging map edges there was some discussion about that here: looping/wraparound/spherical/tiling world and the dev with the most experience in this matter is probably indeed paramat. I think there was more talk about merging edges elsewhere with some snippets of example code but I can't remember where, may have been on IRC.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Thanks for the thoughts a very simple way to clear portions of worlds would be to create extra x, y, and z columns from pos, then use a delete clause on the appropriate column (essentially I'm duplicating the minetest pos -> x,y,z code in SQL). PS This could be done considerably more efficiently, but it's reasonably quick as it is, and it's clearer to debug:
[PS DON'T TRUST THESE I HAVEN'T TEST THEM]
'ALTER TABLE blocks ADD COLUMN xPos INT NOT NULL'
'ALTER TABLE blocks ADD COLUMN yPos INT NOT NULL'
'ALTER TABLE blocks ADD COLUMN zPos INT NOT NULL'
'ALTER TABLE blocks ADD COLUMN iPos INT NOT NULL'
'UPDATE blocks SET xPos = (pos % 4096)'
'UPDATE blocks SET xPos = (CASE WHEN xPos < 2048 THEN xPos ELSE xPos-4096)'
'UPDATE blocks SET iPos = int((pos - xPos) / 4096))'
'UPDATE blocks SET yPos = (iPos % 4096)'
'UPDATE blocks SET yPos = (CASE WHEN yPos < 2048 THEN yPos ELSE yPos-4096)'
'UPDATE blocks SET iPos = int((iPos - yPos) / 4096))'
'UPDATE blocks SET zPos = (iPos % 4096)'
'UPDATE blocks SET zPos = (CASE WHEN zPos < 2048 THEN zPos ELSE zPos-4096)'
'DELETE FROM blocks WHERE yPos <-20'

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

To reveal unknown blocks as the typical minetest unknown node, change line 40 in content.py

Code: Select all

#    name_id_mapping = ["air"]
    name_id_mapping = ["unknown"]
and add to the top of map_content.txt

Code: Select all

0	air
This gives you the following effect
Image
Image


There also seems to be a glitch where content is imported correctly, but immediately deleted by the engine. It seems to always happen in the same locations, so is presumably related to the chunk not been correctly recognised as already generated.
eg the following series were taken at approx 1 second intervals.
Image
Image
Image
Last edited by dgm5555 on Fri Feb 20, 2015 07:54, edited 1 time in total.

Nore
Developer
Posts: 501
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: [Mod] Converter Minecraft world to Minetest world

by Nore » Post

That last bug is due to cavegen, you could try disabling caves in the mapgen flags.

dgm5555
Member
Posts: 245
Joined: Tue Apr 08, 2014 19:45

Re: [Mod] Converter Minecraft world to Minetest world

by dgm5555 » Post

Why does flagging the sections as 'generated' not work. I would have thought this should stop mapgen from modifying them (but it doesn't)
in block.py:-

Code: Select all

...
        #flags
        flags = 0x00
        if self.pos[1] < -1:
            flags |= 0x01       #is_underground
        flags |= 0x02           #day_night_differs
        flags |= 0x04           #lighting_expired
        flags |= 0x08           #generated
        writeU8(os, flags)
...
And deleting caves from map_meta.txt doesn't work either. According to http://dev.minetest.net/Mapgen_Parameters, I should be able to edit it, but minetest ignores any setting I save in the file and adds the flag back in. The file does say hardcoded preceeds all others, but that makes no sense that would mean you could never set any flags...
mg_flags = trees, **caves**, nodungeons, noflat, light

Nore
Developer
Posts: 501
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: [Mod] Converter Minecraft world to Minetest world

by Nore » Post

Try to modify that in a mod, with the function that alters mapgen params then... You should also add no_mudflow or something like that. Also, it may be necessary to completely generate the *chunks* to avoid mapgen regenerating them.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests