Minetest map generation using GPU

[asdf7389]

So currently I'm working on using CUDA to parallel map generation on minetest for my senior project. I'm fairly new to parallel programming, as well as C++ so it's been quite a challenge so far. I'm looking at the mapgen files and comparing to the process of how it runs when a users clicks the create new world button. Any suggestions on a focal point in what takes the longest time, taking aside rendering speeds. My main focus is the map generation. More info to come as I progress.

[rarkenin]

So currently I'm working on using CUDA to parallel map generation on minetest for my senior project. I'm fairly new to parallel programming, as well as C++ so it's been quite a challenge so far. I'm looking at the mapgen files and comparing to the process of how it runs when a users clicks the create new world button. Any suggestions on a focal point in what takes the longest time, taking aside rendering speeds. My main focus is the map generation. More info to come as I progress.

I think that the GPU should be usable for map generation, which is really important, but this may be relatively difficult as Lua would need to compile down to GPU instructions.

[asdf7389]

I was imagining making a file outside of Minetest to generate a world then dump it back into Minetest to explore, but I still don't fully understand how mapgen outputs it's initial file. Still just pouring through source code and beating wolves back with a stick.

[Calinou]

So currently I'm working on using CUDA to parallel map generation on minetest for my senior project. I'm fairly new to parallel programming, as well as C++ so it's been quite a challenge so far. I'm looking at the mapgen files and comparing to the process of how it runs when a users clicks the create new world button. Any suggestions on a focal point in what takes the longest time, taking aside rendering speeds. My main focus is the map generation. More info to come as I progress.

Why not OpenCL? CUDA is NVIDIA-only and it's a PITA to set it up on Linux.

Is there really any noticeable performance improvement by using the GPU for map generation? I bet the gains would be small since it's fast enough already. Feel free to make this just for fun, though.

[asdf7389]

Why not OpenCL? CUDA is NVIDIA-only and it's a PITA to set it up on Linux.

Is there really any noticeable performance improvement by using the GPU for map generation? I bet the gains would be small since it's fast enough already. Feel free to make this just for fun, though.

Well it's in CUDA because my project sponsor suggested that its optimized for my current hardware, however the code is easily translated from CUDA to OpenCL. I'm also trying to generate a maximum mapsize file at once vs progressive generation.

[rarkenin]

I was imagining making a file outside of Minetest to generate a world then dump it back into Minetest to explore, but I still don't fully understand how mapgen outputs it's initial file. Still just pouring through source code and beating wolves back with a stick.

It's just a SQLite database. Take a look at the MapBlock serialization methods, and look for the way they determine block ID.

[asdf7389]

Well finding time to work on this, I've decided to see about generating a map that hits the max size in a square fashion. Looking through to see if initial generation can be stretched to accommodate generation of the entire map at once.

[Jordach]

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

[asdf7389]

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

Hmm...I thought there was a built in limit for the max size of a map file that can be generate in Mintest. Wondering if I can softcap it around 4GB if not.

[asdf7389]

So at this point, I agree that manipulating the sqlite file may be volatile, so I've shifted my focus to just modifying the map node and map generation to be ported to a GPU implementation, I just wish I could figure out the key class and method I need to focus on, because all of it seems to be relevant;.

[Likwid H-Craft]

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

That will be needing a new pc HDD for some so I think it not that great have up to 1TB.

[rarkenin]

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

That will be needing a new pc HDD for some so I think it not that great have up to 1TB.

No, you hit filesystem limits and instability first. Please don't bring up the obvious, especially when there is something more important than it.

[sfan5]

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

Assuming every block is one byte its 216 TB.
As every block has a light level, param1 and param2 that is 4 bytes per block.
That makes 864 TB.
But since almost 50% of the map is air (~ 0 -- +30 000) and 30% of the rest should be easily compressable, a complete minetest map would occupy 172.8 TB.
Comparisons by Wolfram Alpha:

• ~~ ( 0.086 ~~ 1/12 ) × identifiable storage capacity of the human brain (~~ 2 PB )
• ~~ ( 0.086 ~~ 1/12 ) × approximate data content of the Internet Archive Wayback Machine as of 2006 (~~ 2 PB )
• ~~ estimated data content of the surface web (~~ 170 TB )

That file size also shouldn't break any system as it is expressable (in bytes) with a 64-bit integer (signedness doesn't matter)