Multi Map proof of concept

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Multi Map proof of concept

by Beerholder » Thu Jul 12, 2018 23:02

Hi all, after some time off, I started coding/ modding for Minetest again yay :) This time I am looking into something I want to call [MOD] Multiple Map Layers [multi_map] which I want to use for a new subgame that I started working on a week or so ago. I am posting here first before the WIP mod section to discuss the idea a bit and get some feedback, and would like to have something more concrete before posting the mod in the WIP section :)

This very simple piece of code is still very alpha, and to just share the idea, have a working PoC people can try out. Lots of stuff is still missing and lots of work to do, but I wanted to get the basic concept working first, share the idea and a bit of code ...

https://github.com/evrooije/multi_map/b ... mapgen.lua

"Problem":

Standard mapgen generates stone all the way down and requires carving out subworld terrain instad of (re)using the same logic of terrain generation for all layers. To use the sky more effectively, again (different) mods or floatlands can be used but oceans are an issue and the complexity having to deal with multiple mods is increased. Many times, only a few layers (or a single one in case of the sky) are added and a lot of space on the Y axis is still not used effectively. There is not really a straightforward, simple way to increase the usable area of a world.

Proposal and goals:

Create a singlenode pure Lua map generator that allows generation of multiple layers of terrain and ocean stacked on the Y axis, or, multiple worlds if you will. To effectively use all the available space on the Y axis that would otherwise be occupied by stone and air, without having to go through a lot of effort. Each layer using the same logic to create the terrain, each layer having its own ocean and eventually biomes, ore generation, etc. To simulate a world area larger than the current "limitation" of 65536x65536 nodes.

E.g. using 8 layers will "increase the area" to 524288x524288 nodes, each layer being 8192 high. Using 16 layers offers an area of 1048576x1048576, each layer being 4096 high. Please note, this obviously will not (and cannot) offer a seamless transition from one layer to another and requires a travel mechanism from one layer to another.

Pics or it didn't happen, an extreme case of 1024 layers each 64 blocks high.

Image

I want to make something that will run out of the box, but also offer a framework that modders can easily customize or extend. Some of the additional things I have in mind (feel free to add any in the comments):

  • Every layer will have its own noise for the height map, giving each layer an unique terrain (next on my list)
  • Optional "bedrock" and a way to plugin existing bedrock mods by e.g. setting bedrock node type in module's configuration, other (optional) mechanisms to avoid normal player travel between layers
  • Optional invisible impenetrable layer to avoid seeing the bottom of the next layer up, as suggested by GreenDimond and hopefully to also avoid lighting problems as pointed out by duane
  • Hooks for custom terrain generation logic, or use the one supplied by the mod
  • Optional mechanism for travel between the worls (i.e. portals), but of course it can be switched off and subgame developers can implement their own travel mechanism or integrate existing portal mods
  • Customizable number of layers and an optional zoning system allowing sane display of coordinates (e.g. coordinates displayed as "Zone: 6; Position: 10.5, 15.0, -18.4" where the Y is offset based on the zone to show "sane" coordinates). This will not change the debug coordinates of course, just something to show on the HUD
  • Zone numbering (as per above) or zone naming (e.g. "Zone: Nether; Position: 10.5, 15.0, -18.4")
  • Pushing zone information as parameter for terrain generators, e.g. allowing different map generator code per layer (besides the obvious changes one can do with the perlin noise parameters), different biomes and decorations (e.g. jungle zone, desert zone, etc.), etc.
  • Pluggable decorators and ore generators, again with zone information supplied so it can be changed or tweaked per layer

And of course the usual: Caves, biomes (temperature/ humidity maps), dungeons, floatlands, etc.

Let me know what you think or if you have any comments!
Last edited by Beerholder on Sat Jul 14, 2018 23:51, edited 3 times in total.
 

User avatar
GreenDimond
Member
 
Posts: 1125
Joined: Wed Oct 28, 2015 01:26
Location: Yes.
GitHub: GreenXenith
IRC: GreenDimond
In-game: GreenDimond

Re: Multi Map proof of concept

by GreenDimond » Thu Jul 12, 2018 23:32

Something I have thought of many times for this sort of method is a "sky bedrock" sort of thing.
Basically, an invisible, non-pointable block that generates ~6 chunks below the bedrock of the next layer, thus making a fake "top of the world" that hopefully keeps the map above from generating and being viewed, and allows the sky to still be viewed.
Food for thought.
My YuTube channel | I moderate the HOMETOWN Server. | Click here to see my (6) mods! ~Using gradient signatures since 2017. ✂️- - - - -
 

User avatar
duane
Member
 
Posts: 1259
Joined: Wed Aug 19, 2015 19:11
Location: Oklahoma City
GitHub: duane-r

Re: Multi Map proof of concept

by duane » Fri Jul 13, 2018 02:56

Have you had any problems with lighting or shadow propagation? That always caused me headaches.
Believe in people and you don't need to believe anything else.
 

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Multi Map proof of concept

by Beerholder » Fri Jul 13, 2018 08:14

GreenDimond wrote:Something I have thought of many times for this sort of method is a "sky bedrock" sort of thing.
Basically, an invisible, non-pointable block that generates ~6 chunks below the bedrock of the next layer, thus making a fake "top of the world" that hopefully keeps the map above from generating and being viewed, and allows the sky to still be viewed.
Food for thought.


Yes absolutely crossed my mind too, some way to avoid seeing the bottom, and to be honest I think the invisible impenetrable layer is the way to go. Thanks :-)

@duane, I haven't run into those yet, the current PoC focuses on the layers and has full lighting. I need to turn normal lighting on again and see what it does. Hopefully the invisible layer suggested by GD and limited shadow propagation avoids that ... knock on wood O_o
 

User avatar
paramat
Developer
 
Posts: 3132
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat

Re: Multi Map proof of concept

by paramat » Sat Jul 14, 2018 19:41

Nice, this is something i have been interested in for a long time.

You can stop shadow propagation at any Y level using the bool in the Lua Voxel Manipulator https://github.com/minetest/minetest/blob/ca8ec46843da054e656d2f63b23d0b1695c023da/doc/lua_api.txt#L3041 set it to false for 1 layer of mapchunks at the border between stacked realms.

However that only stops mapgen shadows, it is possible (although unlikely) that building just under the border may cause the shadow of the upper realm to be propagated down, but only if you build under the border then build downwards.
This can be avoided by creating an inaccessible gap between levels at least a mapchunk (80 nodes) thick.
 

User avatar
paramat
Developer
 
Posts: 3132
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat
 

User avatar
GreenDimond
Member
 
Posts: 1125
Joined: Wed Oct 28, 2015 01:26
Location: Yes.
GitHub: GreenXenith
IRC: GreenDimond
In-game: GreenDimond

Re: Multi Map proof of concept

by GreenDimond » Sat Jul 14, 2018 20:00

paramat wrote:This can be avoided by creating an inaccessible gap between levels at least a mapchunk (80 nodes) thick.

Yes, this is what the "sky bedrock" idea would do. I don't know what the distance needs to be though, so if one mapchunk does the trick, great!
My YuTube channel | I moderate the HOMETOWN Server. | Click here to see my (6) mods! ~Using gradient signatures since 2017. ✂️- - - - -
 

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Multi Map proof of concept

by Beerholder » Sat Jul 14, 2018 23:38

paramat wrote:Some of my mods may be useful as Lua Mapgen examples, especially https://forum.minetest.net/viewtopic.php?f=11&t=10888


Awesome, thanks for the pointers and the link to your mod paramat! Will do some changes and testing with the propagate_shadow flag, have a good close look at this mod and sift through your Github for some more ;) P.s. floatlands inspired me to do this :) P.p.s. floatland support as an option has been on my mind to fill the gap above the terrain and below the next layer (as there still is unused space above the terrain, and quite a lot of you only use just a few layers ...)

Anyways, as for the latest changes, I have added bedrock and "skyrock" nodes. I separated everything properly with init.lua dofiling the mapgen and proper namespaces (at first I just modified the mapgen in default in minimal)

Right now it also uses x amount of noises for x amount of layers so each layer now has its own terrain map and unique terrain. Woohoo

I was at first a bit worried, but memory wise it seems to do okay knock on wood, in the range of 2.0 - 2.3 GB virtual so far. This remains to be seen when combined with more mods and humidity/ temperature noises of course... Keeping a close eye on that too O_o
 

User avatar
paramat
Developer
 
Posts: 3132
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat

Re: Multi Map proof of concept

by paramat » Mon Jul 16, 2018 00:27

Having a different 2D noise per layer is an issue i have come across in core mapgen.

A possible solution, no idea if it will work:
Have 1 noise for terrain but alter the seed for each layer, make the seed Y-dependent. To do this the 'noise object' will have to be redefined per mapchunk. Just have 1 noise object and re-initialise it every mapchunk, to avoid multiple noise objects using lots of memory.

Also see https://forum.minetest.net/viewtopic.php?f=18&t=19836 for latest advice on good practice in lua mapgens.
 

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Multi Map proof of concept

by Beerholder » Mon Jul 16, 2018 14:25

paramat wrote:Having a different 2D noise per layer is an issue i have come across in core mapgen.
<snip>
Also see viewtopic.php?f=18&t=19836 for latest advice on good practice in lua mapgens.


Mmmmh food for thought. I considered using a single 3D noise and only use the noise values of certain "Y slices" in the noise as a possible approach for having multiple 2D terrain maps, e.g. using a slice of y=0, then y=4096 (or 2048 depending on number of layers), etc.

On using a single 2D nose, altering the seed, I haven't even considered that ... I could keep track of the "latest used" noise layer and only re-initialize when the map chunk generated is different from the last one used/ missing from cache, but OTOH this does not work well multiplayer and not sure how much the overhead is in the initialization. I'll put some timers in my local version to see what it does.

But also, the more I am looking at the density gradient approach in order to support a single noise, the more I think it is a very good candidate for supporting multiple layers (since, if you go to the next layer, Y is so much different that it would render a completely different terrain anyways, I would expect).

I am now in the stage of decoupling mapgen code and looking into pluggable mapgens, so to retrofit one or more of your density gradient mapgens so that they can be plugged in would be a good test for the "extensibility part" of multi_map so to say :)

With a bit of luck it will just be a matter of using the "offset Y" (a.k.a. Y relative to a layer's Y center point) to the density calculation ... Knock on wood ... :)
 

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Multi Map proof of concept

by Beerholder » Tue Jul 17, 2018 09:16

Jeez ... Whoever thought a world side length of 61840 was a good idea ... What's wrong with 65536 -_-

Sorry, just ranting
 

User avatar
Krock
Developer
 
Posts: 3989
Joined: Thu Oct 03, 2013 07:48
Location: Switzerland
GitHub: SmallJoker

Re: Multi Map proof of concept

by Krock » Tue Jul 17, 2018 09:35

@Beerholder Please check out how the map generator works before ranting. Instead of generating 16³ nodes (mapblocks), it generates 5³ mapblocks at once, which are then called "chunk". This may makes the map generator look a bit slow, but is more efficient at the end because less contents have to be written outside the current mapblock (schematics, like trees).
To ensure the map generator doesn't run into a coordinate overflow and for simplicity, the map is limited to 31000m in each direction. Since 31000 is no multiple of 5 (chunksize) * 16 (blocksize), it's a little bit smaller than that.
Change the constant (and maybe also your map_meta.txt setting) if you really want to see what happens with 0x7FFF.
Mod Search Engine - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
 

User avatar
Beerholder
Member
 
Posts: 170
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Multi Map proof of concept

by Beerholder » Tue Jul 17, 2018 10:40

Nah don't want to change that, as I know it could break things exactly due to potential missing bounds checks and the possibility of overflows, it not being compatible with standard MT configurations users may have, etc. Not gonna go there... Problem is that my original idea was to just use nice multiples, 2, 4, 8, 16, 32 layers, and generate these from e.g. -2048 to 2048, 2048 to 6144, etc.

This worked perfectly when I had everything (bedrock, skyrock, terrain) in a single loop that strided on the Y axis and generated the bedrock/ skyrock, though with a minor misalignment near the world borders.

I am now rewriting it to separate bedrock/ skyrock generation and the terrain generation. I thought it would be handy and efficient to generate the bedrock/ skyrock from minp to maxp in order to avoid having two loops iterating over zyx (one for bedrock/ skyrock and a separate one for terrain). This is where I immediately ran into the issue of misalignment of course.

It's too late for this at this stage of engine development obviously, but as to writing outside of world boundaries, a bounds check is the easiest thing there is... 4^3 or 8^3 are options for number of mapblocks to generate, that would align with sane 16 bits. Also, sorry to say, but limiting the world size just to avoid these overflows and for simplicity sounds like "lazyness" to me ;-) I'm sure though the code was put together quickly in order to get something to work, possibly 5^3 worked better for the lower end devices at that time, other things may not have knowledge of and at some stage it's not worth changing. And I guess I just like my powers of 2 :-(

Alas, MT engine is the way it is, and I am already working around it. Just need to rewrite to stack the layers going from -30912 (or user specified mapchunk) up and allow the user to specify height (in mapchunks) and number of layers to stack.

*Including a bounds check* to avoid stacking above 30927 X-))> (sorry for rubbing it in ;-)
 


Return to Modding Discussion



Who is online

Users browsing this forum: No registered users and 1 guest