Real world terrain (mapgen idea)
Real world terrain (mapgen idea)
I was just kinda thinking about the current mapgen, and the issues associated with the current default mapgen, and just kinda wondered if we could have the mapgen based off real locations. My thoughts are these: topographical maps exist with the hights already in them. Would it be too hard to have some code to turn that map into real terrain? it could have a co-ordinate set point with longitudes and latitudes. example: 38.9536 and 120.0939 with an altitude correction of 1,890 meters to generate 0,0,0. just throwing this idea out there, hoping to inspire someone who works on mapgen.
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Not difficult at all. However, it might be worth considering scale. Minetest's coordinates span from like -32k to +32k in each dimension I believe, which if you consider a scale of 1m per node would only give you a 64km square area to play with. Unless you don't use a 1:1 scale, of course.
It would be interesting to also use real geological data if it could be found to create the underground strata and ore deposits (probably with a decent amount of fudging).
It would be interesting to also use real geological data if it could be found to create the underground strata and ore deposits (probably with a decent amount of fudging).
I like this idea, I would like to see it implementedNovacain wrote:I was just kinda thinking about the current mapgen, and the issues associated with the current default mapgen, and just kinda wondered if we could have the mapgen based off real locations. My thoughts are these: topographical maps exist with the hights already in them. Would it be too hard to have some code to turn that map into real terrain? it could have a co-ordinate set point with longitudes and latitudes. example: 38.9536 and 120.0939 with an altitude correction of 1,890 meters to generate 0,0,0. just throwing this idea out there, hoping to inspire someone who works on mapgen.
sorry for bad english
Linux debian 7 wheezy 64
kde
Linux debian 7 wheezy 64
kde
-
- Member
- Posts: 1482
- Joined: Fri Apr 19, 2013 16:19
- GitHub: twoelk
- IRC: twoelk
- In-game: twoelk
- Location: northern Germany
as has been done for mc
here using OrdnanceSurvey OpenData?
Check this developing bitmap mapgen mod that might use hightmaps
and the older Image Loader by Kaeza that Mauvebic has used to import map outlines.
here using OrdnanceSurvey OpenData?
Check this developing bitmap mapgen mod that might use hightmaps
and the older Image Loader by Kaeza that Mauvebic has used to import map outlines.
Scale is a good point. real world proportions tend to be a little... large for a voxel sandbox game like minetest. so maybe account for a 3:1 scale? wouldn't want to remove too muchprestidigitator wrote:Not difficult at all. However, it might be worth considering scale. Minetest's coordinates span from like -32k to +32k in each dimension I believe, which if you consider a scale of 1m per node would only give you a 64km square area to play with. Unless you don't use a 1:1 scale, of course.
It would be interesting to also use real geological data if it could be found to create the underground strata and ore deposits (probably with a decent amount of fudging).
The code could also probably be used to create life size if someone happened to want to make a Mt. Everest map to climb it (hey, that could be a game!!).
that is similar, but it looks like it lacks the ability to make an actual playable world. the size on it is such that it seems kind of pointless.twoelk wrote:as has been done for mc
here using OrdnanceSurvey OpenData?
Check this developing bitmap mapgen mod that might use hightmaps
and the older Image Loader by Kaeza that Mauvebic has used to import map outlines.
Last edited by Novacain on Tue Jan 28, 2014 17:37, edited 1 time in total.
- rubenwardy
- Moderator
- Posts: 6978
- Joined: Tue Jun 12, 2012 18:11
- GitHub: rubenwardy
- IRC: rubenwardy
- In-game: rubenwardy
- Location: Bristol, United Kingdom
- Contact:
I started writing a map generator like this for Minetest about a month ago, but the api does not support the connecting to the internet.
It used data from maps.google.com - they have a nice API.
It used data from maps.google.com - they have a nice API.
Last edited by rubenwardy on Tue Jan 28, 2014 17:41, edited 1 time in total.
would it be possible to download the map to solve the connection issue? I really wish I knew more about programming so I could contribute.rubenwardy wrote:I started writing a map generator like this for Minetest about a month ago, but the api does not support the connecting to the internet.
It used data from maps.google.com - they have a nice API.
- BrandonReese
- Member
- Posts: 839
- Joined: Wed Sep 12, 2012 00:44
- GitHub: bremaweb
- IRC: BrandonReese
- In-game: BrandonReese
- Location: USA
I think you would need to write a separate program/script to download all of your data from whatever source you choose and parse it into a usable format (maybe a sqlite database?), and then feed that info to the mapgen.
My Twitch Channel - https://www.twitch.tv/brandonreese81/profile
- philipbenr
- Member
- Posts: 1897
- Joined: Fri Jun 14, 2013 01:56
- GitHub: philipbenr
- IRC: philipbenr
- In-game: robinspi
- Location: United States
Yes, certainly the "magpen" king . And I did talk to him. using real-world data isn't very interesting to him. I have the feeling it's not complicated enough for him (based on his previous amazing projects).philipbenr wrote:Talk to paramat, the magpen king. After you figure out exactly what you have in mind, tell him, and if he likes it, he probably could whip out something awesome.Novacain wrote:Just throwing this idea out there, hoping to inspire someone who works on mapgen.
Re: Real world terrain (mapgen idea)
There is a format of elevation data called DEM which is already in a raster form. Publicly available DEMs are usually 10m or 30m resolution with elevation resolution to 1m. However, translating one pixel of DEM to one minetest voxel might be a simple way to scale the real world data.
There are also freely available land cover data sets in raster form, which could be used to create the land cover and tree distribution. digital scans of topo maps might also be used for this, but correlating the spatial reference system of one dataset to another can be tricky.
As for underground strata, there are datasets for this too, but probably is overkill for a fun game. However, there aren't (m)any mapgens with "strata" in minetest, the focus seems to be more about caves, etc. exposed rock faces with strata visible would be cool.
But I have been interested in creating a mod or simply a tool to load DEM data into .mts format for some time now. Freely available elevation data may be available from google or OSM but it would require further processing since it is in vector format. DEMs are so well suited for block world games as they are already rasters. Also, you would not need to worry about the spatial reference if all you wanted was real-world elevation data. as soon as you access OSM or other datasets, or attempt to combine them, you will need GIS libraries to "sync" them or to convert between real-life spatial coordinates and game pixels -- DEMs are usable as-is for initial experimentation.
canadian dem data is available at: http://geogratis.gc.ca/api/en/nrcan-rnc ... E738E.html
in the US: http://earthexplorer.usgs.gov/
a freeware program called MicroDEM is useful for automatically downloading and merging DEMs:
http://www.usna.edu/Users/oceano/pguth/ ... crodem.htm
an example of a vegetation dataset already in raster format but only at 250m resolution might be useful for determining BIOMES if the spatial reference is the same as your elevation data: from MODIS: http://modis.gsfc.nasa.gov/data/dataprod/mod13.php
anyway, I would be willing to help with any project along these lines, though my lua and minetest modding is only in its infancy at this point.
There are also freely available land cover data sets in raster form, which could be used to create the land cover and tree distribution. digital scans of topo maps might also be used for this, but correlating the spatial reference system of one dataset to another can be tricky.
As for underground strata, there are datasets for this too, but probably is overkill for a fun game. However, there aren't (m)any mapgens with "strata" in minetest, the focus seems to be more about caves, etc. exposed rock faces with strata visible would be cool.
But I have been interested in creating a mod or simply a tool to load DEM data into .mts format for some time now. Freely available elevation data may be available from google or OSM but it would require further processing since it is in vector format. DEMs are so well suited for block world games as they are already rasters. Also, you would not need to worry about the spatial reference if all you wanted was real-world elevation data. as soon as you access OSM or other datasets, or attempt to combine them, you will need GIS libraries to "sync" them or to convert between real-life spatial coordinates and game pixels -- DEMs are usable as-is for initial experimentation.
canadian dem data is available at: http://geogratis.gc.ca/api/en/nrcan-rnc ... E738E.html
in the US: http://earthexplorer.usgs.gov/
a freeware program called MicroDEM is useful for automatically downloading and merging DEMs:
http://www.usna.edu/Users/oceano/pguth/ ... crodem.htm
an example of a vegetation dataset already in raster format but only at 250m resolution might be useful for determining BIOMES if the spatial reference is the same as your elevation data: from MODIS: http://modis.gsfc.nasa.gov/data/dataprod/mod13.php
anyway, I would be willing to help with any project along these lines, though my lua and minetest modding is only in its infancy at this point.
-
- Member
- Posts: 4290
- Joined: Sun Sep 09, 2012 17:31
- GitHub: Sokomine
- IRC: Sokomine
- In-game: Sokomine
Re: Real world terrain (mapgen idea)
Yes, that would be nice. There are sometimes large mapgen artefacts where huge stone cliffs are visible due to either mapgen change, change of parameters, or errors in the mapgen. Such places could be produced deliberately and show diffrent colored rocks. Might even be useful as a ressource for digging it up and building more colorful buildings with it. I think there's at least one mod in development that aims at that direction.bobbomb wrote: As for underground strata, there are datasets for this too, but probably is overkill for a fun game. However, there aren't (m)any mapgens with "strata" in minetest, the focus seems to be more about caves, etc. exposed rock faces with strata visible would be cool.
With voxelmanip, it boils down to a bit of code you can copy from other mods plus manipulating a kind of 3d array. That in theory allows you to do all you might ever want. However, the needs of terrain for gameplay may differ considerably from RL terrain. Maybe the easiest way would be to take a flat mapgen, add the occasional river and proclaim that it's terrain from a flat region. Larger geological structures like mountains or mountain ranges are invisible due to not enough land beeing loaded in the client.bobbomb wrote: anyway, I would be willing to help with any project along these lines, though my lua and minetest modding is only in its infancy at this point.
A list of my mods can be found here.
Re: Real world terrain (mapgen idea)
Re: strata. One way could be to make a flat map, with layers of stone, clay, coal, etc, then the map could be lifted and sunk to the surface levels desired, which would expose the strata, and create runs and veins of minerals in a novel way. I'm not as interested in new mineral types, yet, this is just a mapgen idea.Such places could be produced deliberately and show diffrent colored rocks.
This misses the point of this thread entirely. The idea is to get actual real world terrain into the game, not to tweak existing mapgen settings. (Time spent in a virtual world could actually have the incidental benefit of familiarizing yourself with real world topography.)However, the needs of terrain for gameplay may differ considerably from RL terrain. Maybe the easiest way would be to take a flat mapgen, add the occasional river and proclaim that it's terrain from a flat region.
Frankly the notion of using a heightmap to set the game map surface height should be getting everyone's panties wet, whether for importing real world terrain or having an instant "level editor" for minetest with nothing more than ms paint: one texture for elevation, one for surface cover (simple color code for biome, water source, road and village coverage), and even one for mineral probability below ground.
These issues of scale are easy to address. The real world DEM pixels can be scaled to represent only a single in-game square, a 16x16 block, or whatever size you want to handle with voxelmanip. Also, vertical relief can be exaggerated. A common tactic with DEMs is to "normalize" the elevation ranges, so that whatever the range is, it is spread to a max and min value, with or without clipping. Another simple solution could just be a vertical exaggeration factor, where 1m real world elevation would mean 3m in game height.Larger geological structures like mountains or mountain ranges are invisible due to not enough land beeing loaded in the client.
help needed
What is really needed next is a way to read large files into the mapgen, without loading the entire file, in order to set the surface height. This lua function might help: http://www.gamedev.net/topic/572784-lua ... try4662735
Then I have no idea where the mapgen sets the final surface height. Is it something that is set first then filled below, or is it set as the result of some other algorithm? I might use the novagen mod as a basis for this project: https://github.com/Novatux/mg/
Another possibility that bypasses mapgen could be adding a worldedit command that can read from the heightmap/DEM.
- paramat
- Developer
- Posts: 3700
- Joined: Sun Oct 28, 2012 00:05
- GitHub: paramat
- IRC: paramat
- Location: UK
Re: Real world terrain (mapgen idea)
If you get stuck i could edit a quick simple mapgen for you, then just add your own line of code that returns a height for a given (x, z) co-ordinate.
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Real world terrain (mapgen idea)
You could break them up into files with name patterns related to what areas they cover (map_0_0, map_0_1, map_1_0, map_1_1, etc.). Or, for raw data (and probably uncompressed image data like BMP files too, but I can't remember right now) you can seek within files and read a limited amount of data using Lua's file API (see file:seek() and file:read(); might need to io.open in binary mode to be as portable across platforms as possible).bobomb wrote:What is really needed next is a way to read large files into the mapgen, without loading the entire file, in order to set the surface height. This lua function might help: http://www.gamedev.net/topic/572784-lua ... try4662735
Yeah, that is definitely a problem. I think it comes up particularly when automatically spawning players that have died, as I suspect the fact that they sometimes spawn underground for custom mapgens indicates its using the ground height behind the scenes. I don't think there's any way to set the ground height from Lua. You can get the ground height using a "heightmap" mapgen object, but I don't think you can set it. This is still quite an obstacle to full Lua mapgens.bobomb wrote:Then I have no idea where the mapgen sets the final surface height. Is it something that is set first then filled below, or is it set as the result of some other algorithm? I might use the novagen mod as a basis for this project: https://github.com/Novatux/mg/
Re: Real world terrain (mapgen idea)
paramat has provided me with a basic magen mod that I have used to get some initial results. (the code is in this gist: https://gist.github.com/paramat/389ee65ce9378d9dad25 It. makes things like setting heights, water levels, and the ease with which strata could be implemented plain as day.
My main hurdles right now are properly decoding the binary .tiff (geotiff) file format. I found some code to decypher the header, and some other code to interpret 16-bit signed integers, but I am still working out some details. I would really like to overcome this, because I think image maps for things like biome variables (moisture, heat, etc) and other ground-cover considerations would be awesome, but if it gets too complicated, I might switch to using the plaintext .dem format. .dem files cover much less area than the mosaiced geotiff I am using, which covers 9601m east/west and 4801m north/south and encompasses the entire "e" tile of the 082 section of the southern BC freely available dem data. I am doing a 1:1 pixel scaling, so the terrain is exaggerated by a factor of about 30:1. There are steep areas on the map where I think i might be interpreting the 16-bit value incorrectly. Switching to a system like presti mentioned where the mod is able to switch to different .dem tiles as we move further from the origin might mean that using the .dem file format might be better for this project than geotiff. at this time the origin of the map is the center of the tiff tile. no effort has been made to translate the real world coordinates yet. This information needs to be extracted from the header. also because the actual pixels are sometimes rotated to the real world, the edges of .dem and geoTiffs are often packed out with "transparent" pizels set to -32767 which would need to be handled from the next tile.
the code for this is at: https://github.com/bobombolo/realterrain
this is very experimental code, maybe someone can help me write a better bitmap reading function
Re: Real world terrain (mapgen idea)
more progress...
having to discard every other row, and still having major incongruity at z = 1203, 3 and -1199
having to discard every other row, and still having major incongruity at z = 1203, 3 and -1199
Re: Real world terrain (mapgen idea)
I have discarded the 16-bit dem and reduced it to 8-bit, this makes the dem tiff substantially smaller in size and eliminates all the complications I was having with regard to interpreting the two byte elevation (which was basically getting scaled out anyway). Here is the result, with no incongruities that I have found:
-
- Member
- Posts: 1482
- Joined: Fri Apr 19, 2013 16:19
- GitHub: twoelk
- IRC: twoelk
- In-game: twoelk
- Location: northern Germany
Re: Real world terrain (mapgen idea)
so what part of the real world are we seeing or are these simple fictional hightmaps?
will you offer maps like these? TopoMC: Minecraft Worlds From Real Data
or something combined with OSM data? Minecrafting with OS OpenData You could use the OSM data to simply drop roads onto the otherwise generated terrain.
Anyways a simple way to import pixelmap hightmaps, ficional or not, would really be nice
will you offer maps like these? TopoMC: Minecraft Worlds From Real Data
or something combined with OSM data? Minecrafting with OS OpenData You could use the OSM data to simply drop roads onto the otherwise generated terrain.
Anyways a simple way to import pixelmap hightmaps, ficional or not, would really be nice
Re: Real world terrain (mapgen idea)
I like how the mountains go through the clouds :)
Re: Real world terrain (mapgen idea)
This is not fictional, this is southern British Columbia. As for importing data from an online service, I think the hurdle there is the ability to access the internet through the Lua scripting API. Not sure that is possible, at least not in a cross-platform way. It's just what I've heard (top of this thread?).twoelk wrote:so what part of the real world are we seeing or are these simple fictional hightmaps?
will you offer maps like these? TopoMC: Minecraft Worlds From Real Data
or something combined with OSM data? Minecrafting with OS OpenData You could use the OSM data to simply drop roads onto the otherwise generated terrain.
Anyways a simple way to import pixelmap hightmaps, ficional or not, would really be nice
At this point, I am going to make nice normalized DEMs for some of the major cities and geological formations of North America (Vancouver, The Grand Canyon, New York, Bent Armpit Wyoming), then they can be chosen by typing their name in the "SEED" section when you create a new map. In the same way you can drop any of your own heightmaps in the mod folder and put the filename in the seed field and voila.
As for roads and landcover, I am going to experiment with an indexed-color bitmap for doing things like "dropping roads" and rivers and determining the vegetation a little bit. that way a heightmap and a colormap that cover the same extent could do the job you're looking for, albeit offline (albeit, possibly fictional, unlike OSM data)...
Re: Real world terrain (mapgen idea)
just messin around:
- paramat
- Developer
- Posts: 3700
- Joined: Sun Oct 28, 2012 00:05
- GitHub: paramat
- IRC: paramat
- Location: UK
Re: Real world terrain (mapgen idea)
Good stuff.
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Real world terrain (mapgen idea)
So don't, then. Create a separate daemon that either works on its own or communicates with the Minetest mod through the filesystem (e.g. requests for data go in one location, and downloaded data goes in another).bobomb wrote:As for importing data from an online service, I think the hurdle there is the ability to access the internet through the Lua scripting API. Not sure that is possible, at least not in a cross-platform way. It's just what I've heard (top of this thread?).
Who is online
Users browsing this forum: No registered users and 2 guests