[Mod] Mapgen from geographical data [geo-mapgen] (Biomes!)
- TumeniNodes
- Member
- Posts: 2941
- Joined: Fri Feb 26, 2016 19:49
- GitHub: TumeniNodes
- IRC: tumeninodes
- In-game: TumeniNodes
- Location: in the dark recesses of the mind
- Contact:
Re: [Mod] Map generation from DEM [geo-mapgen] (WIP)
Great work Gael.
The new screenshots are really nice, especially the first one, it's beautiful.
The new screenshots are really nice, especially the first one, it's beautiful.
A Wonderful World
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
I'm trying to learn to use QGIS and related Python libs that seems very interesting for what I'm trying to do.
QGIS is the only application I've found that has managed to open huge TIFF files without literally burning my RAM.
The second step is to know what to do with the loaded image.
QGIS is the only application I've found that has managed to open huge TIFF files without literally burning my RAM.
The second step is to know what to do with the loaded image.
Last edited by Gael de Sailly on Sun Jan 21, 2018 23:37, edited 1 time in total.
Just realize how bored we would be if the world was perfect.
- TumeniNodes
- Member
- Posts: 2941
- Joined: Fri Feb 26, 2016 19:49
- GitHub: TumeniNodes
- IRC: tumeninodes
- In-game: TumeniNodes
- Location: in the dark recesses of the mind
- Contact:
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
Looks promising, and like a really great project (meaning QGIS)
Do you think the mtmapprune could be incorporated somehow, eventually to scale large maps?
viewtopic.php?f=14&t=18579
I fully understand it is probably way to early to know, I'm just wondering.
Do you think the mtmapprune could be incorporated somehow, eventually to scale large maps?
viewtopic.php?f=14&t=18579
I fully understand it is probably way to early to know, I'm just wondering.
Last edited by TumeniNodes on Sun Jan 28, 2018 21:24, edited 1 time in total.
A Wonderful World
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
There are many possibilities with a mod like this. Why, I could end up with a map based on the mountainous terrain surrounding Great Cacapon, WV.
[BitChute: https://www.bitchute.com/channel/fCcBQxrYQjNX/] [Rumble: https://rumble.com/user/HPoorHMagentaHChildH]
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
I found something interesting with GDAL library. It's able to crop, resample and change the projection of a raster image (through function "Warp"). It could allow using several images of different projections, and resample them to use only the part of the world we want.
Potentially good data source for biomes: Corine Land Cover (in Europe). I suppose (and I hope) there are equivalent data for other parts of the world. IIRC bobomb found one for United States when he developed RealTerrain some years ago.
Potentially good data source for biomes: Corine Land Cover (in Europe). I suppose (and I hope) there are equivalent data for other parts of the world. IIRC bobomb found one for United States when he developed RealTerrain some years ago.
Just realize how bored we would be if the world was perfect.
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
Today I've worked on graphical interface:
Code is on tkinter branch. EDIT: merged
You need tkinter library to use it.
Command to start on graphical mode:
For now only the input is on GUI. The output is still on the console.
Code is on tkinter branch. EDIT: merged
You need tkinter library to use it.
Command to start on graphical mode:
Code: Select all
./image_convert.py -g
Just realize how bored we would be if the world was perfect.
- TumeniNodes
- Member
- Posts: 2941
- Joined: Fri Feb 26, 2016 19:49
- GitHub: TumeniNodes
- IRC: tumeninodes
- In-game: TumeniNodes
- Location: in the dark recesses of the mind
- Contact:
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
Very very nice Gael de Sailly
this makes it very much easier to work with
this makes it very much easier to work with
A Wonderful World
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
I finally managed to get my Python script working with the GDAL library. It was difficult. I've pushed it on a new branch: https://github.com/Gael-de-Sailly/geo-mapgen/tree/gdal.
GDAL allows to manipulate and change the projection of geographical data like the DEMs this script uses.
I did this because it's an essential feature for a more long-run project: using biome maps like Corine Land Cover, superimposed on the heightmap, that may be in a different projection.
The script as it is now could deal with several maps in different projections with very few more work.
GDAL allows to manipulate and change the projection of geographical data like the DEMs this script uses.
I did this because it's an essential feature for a more long-run project: using biome maps like Corine Land Cover, superimposed on the heightmap, that may be in a different projection.
The script as it is now could deal with several maps in different projections with very few more work.
Just realize how bored we would be if the world was perfect.
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
New branch with good changes: it's now able to handle land cover images!
https://github.com/Gael-de-Sailly/geo-m ... /landcover
For now only the ground nodes are changed, but decorations are planned.
See the result on the Mont Blanc (I've not taken the time to make more good screenshots for now)
However, it's quite complicated to generate such maps. You need to find an heightmap like before, and also a raster land cover image covering the same region (this is already hard to find). But I faced a major problem: land cover maps are not standardized, and according to the organization who provide the map, the map legend can be different. You must write a file that associates each value with one of the biomes defined in biomes.csv. See the file "clc.lct" that matches Corine Land Cover.
I'll explain this in more details when it will be finished. I think I should even make a complete tutorial for a very special mod like this one, but that's something hard to do for me. I always have difficulties in explaining my work to others.
https://github.com/Gael-de-Sailly/geo-m ... /landcover
For now only the ground nodes are changed, but decorations are planned.
See the result on the Mont Blanc (I've not taken the time to make more good screenshots for now)
However, it's quite complicated to generate such maps. You need to find an heightmap like before, and also a raster land cover image covering the same region (this is already hard to find). But I faced a major problem: land cover maps are not standardized, and according to the organization who provide the map, the map legend can be different. You must write a file that associates each value with one of the biomes defined in biomes.csv. See the file "clc.lct" that matches Corine Land Cover.
I'll explain this in more details when it will be finished. I think I should even make a complete tutorial for a very special mod like this one, but that's something hard to do for me. I always have difficulties in explaining my work to others.
Just realize how bored we would be if the world was perfect.
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
News of the day, on landcover branch:
Trees and plants, according to the biome!
Look at the Mont Blanc now:
Trees and plants, according to the biome!
Look at the Mont Blanc now:
Just realize how bored we would be if the world was perfect.
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
This looks great!
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Mount Blanc looks nice, can you make it 1:1 scale? That will be epic for me, I will waste my time to conquer it :D
- ExeterDad
- Member
- Posts: 1717
- Joined: Sun Jun 01, 2014 20:00
- In-game: ExeterDad
- Location: New Hampshire U.S.A
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
This is very exciting stuff! Although I wouldn't let players dig a thing. It's too pretty.
HOMETOWN -Our little server. Keep the HOMETOWN chatter @ http://hometownserver.com - Our server map: http://media.hometownserver.com
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Unfortunately I don't think you can find free data that are accurate enough. The best I know is Copernicus EU-DEM 1.1 (resolution 25m). It's possible to rescale it to 1 node = 1m, but it would be completely smoothed, and you will not see far.Fixerol wrote:Mount Blanc looks nice, can you make it 1:1 scale? That will be epic for me, I will waste my time to conquer it :D
Just realize how bored we would be if the world was perfect.
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
I'm fine with approximated version
- azekill_DIABLO
- Member
- Posts: 7507
- Joined: Wed Oct 29, 2014 20:05
- GitHub: azekillDIABLO
- In-game: azekill_DIABLO
- Location: OMICRON
- Contact:
Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)
I have some French docs on Qgis because I used it a while ago! PM me if you need help!Gael de Sailly wrote:QGIS is the only application I've found that has managed to open huge TIFF files without literally burning my RAM.
Gone, but not dead. Contact me on discord: azekill_DIABLO#6565
DMs are always open if you want to get in touch!
DMs are always open if you want to get in touch!
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Thanks, but I have found what I was looking for, with GDAL (Geospatial Data Abstraction Library). It's the library used by Qgis to open raster layers. The python script is currently using it, works well.azekill_DIABLO wrote:I have some French docs on Qgis because I used it a while ago! PM me if you need help!
Just realize how bored we would be if the world was perfect.
-
- New member
- Posts: 5
- Joined: Sun Jan 29, 2017 10:05
- GitHub: dkartaschew
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Excellent mod. Been able to do a few areas with data from SRTM, however encountered an area where it fails.
I've posted bug reports to github for tracking. https://github.com/Gael-de-Sailly/geo-mapgen/issues
I've posted bug reports to github for tracking. https://github.com/Gael-de-Sailly/geo-mapgen/issues
- BirgitLachner
- Member
- Posts: 393
- Joined: Thu May 05, 2016 10:18
- In-game: Bibs
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Very nice ... I was already looking for Real Terrain but I need the data.
- TumeniNodes
- Member
- Posts: 2941
- Joined: Fri Feb 26, 2016 19:49
- GitHub: TumeniNodes
- IRC: tumeninodes
- In-game: TumeniNodes
- Location: in the dark recesses of the mind
- Contact:
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
Gael de Sailly,
Thought you might be interested in this:
https://www.michaelfogleman.com/projects/terrarium/
Thought you might be interested in this:
https://www.michaelfogleman.com/projects/terrarium/
A Wonderful World
- ChimneySwift
- Member
- Posts: 320
- Joined: Fri Sep 22, 2017 06:46
- GitHub: ChimneySwift
- IRC: ChimneySwift
- In-game: ChimneySwift
- Location: 127.0.0.1
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
A spoon is basically a tiny bowl with a stick on it
- Gael de Sailly
- Member
- Posts: 845
- Joined: Sun Jan 26, 2014 17:01
- GitHub: gaelysam
- IRC: Gael-de-Sailly
- In-game: Gael-de-Sailly gaelysam
- Location: Voiron, France
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
It fails to decompress data while reading a chunk. There is 2 reasons I can see: there may be an error in heightmap.dat file structure, or the mod is trying to read the file from the wrong position.
What map were you using?
What map were you using?
Just realize how bored we would be if the world was perfect.
- ChimneySwift
- Member
- Posts: 320
- Joined: Fri Sep 22, 2017 06:46
- GitHub: ChimneySwift
- IRC: ChimneySwift
- In-game: ChimneySwift
- Location: 127.0.0.1
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
I tried both of the demo heightmaps and a height map I generated from a section of Australia from http://srtm.csi.cgiar.org/SELECTION/inputCoord.aspGael de Sailly wrote:It fails to decompress data while reading a chunk. There is 2 reasons I can see: there may be an error in heightmap.dat file structure, or the mod is trying to read the file from the wrong position.
What map were you using?
A spoon is basically a tiny bowl with a stick on it
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
It's great to see that you have taken this approach. In fact I had abandoned realterrain because I came to realize that the better approach was indeed to do something like this where you use QGIS or python GDAL to prebuild the map using that minetest-lib python library, but that had issues too and your idea to build an intermediate database format that the lua mapgen can read directly is probably the only way this will ever work. I also realized that realterrain would have to give up on using the in-game form-based mapgen settings tool I had built, which was actually a huge part of the work, for an out-of-game interface. I tinkered with the idea of building a form in tkinter or something to simulate a "worldpaint" type of approach, but ultimately, why not just build a QGIS plugin to take any variety of geographic source files and output to your database format... the only thing remaining is how to give the user the tools to map biomes to LCLU (land cover land use) source data. Anyway I am going to get up to speed with what you have built here and try to contribute.
ps: as for huge DEM files, that is why imagemagick was the best image library: it didn't load the entire image into memory to extract pixel data... but it would be way better to use GDAL to extract this data anyway so that you can scale and distort it using map projections rather than image tools. ideally, this project should get to the point where it can handle vector source files and not just rasters. For example British Columbia has "biomes" called biogeoclimatic zones which are freely available as shapefiles.
ps: as for huge DEM files, that is why imagemagick was the best image library: it didn't load the entire image into memory to extract pixel data... but it would be way better to use GDAL to extract this data anyway so that you can scale and distort it using map projections rather than image tools. ideally, this project should get to the point where it can handle vector source files and not just rasters. For example British Columbia has "biomes" called biogeoclimatic zones which are freely available as shapefiles.
Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes
it would be nice to have real world coordinates available to the player in the game but I can't think how to do it best.
Who is online
Users browsing this forum: No registered users and 8 guests