[Mod] Mapgen from geographical data [geo-mapgen] (Biomes!)

User avatar
TumeniNodes
Member
 
Posts: 2827
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: [Mod] Map generation from DEM [geo-mapgen] (WIP)

by TumeniNodes » Thu Jan 18, 2018 02:23

Great work Gael.
The new screenshots are really nice, especially the first one, it's beautiful.
Ich mag keine grünen Eier und Schinken, ich mag sie nicht Sam I Am
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Sun Jan 21, 2018 21:44

I'm trying to learn to use QGIS and related Python libs that seems very interesting for what I'm trying to do.
QGIS is the only application I've found that has managed to open huge TIFF files without literally burning my RAM.

The second step is to know what to do with the loaded image.
Last edited by Gael de Sailly on Sun Jan 21, 2018 23:37, edited 1 time in total.
Just realize how bored we would be if the world was perfect.
 

User avatar
TumeniNodes
Member
 
Posts: 2827
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by TumeniNodes » Sun Jan 21, 2018 22:50

Looks promising, and like a really great project (meaning QGIS)

Do you think the mtmapprune could be incorporated somehow, eventually to scale large maps?
viewtopic.php?f=14&t=18579

I fully understand it is probably way to early to know, I'm just wondering.
Last edited by TumeniNodes on Sun Jan 28, 2018 21:24, edited 1 time in total.
Ich mag keine grünen Eier und Schinken, ich mag sie nicht Sam I Am
 

User avatar
Inocudom
Member
 
Posts: 3071
Joined: Sat Sep 29, 2012 01:14
IRC: Inocudom
In-game: Inocudom

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Inocudom » Sun Jan 21, 2018 23:40

There are many possibilities with a mod like this. Why, I could end up with a map based on the mountainous terrain surrounding Great Cacapon, WV.
The comedic value of my videos is how abysmal they are: https://www.youtube.com/channel/UC5cyeM ... YTfYFlih_A
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Sun Jan 28, 2018 21:20

I found something interesting with GDAL library. It's able to crop, resample and change the projection of a raster image (through function "Warp"). It could allow using several images of different projections, and resample them to use only the part of the world we want.

Potentially good data source for biomes: Corine Land Cover (in Europe). I suppose (and I hope) there are equivalent data for other parts of the world. IIRC bobomb found one for United States when he developed RealTerrain some years ago.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Sat Feb 03, 2018 23:01

Today I've worked on graphical interface:
Image
Code is on tkinter branch. EDIT: merged
You need tkinter library to use it.

Command to start on graphical mode:
Code: Select all
./image_convert.py -g


For now only the input is on GUI. The output is still on the console.
Just realize how bored we would be if the world was perfect.
 

User avatar
TumeniNodes
Member
 
Posts: 2827
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by TumeniNodes » Sun Feb 04, 2018 19:15

Very very nice Gael de Sailly
this makes it very much easier to work with
Ich mag keine grünen Eier und Schinken, ich mag sie nicht Sam I Am
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Mon Feb 19, 2018 00:31

I finally managed to get my Python script working with the GDAL library. It was difficult. I've pushed it on a new branch: https://github.com/Gael-de-Sailly/geo-mapgen/tree/gdal.
GDAL allows to manipulate and change the projection of geographical data like the DEMs this script uses.
I did this because it's an essential feature for a more long-run project: using biome maps like Corine Land Cover, superimposed on the heightmap, that may be in a different projection.
The script as it is now could deal with several maps in different projections with very few more work.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Tue Feb 20, 2018 23:44

New branch with good changes: it's now able to handle land cover images!
https://github.com/Gael-de-Sailly/geo-mapgen/tree/landcover
For now only the ground nodes are changed, but decorations are planned.

See the result on the Mont Blanc (I've not taken the time to make more good screenshots for now)
Image

However, it's quite complicated to generate such maps. You need to find an heightmap like before, and also a raster land cover image covering the same region (this is already hard to find). But I faced a major problem: land cover maps are not standardized, and according to the organization who provide the map, the map legend can be different. You must write a file that associates each value with one of the biomes defined in biomes.csv. See the file "clc.lct" that matches Corine Land Cover.

I'll explain this in more details when it will be finished. I think I should even make a complete tutorial for a very special mod like this one, but that's something hard to do for me. I always have difficulties in explaining my work to others.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by Gael de Sailly » Wed Feb 21, 2018 15:44

News of the day, on landcover branch:
Trees and plants, according to the biome!

Look at the Mont Blanc now:
Image
Just realize how bored we would be if the world was perfect.
 

slemon
Member
 
Posts: 104
Joined: Sun Apr 27, 2014 03:56
 

User avatar
Fixer
Member
 
Posts: 891
Joined: Sun Jul 31, 2011 11:23
Location: Ukraine
IRC: Fixer
In-game: Fixer

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Fixer » Mon Mar 05, 2018 22:32

Mount Blanc looks nice, can you make it 1:1 scale? That will be epic for me, I will waste my time to conquer it :D
 

User avatar
ExeterDad
Member
 
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
Location: New Hampshire U.S.A
In-game: ExeterDad

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by ExeterDad » Tue Mar 06, 2018 00:46

This is very exciting stuff! Although I wouldn't let players dig a thing. It's too pretty.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Tue Mar 06, 2018 18:54

Fixerol wrote:Mount Blanc looks nice, can you make it 1:1 scale? That will be epic for me, I will waste my time to conquer it :D

Unfortunately I don't think you can find free data that are accurate enough. The best I know is Copernicus EU-DEM 1.1 (resolution 25m). It's possible to rescale it to 1 node = 1m, but it would be completely smoothed, and you will not see far.
Just realize how bored we would be if the world was perfect.
 

User avatar
Fixer
Member
 
Posts: 891
Joined: Sun Jul 31, 2011 11:23
Location: Ukraine
IRC: Fixer
In-game: Fixer
 

User avatar
azekill_DIABLO
Member
 
Posts: 7489
Joined: Wed Oct 29, 2014 20:05
Location: OMICRON
GitHub: azekillDIABLO
In-game: azekill_DIABLO

Re: [Mod] Mapgen from geographical data [geo-mapgen] (WIP)

by azekill_DIABLO » Fri Mar 09, 2018 11:51

Gael de Sailly wrote:QGIS is the only application I've found that has managed to open huge TIFF files without literally burning my RAM.

I have some French docs on Qgis because I used it a while ago! PM me if you need help!
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Fri Mar 09, 2018 19:29

azekill_DIABLO wrote:I have some French docs on Qgis because I used it a while ago! PM me if you need help!

Thanks, but I have found what I was looking for, with GDAL (Geospatial Data Abstraction Library). It's the library used by Qgis to open raster layers. The python script is currently using it, works well.
Just realize how bored we would be if the world was perfect.
 

Chewy509
New member
 
Posts: 5
Joined: Sun Jan 29, 2017 10:05
GitHub: dkartaschew

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Chewy509 » Sat Mar 10, 2018 09:09

Excellent mod. Been able to do a few areas with data from SRTM, however encountered an area where it fails.

I've posted bug reports to github for tracking. https://github.com/Gael-de-Sailly/geo-mapgen/issues
 

User avatar
BirgitLachner
Member
 
Posts: 379
Joined: Thu May 05, 2016 10:18
In-game: Bibs

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by BirgitLachner » Tue Mar 13, 2018 20:19

Very nice ... I was already looking for Real Terrain but I need the data.
 

User avatar
TumeniNodes
Member
 
Posts: 2827
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by TumeniNodes » Mon Jul 16, 2018 04:18

Gael de Sailly,
Thought you might be interested in this:
https://www.michaelfogleman.com/projects/terrarium/
Ich mag keine grünen Eier und Schinken, ich mag sie nicht Sam I Am
 

User avatar
ChimneySwift
Member
 
Posts: 320
Joined: Fri Sep 22, 2017 06:46
Location: 127.0.0.1
GitHub: ChimneySwift
IRC: ChimneySwift
In-game: ChimneySwift

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by ChimneySwift » Tue Sep 04, 2018 00:46

Unfortunately I get an error on 5.0-dev Win64 https://pastebin.com/1QRa8RxZ

Any ideas?
A spoon is basically a tiny bowl with a stick on it
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Tue Sep 04, 2018 07:19

It fails to decompress data while reading a chunk. There is 2 reasons I can see: there may be an error in heightmap.dat file structure, or the mod is trying to read the file from the wrong position.
What map were you using?
Just realize how bored we would be if the world was perfect.
 

User avatar
ChimneySwift
Member
 
Posts: 320
Joined: Fri Sep 22, 2017 06:46
Location: 127.0.0.1
GitHub: ChimneySwift
IRC: ChimneySwift
In-game: ChimneySwift

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by ChimneySwift » Tue Sep 04, 2018 07:42

Gael de Sailly wrote:It fails to decompress data while reading a chunk. There is 2 reasons I can see: there may be an error in heightmap.dat file structure, or the mod is trying to read the file from the wrong position.
What map were you using?


I tried both of the demo heightmaps and a height map I generated from a section of Australia from http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp
A spoon is basically a tiny bowl with a stick on it
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 14:57

It's great to see that you have taken this approach. In fact I had abandoned realterrain because I came to realize that the better approach was indeed to do something like this where you use QGIS or python GDAL to prebuild the map using that minetest-lib python library, but that had issues too and your idea to build an intermediate database format that the lua mapgen can read directly is probably the only way this will ever work. I also realized that realterrain would have to give up on using the in-game form-based mapgen settings tool I had built, which was actually a huge part of the work, for an out-of-game interface. I tinkered with the idea of building a form in tkinter or something to simulate a "worldpaint" type of approach, but ultimately, why not just build a QGIS plugin to take any variety of geographic source files and output to your database format... the only thing remaining is how to give the user the tools to map biomes to LCLU (land cover land use) source data. Anyway I am going to get up to speed with what you have built here and try to contribute.

ps: as for huge DEM files, that is why imagemagick was the best image library: it didn't load the entire image into memory to extract pixel data... but it would be way better to use GDAL to extract this data anyway so that you can scale and distort it using map projections rather than image tools. ideally, this project should get to the point where it can handle vector source files and not just rasters. For example British Columbia has "biomes" called biogeoclimatic zones which are freely available as shapefiles.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 16:44

it would be nice to have real world coordinates available to the player in the game but I can't think how to do it best.
 

PreviousNext

Return to WIP Mods



Who is online

Users browsing this forum: Google [Bot] and 7 guests