[Mod] Mapgen from geographical data [geo-mapgen] (Biomes!)

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 17:05

Another approach to cropping/scaling would be to define an extent using real world coordinates (manually entering long/lat or UTM coords), or selecting the extent from a shapefile that you have created for the purpose, or from the extent of one of your input files. Many QGIS tools already act this way. Then you can set your x/y/z scale and then using gdal you can sample the source files (possibly taking a neighborhood average) rather than handling them pixel by pixel. It's a different approach but it eliminates issues with large DEM files, resolution or extent mismatches, and lends itself to future features better (using vector inputs, querying map services, etc).
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 17:51

Is there any reason why this project shouldn't BE a QGIS tool? anybody interested in this kind of thing would be able to install and run QGIS. It would eliminate the need to make a custom tkInter GUI tool. it would bring people from the QGIS world to the minetest world.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 19:09

On windows the best way to install QGIS and gdal is OSGEO4W
http://download.osgeo.org/osgeo4w/osgeo4w-setup.exe
This installs a newer version of QGIS than the default installed on most linux distros.
For linux you must install the unstable repo to get the most recent QGIS.
https://www.qgis.org/en/site/forusers/a ... html#linux
Basically, don't waste your time with QGIS 2.x
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 19:13

It would be nice to save the settings in the tkinter form. This would allow for saving presets for comparison and sharing. It would be nice if the final form of the mod shipped with some presets including some datasets so that right out of the box you can play on a world map, Canada map, France map, etc. DEMs are too big for this so the mod would have to work with smaller files (vector based files).
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Sun Oct 14, 2018 20:31

the final database format should include underground geoformations, soil layers, aquifers and also bathymetry.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Sun Oct 14, 2018 23:40

Well... I was not expecting to have so many posts to read, let me answer your questions one by one.

bobomb wrote:it would be way better to use GDAL to extract this data anyway so that you can scale and distort it using map projections rather than image tools.

It is already using GDAL, btw you need to have this lib installed if you want to use the python script.

bobomb wrote:it would be nice to have real world coordinates available to the player in the game but I can't think how to do it best.

This would need to convert MT coordinates into lat/lon, using the projection. but the Lua has no access to GDAL. We could still implement a conversion function for several particular projections like Mercator.

bobomb wrote:Another approach to cropping/scaling would be to define an extent using real world coordinates (manually entering long/lat or UTM coords), or selecting the extent from a shapefile that you have created for the purpose, or from the extent of one of your input files. Many QGIS tools already act this way. Then you can set your x/y/z scale and then using gdal you can sample the source files (possibly taking a neighborhood average) rather than handling them pixel by pixel.

I'm not sure to understand everything but it seems that it's already what I do. I use GDAL to reproject and/or resample the image: https://github.com/Gael-de-Sailly/geo-mapgen/blob/master/map_transform.py#L134

bobomb wrote:Is there any reason why this project shouldn't BE a QGIS tool? anybody interested in this kind of thing would be able to install and run QGIS. It would eliminate the need to make a custom tkInter GUI tool. it would bring people from the QGIS world to the minetest world.

This, is a very interesting way to explore, would make it much easier to install and allow a cleaner interface. This idea has already crossed my mind, but I had chosen to stick on plain Python until I have something working, and possibly turning it into a QGIS plugin after. But as always, once it was working, I spent my time exploring my worlds and forgot everything else xD

bobomb wrote:It would be nice to save the settings in the tkinter form. This would allow for saving presets for comparison and sharing.

Yea the content of the form should be stored somewhere, also to find again the same parameters on next use if you often use the same. Also would be nice to store the params in some metadata of the database.

bobomb wrote:It would be nice if the final form of the mod shipped with some presets including some datasets so that right out of the box you can play on a world map, Canada map, France map, etc. DEMs are too big for this so the mod would have to work with smaller files (vector based files).

I'm hesitant to pack big files in the download, that not everyone will use. This can discourage some people with very low internet from downloading the mod. I think we should always keep a light, purely functional download, and provide in addition, either easily downloadable presets, or another version including these presets.

bobomb wrote:the final database format should include underground geoformations, soil layers, aquifers and also bathymetry.

Bathymetry is already supported, just use a negative value for elevation. Current presets (Alps and High-res world) just doesn't use it because of the input data I was using. For the other features, it would need a 3D geological model, and I very highly doubt this could be found easily on the web. Maybe I can find some 2D data of geological land cover for France or US, there are already things to do with that.

Thank you very much for your interest. You have excellent ideas, and you are giving me motivation to work on this again (this project has been a bit abandoned too for the last 6 months). By being several at working on a project, we can move mountains, but literally in our case ;)
Just realize how bored we would be if the world was perfect.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by bobomb » Mon Oct 15, 2018 18:52

thanks for replying to my many posts, i also am excited about this.
regarding the real world coordinates, if we stuck to UTM we could use simple maths to convert. if we wanted long/lat maybe we can embed the coordinates in param1 or param2 of a node every so often (the chunk corner?) and do some simple maths to at least approximate...
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Sun May 26, 2019 11:44

I'm now retaking this old –but not dying– project and I'm rewriting the Python side from scratch (it was a bit of a mess). The idea is to turn it into a flexible Python library that could be used either with standalone Python or with QGIS.

Also found this: https://luarocks.org/modules/geoffleyland/proj, a small Lua library for map projection. Doesn't seem actively developed but is working as is. It could be used to provide extra in-game features like teleporting to a location, or displaying real-world coordinates (would be optional anyway because I want the mod to be usable without extra Lua libs).

EDIT: new branch on github https://github.com/Gael-de-Sailly/geo-mapgen/tree/rewrite
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Fri May 31, 2019 00:46

I've been working on it quite a lot these days, and now I've got the rewritten Python side working as a library, you can find it on the rewrite branch. No user interface for now for this version but this concept will allow more simple and efficient work to develop one, that would ideally be based on QGIS.

Teleporting to real-world coordinates has been implemented, based on the proj library of my previous post. The database now contains georeferences to allow this.

And one screenshot: Sunset above Loch Awe and Ben Cruachan summit, Scottish Highlands, UK. 1 node is only 25 meters.
Image
Just realize how bored we would be if the world was perfect.
 

Extex
Member
 
Posts: 112
Joined: Wed Mar 14, 2018 23:14
GitHub: Extex101
In-game: Extex

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Extex » Tue Oct 01, 2019 01:55

Will this work on android?
Player of PK, RIP. Creator of the light_tool mod. Working on another mod which is pretty cool so far
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Mapgen from geographical data [geo-mapgen] (Biomes

by Gael de Sailly » Tue Oct 01, 2019 10:54

Extex wrote:Will this work on android?


I've never used Minetest on Android so I don't have an absolute answer, but probably the Lua part will work fine because it doesn't rely on anything external. However the pre-processing part needs a Python installation with Tkinter and GDAL libs (I don't know whether it's easy to set up on Android) and can be quite heavy (this depends on the size of your map). I would recommend to generate the file using a computer and then transfer the output file on Android, but if you think you can do it on Android, you can try too.

I plan to do a command-line only version that would be easier to use on some devices, the Tkinter backend becoming optional. IIRC I already have a start of it somewhere, I just need enough motivation to dig into it again.
Just realize how bored we would be if the world was perfect.
 

Previous

Return to WIP Mods



Who is online

Users browsing this forum: runs and 4 guests