[Mod] Real Terrain [0.1.0] [realterrain]

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by Gael de Sailly » Fri Mar 04, 2016 18:28

That's especially for the image library. The core uses an image library. It could simply (I know that technically, it may not be that simple) provide the functions to the Lua interface, no need to add another library to the core.
Just realize how bored we would be if the world was perfect.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by bobomb » Sat Mar 05, 2016 04:33

yes that's what I mean. should we ask the core developers to include pixel-access to images as a lua method?
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by Gael de Sailly » Sat Mar 05, 2016 08:25

I've sent a pull request to realterrain some weeks ago : PR 97.

For the image library, yes we should ask the devs. That's the moment : they're very active and make very good things these days. We should open an issue on minetest/minetest.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by Gael de Sailly » Sun Mar 06, 2016 12:11

I have another idea:

making a separate python script that converts the image into raw data that Lua can easily use. Run the script before starting Minetest.
With this method, the raw data can even been split up into many different tiles, each of them matching a… maybe 200x200 pixels, hugely reducing the amount of used RAM (unload a tile when it has not been used during 5 consecutive chunk generations for example). We can even use zlib compression, performed by python, and decompressed by the Minetest API:
Code: Select all
minetest.decompress(data)


I mean, in brief, what has been done for the Moon Mapgen mod by arpruss.
Just realize how bored we would be if the world was perfect.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by bobomb » Sun Mar 06, 2016 18:41

there are issues with most of the image processor modes under windows, but if you had imagemagick working in the past then changing the MAGICK_AS_CONVERT = false on init.lua line 40 should get you working again.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Real Terrain [0.0.9] [realterrain]

by bobomb » Tue Mar 15, 2016 17:56

Gael de Sailly wrote:I have another idea:

making a separate python script that converts the image into raw data that Lua can easily use. Run the script before starting Minetest.
With this method, the raw data can even been split up into many different tiles, each of them matching a… maybe 200x200 pixels, hugely reducing the amount of used RAM (unload a tile when it has not been used during 5 consecutive chunk generations for example). We can even use zlib compression, performed by python, and decompressed by the Minetest API:
Code: Select all
minetest.decompress(data)


I mean, in brief, what has been done for the Moon Mapgen mod by arpruss.


I have considered this approach but am not, personally, interested in it. The current method, with most of the (working) libraries does not load the entire image into memory anyway (the native png processor lifted from imageloader mod is an exception and crashes on even mid-sized images -- lua image libraries that load the entire image into memory were not considered for this mod. the imagemagick and graphicsmagic libs read the entire file and create a disk cache of it but at least do not load it into memory).
Mainly though I don't want to create any temp files because many of the source files of DEMs are huge and because I want them to be usable without any extra processing steps, otherwise we might as well convert DEM formats to ESRI's GRID (text-based) format. However, I envision a tiling system for version 0.2.0 of the mod which would allow any kind of tiles to be used, whether they be multiple contiguous tiles from a DEM set or from a python script as you say.
the mod already performs "caching" of sorts and only looks up pixels for map chunk "footprints" once in a vertical column -- it knows when a sky or sub-surface chunk is being rendered and does no pixel lookups...
I also experimented with cropping the image into smaller images for efficiency (not 200x200 but mapchunk size of 80x80 and multiples thereof) but with large source DEMs (which are the ones we'd want to do this with) re-opening the source file for cropping took several seconds each time. certain processors actually do this out of necessity. gm and convert for example -- which then translate the image into and parse a text pixel enumeration, which isn't working in windows...)
I think the best way forward is to use core's image handling functions to create lua methods that will open an image and get pixel values. anyone willing to help with this task please jump in ASAP. we could eliminate all outside libraries, command line (python) calls, and remove the mod security settings for this mod, making it much more portable and easy to use by a wider audience.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Gael de Sailly » Sat Mar 19, 2016 17:11

I've made a script that produces a rivermap from a DEM heightmap. Might be useful in the future.

It's here: rivermapper on GitHub.

Requires python3 and Wand library 0.4.2
Works only for 16-bit depth image for now.

RAM taken: ~ 20 Mo per million pixels, around 700 Mo for 6001*6001 SRTM tiles.
Speed: on my computer, it took 7 minutes to generate the rivers for this 6001*6001 SRTM tile.

Edit: Wand library version, and optimization.
Last edited by Gael de Sailly on Mon Mar 28, 2016 11:02, edited 2 times in total.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Gael de Sailly » Mon Mar 21, 2016 20:04

Sorry, I forgot to say that the code is only working for wand library 0.4.2.
Just realize how bored we would be if the world was perfect.
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Gael de Sailly » Wed Mar 23, 2016 21:27

Optimized (Reduced taken memory by 60%), and bugs fixed.
Just realize how bored we would be if the world was perfect.
 

User avatar
Andrey01
Member
 
Posts: 2309
Joined: Wed Oct 19, 2016 15:18
Location: Russia, Moscow
GitHub: Andrey2470T
In-game: Andrey01
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by bobomb » Mon Dec 12, 2016 17:36

fyi the latest mod includes a mode that does in fact preload the map.sqlite file using a python script so that no image libraries are needed. it is still somewhat experimental. python 3 needed
 

User avatar
captpete
Member
 
Posts: 134
Joined: Fri Nov 25, 2016 03:02

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by captpete » Mon Feb 27, 2017 01:59

YouTube video tutorial no longer available - any other detailed help on this?
 

User avatar
Open_Future
New member
 
Posts: 6
Joined: Sat Apr 09, 2016 19:47
Location: Ontario, Canada
IRC: Open_Future
In-game: dante123

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Open_Future » Wed Dec 20, 2017 15:30

I'm wondering if a map of Canada could be produced with this mod that has the major regions of our country. Namely:

Western Cordillera
Interior Plains
Canadian Shield
Hudson Bay Lowlands
St. Lawrence Lowlands
Appalachia

Kind of like as shown here: http://www.craigmarlatt.com/canada/geog ... sical.html

Or perhaps there is some elevation data for Canada that could be used. Any help or suggestions is appreciated. This would be used with students in grades 3 and 4 as part of social studies and geography.
 

User avatar
captpete
Member
 
Posts: 134
Joined: Fri Nov 25, 2016 03:02

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by captpete » Fri Jul 20, 2018 20:59

Tried this mod before on an older version of MineTest but now, on a clean world with [realterrain] the only mod loaded I get this:

Image

No debug.txt generated.
Attachments
Untitled.png
(14.87 KiB) Not downloaded yet
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Gael de Sailly » Sat Jul 21, 2018 09:24

Realterrain tries to access external libraries, what mod security forbids. You must disable mod security (at your own risks, don't run any untrustworthy mod).
Just realize how bored we would be if the world was perfect.
 

bobomb
Member
 
Posts: 115
Joined: Sat May 23, 2015 20:28
GitHub: bobombolo
IRC: bobomb

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by bobomb » Sat Oct 13, 2018 22:00

going to revive this project. lost some data along the way: does anyone have the last copy of the project? the one with the python script that pre-generates the map?
 

User avatar
Gael de Sailly
Member
 
Posts: 733
Joined: Sun Jan 26, 2014 17:01
Location: Lyon, France
GitHub: Gael-de-Sailly
IRC: Gael-de-Sailly
In-game: Gael-de-Sailly Ginkgoo

Re: [Mod] Real Terrain [0.1.0] [realterrain]

by Gael de Sailly » Sun Oct 14, 2018 09:52

Hi bobomb. I am very glad you are back on this project. Much time has gone, and, a bit disappointed by the abandon of your project, I started my own, that you can find here: https://forum.minetest.net/viewtopic.php?f=9&t=19387. It works differently, in 2 steps, using a Python script to convert the images into some kind of database, and then the Lua mod generates the map based on this database. If you want to join me, you're welcome.

bobomb wrote:lost some data along the way: does anyone have the last copy of the project? the one with the python script that pre-generates the map?

The latest I can find is this one: https://github.com/Gael-de-Sailly/realterrain/tree/neighborhood_fix.
I might have a more recent local copy on my old computer in my family's home, where I will be next Saturday.
Just realize how bored we would be if the world was perfect.
 

Previous

Return to Mod Releases



Who is online

Users browsing this forum: No registered users and 13 guests