Minetest mapserver

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Minetest mapserver

by BuckarooBanzay » Sun Jan 27, 2019 18:37

Minetest mapserver
Realtime mapserver for Minetest

Successor of viewtopic.php?f=14&t=20371

Features:
  • Click-and-run installation (TM)
  • Initial and incremental map rendering
  • Configurable layers (default: "Base" from y -16 to 160)
  • Tile-Caching
  • POI markers / mod integration
  • Player markers and infos (HP, breath, etc)
  • Train/Composition display
  • Search function

Planned features:
  • Street names
  • Street view / 3D-View

Links:
Demo: https://pandorabox.io/map/#0/-1782.25/493.5/10
Downloads: https://github.com/minetest-tools/mapserver/releases
Download (Mod): https://github.com/minetest-tools/mapserver_mod
Github/Docs: https://github.com/minetest-tools/mapserver


Supported architectures/platforms
  • Linux x86 (32bit and 64bit) arm (raspberry pi, 32bit)
  • MacOS x86 32bit and 64bit
  • Windows x86 32bit and 64bit

Supported minetest variations
  • Minetest around 0.4.15-17 / 5.0
  • Sqlite3 or postgresql

Installation
NOTE: Please make a backup of your world in case something goes wrong!


See: https://github.com/minetest-tools/mapse ... install.md

Configuration

All config options reside in the mapserver.json file with the default values
Please stop the server if you make any changes there.

Bugs
There will be bugs, please file them here: https://github.com/minetest-tools/mapserver/issues

Contributions
Contributions are always welcome via pull/merge requests



Demo gif:
Image

Web interface
Image

Terminal
Image

Map markers
Image
Image
Image
Image
Image
Image

Train markers
Image
Image

Search
Image
Last edited by BuckarooBanzay on Sat Jul 27, 2019 17:00, edited 16 times in total.
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.1]

by DoyleChris » Sun Jan 27, 2019 19:55

I downloaded it for raspberry pi and moved it over to the worlds folder inside the map i want to use. and go to run ./mapserver and i get bash: ./mapserver: No such file or directory
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.1]

by BuckarooBanzay » Sun Jan 27, 2019 20:03

DoyleChris wrote:I downloaded it for raspberry pi and moved it over to the worlds folder inside the map i want to use. and go to run ./mapserver and i get bash: ./mapserver: No such file or directory


Did you try it with
Code: Select all
chmod 755 ./mapserver

Maybe just a permission issue (executable permission)
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.1]

by BuckarooBanzay » Mon Jan 28, 2019 19:03

DoyleChris wrote:I downloaded it for raspberry pi and moved it over to the worlds folder inside the map i want to use. and go to run ./mapserver and i get bash: ./mapserver: No such file or directory


Sorry about that. That was a mistake on my side: i thought cross-compiling with go would be easier than in C/C++ :D
Anways, i released version 0.0.2 for windows and linux (32,64 bit x86 and arm)
I tested the linux-arm release in an emulator and it looks good so far (if your map is big though it will be slow initially)
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.2]

by DoyleChris » Tue Jan 29, 2019 22:02

Well i tried the latest version on my RPI and i get the same error. Even by running chmod 755 ./mapserver i still get, chmod: cannot access './mapserver': No such file or directory.
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.2]

by DoyleChris » Wed Jan 30, 2019 01:21

i got it to work had to change mapserver-linux-arm to mapserver
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.2]

by DoyleChris » Wed Jan 30, 2019 02:08

I get this though
Code: Select all
INFO[2595] incremental rendering                         elapsed=17.452627ms mapblocks=5 tiles=20
panic: database is locked

goroutine 5 [running]:
mapserver/tilerendererjob.worker(0x10a2820, 0x105a240)
        /home/thomas/git/mapserver/server/tilerendererjob/worker.go:16 +0xb0
created by mapserver/tilerendererjob.Job
        /home/thomas/git/mapserver/server/tilerendererjob/job.go:14 +0x64
 

sofar
Developer
 
Posts: 2064
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Thu Jan 31, 2019 18:01

DoyleChris wrote:I get this though
Code: Select all
INFO[2595] incremental rendering                         elapsed=17.452627ms mapblocks=5 tiles=20
panic: database is locked

goroutine 5 [running]:
mapserver/tilerendererjob.worker(0x10a2820, 0x105a240)
        /home/thomas/git/mapserver/server/tilerendererjob/worker.go:16 +0xb0
created by mapserver/tilerendererjob.Job
        /home/thomas/git/mapserver/server/tilerendererjob/job.go:14 +0x64


Can you try it with the newest release (v0.0.3):
https://github.com/thomasrudin-mt/mapse ... /tag/0.0.3

I changed the database flags to account for the increased delay
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Thu Jan 31, 2019 18:02

sofar wrote:Yay! Another Go tool! Congrats!


Thx, i wanted a simpler, more maintanable solution than the java-bloat i already had :)
Its a fun language with a pretty good learning curve...
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.2]

by DoyleChris » Sat Feb 02, 2019 18:27

I got another error to do with the map.

Code: Select all
2019-02-02 13:23:53: ERROR[Server]: An unhandled exception occurred: Failed to save block: database or disk is full
2019-02-02 13:23:53: ERROR[Server]: In thread 74aff440:
2019-02-02 13:23:53: ERROR[Server]: /home/pi/Minetest/4.17.1/minetest-0.4.17.1/src/server.cpp:73: virtual void* ServerThread::run(): A fatal error occured: Failed to save block: database or disk is full
2019-02-02 13:23:53: ERROR[Server]: Debug stacks:
2019-02-02 13:23:53: ERROR[Server]: DEBUG STACK FOR THREAD 7314e440:
2019-02-02 13:23:53: ERROR[Server]: #0  virtual void* EmergeThread::run()
2019-02-02 13:23:53: ERROR[Server]: DEBUG STACK FOR THREAD 74aff440:
2019-02-02 13:23:53: ERROR[Server]: #0  virtual void* ServerThread::run()
2019-02-02 13:23:53: ERROR[Server]: DEBUG STACK FOR THREAD 76f74790:
2019-02-02 13:23:53: ERROR[Server]: #0  int main(int, char**)
2019-02-02 13:23:53: ERROR[Server]: #1  Dedicated server branch
2019-02-02 13:23:53: ERROR[Server]: #2  void dedicated_server_loop(Server&, bool&)
 

sofar
Developer
 
Posts: 2064
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Minetest mapserver [0.0.2]

by sofar » Sat Feb 02, 2019 19:20

BuckarooBanzay wrote:Thx, i wanted a simpler, more maintanable solution than the java-bloat i already had :)
Its a fun language with a pretty good learning curve...


Java isn't bad if you're prototyping, but if you're deploying on a smallish cloud instance, it's impossible, literally, because you need half a gig of ram to even run. With golang, you can deploy without any of that.
 

sofar
Developer
 
Posts: 2064
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Minetest mapserver [0.0.2]

by sofar » Sat Feb 02, 2019 19:25

DoyleChris wrote:I got another error to do with the map.

Code: Select all
2019-02-02 13:23:53: ERROR[Server]: An unhandled exception occurred: Failed to save block: database or disk is full
2019-02-02 13:23:53: ERROR[Server]: In thread 74aff440:
2019-02-02 13:23:53: ERROR[Server]: /home/pi/Minetest/4.17.1/minetest-0.4.17.1/src/server.cpp:73: virtual void* ServerThread::run(): A fatal error occured: Failed to save block: database or disk is full


Could be that the mapserver is still locking the sqlite database, and MT really does not like that.
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Sat Feb 02, 2019 19:38

sofar wrote:
DoyleChris wrote:I got another error to do with the map.

Code: Select all
2019-02-02 13:23:53: ERROR[Server]: An unhandled exception occurred: Failed to save block: database or disk is full
2019-02-02 13:23:53: ERROR[Server]: In thread 74aff440:
2019-02-02 13:23:53: ERROR[Server]: /home/pi/Minetest/4.17.1/minetest-0.4.17.1/src/server.cpp:73: virtual void* ServerThread::run(): A fatal error occured: Failed to save block: database or disk is full


Could be that the mapserver is still locking the sqlite database, and MT really does not like that.


The cause was/is a full disk, as the message said :)
database or disk is full


The mapserver opens a read-only connection to the sqlite database to circumvent such things as locking, works good as far as i tested it. That won't be a problem for the next database-connectors (psql to name one).

One exception for the read-only connection is the initial connect though (in which the timestamp-columns are added)

@DoyleChris thx for testing :D let me know if you have other issues

Next item on my list: realtime updates of travelnets, lua-displays and poi's xD
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Sun Feb 03, 2019 20:50

DoyleChris wrote:Cant see players on the surface.


Thats on my roadmap, don't worry :)
Planned features:
  • POI markers / mod integration
  • Player markers and infos (HP, breath, etc)
  • Street names / Train stuff
 

DoyleChris
Member
 
Posts: 252
Joined: Sat Jul 25, 2015 19:54
In-game: DoyleChris

Re: Minetest mapserver [0.0.2]

by DoyleChris » Tue Feb 05, 2019 21:55

So i moved my minetest server from the sdcard to a external harddrive. The harddrive is NTFS.
i go to run the ./mapserver command and i get this message.

Code: Select all
panic: unexpected end of JSON input

goroutine 1 [running]:
main.main()
        /home/thomas/git/mapserver/server/main.go:43 +0x2ec
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Wed Feb 06, 2019 19:12

DoyleChris wrote:So i moved my minetest server from the sdcard to a external harddrive. The harddrive is NTFS.
i go to run the ./mapserver command and i get this message.

Code: Select all
panic: unexpected end of JSON input

goroutine 1 [running]:
main.main()
        /home/thomas/git/mapserver/server/main.go:43 +0x2ec


I think your mapserver.json got somehow corrupted :(
How does that file look like on your drive?
You could try to replace it with the following snippet or deleting it alltogether (it needs to re-render the whole map though)

Code: Select all
{
   "port": 8080,
   "enablerendering": true,
   "webdev": true,
   "webapi": {
      "enablemapblock": false,
      "secretkey": "QKnlxTTeqdNtUajz"
   },
   "renderstate": {
      "initialrun": false,
      "legacyprocessed": 0,
      "lastx": -2048,
      "lasty": -2048,
      "lastz": -2048,
      "lastmtime": 1548495898
   },
   "layers": [
      {
         "id": 0,
         "name": "Base",
         "to": 160,
         "from": -16
      }
   ],
   "renderingfetchlimit": 1000,
   "renderingjobs": 4,
   "renderingqueue": 100
}
 

ShadMOrdre
Member
 
Posts: 523
Joined: Mon Dec 29, 2014 08:07
Location: USA
GitHub: ShadMOrdre
In-game: shadmordre

Re: Minetest mapserver [0.0.2]

by ShadMOrdre » Thu Feb 07, 2019 07:12

How or can I specify a colors.txt file?

I use my lib_materials and lib_ecology mods. lib_materials defines a lot of new grass, dirt, ore, stone, sand and liquid types, that make up the world. lib_ecology adds to this adding plants and trees, and defining biomes.

I do not currently have an updated colors.txt for minetestmapper that accommodates my mods, but nothing here indicates this would be necessary. If this is not the case, which mods, specifically, does this support?
MY MODS: lib_ecology lib_materials lib_clouds lib_node_shapes ---- Inspired By: Open Source Virtual World Simulator Opensimulator.
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.0.2]

by BuckarooBanzay » Fri Feb 08, 2019 20:31

ShadMOrdre wrote:How or can I specify a colors.txt file?

I use my lib_materials and lib_ecology mods. lib_materials defines a lot of new grass, dirt, ore, stone, sand and liquid types, that make up the world. lib_ecology adds to this adding plants and trees, and defining biomes.

I do not currently have an updated colors.txt for minetestmapper that accommodates my mods, but nothing here indicates this would be necessary. If this is not the case, which mods, specifically, does this support?


I just released version 0.1.0 with a lot of new features, the custom colors.txt is documented here: https://github.com/thomasrudin-mt/mapse ... /config.md
 

ShadMOrdre
Member
 
Posts: 523
Joined: Mon Dec 29, 2014 08:07
Location: USA
GitHub: ShadMOrdre
In-game: shadmordre

Re: Minetest mapserver [0.1.0]

by ShadMOrdre » Sun Feb 17, 2019 05:46

BuckarooBanzay,

Just wanted to let you know that I have generated values for my mods for colors.txt. The maps look great.

Thanks,
Shad
MY MODS: lib_ecology lib_materials lib_clouds lib_node_shapes ---- Inspired By: Open Source Virtual World Simulator Opensimulator.
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [0.1.0]

by BuckarooBanzay » Tue Feb 19, 2019 21:02

ShadMOrdre wrote:BuckarooBanzay,

Just wanted to let you know that I have generated values for my mods for colors.txt. The maps look great.

Thanks,
Shad


Good to hear :)
I'm currently working on the postgres implementation.
After that i will move the map for pandorabox to this mapserver.
30 GB of map-data with 4 layers... i wonder how long this will take :D
 

BuckarooBanzay
Member
 
Posts: 289
Joined: Tue Apr 24, 2018 05:58
Location: Switzerland
GitHub: thomasrudin-mt
In-game: BuckarooBanzai

Re: Minetest mapserver [1.0.0]

by BuckarooBanzay » Sun Feb 24, 2019 19:50

Update 2018-02-24 Version 1.0.0
Updated to V1.0.0

This release is not backward compatible, sorry...

Important changes:
  • Postgres support (experimental)
  • More mapobjects (technic anchor/quarry/switching station)
  • Realtime train/composition display
  • New tile-storage (not backwards-compatible)

Train markers
Image
Image
 

User avatar
gpcf
Member
 
Posts: 317
Joined: Fri May 27, 2016 10:48
In-game: gabriel

Re: Minetest mapserver [1.0.0]

by gpcf » Thu Feb 28, 2019 17:22

Is this already operational on your server?
 

Next

Return to Minetest-related projects



Who is online

Users browsing this forum: No registered users and 1 guest