Minetest on Ubuntu as service (with RAMDISK and backups)

Post Reply
User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Minetest on Ubuntu as service (with RAMDISK and backups)

by lag01 » Post

2018
Rewritten in systemd.
Now uses only single ramdisk for entire world folder.
No need to custom minetest compilation.
Little simplified installation.

https://github.com/AndrejIT/RAMDISK_setup

Here is old obsolete guide:
Spoiler
This guide assumes that the one who try to follow it, have basic knowledge in installing and using Linux (i was beginner in Linux when i started this). Also some of programming knowledge will be needed.
This is not complete guide, so use other guides to configure port forwarding, bash commands, Linux installing etc.

So, my big idea about making fast fast server is to store map file and player files in ram. It increases ram requirement and makes server theoretically less stable, but it is not a big deal if you have much ram and UPS.
I know that minetest can be compiled with different database engine, but i feel it would be harder for me and you need ramdisk for player profiles anyway (at least until developers have fixed intensive read/write to profiles).

I have tested this setup on my "just test" server for two months and it can easily handle 30 players and more.
  • You need computer with enough ram - 4GB or more. Also, you better have UPS.
  • Install Debian or Ubuntu. Better do not install graphical interface or other modules, except SSH. Better do not install swap partition.
    Later you can use Putty and WinSCP to connect to your server.
  • If you installed Debian stable, add unstable repository to get lates minetest. To file /etc/apt/sources.list add line "deb http://ftp.de.debian.org/debian jessie main"
  • Install minetest:

    Code: Select all

    apt-get install minetest-server minetest-data
  • Define ramfs to store map and folder with user profiles in ram: in file /etc/fstab add lines like these:

    Code: Select all

    ramfs   /home/user/.minetest/worlds/test0/players       ramfs   noauto,rw,users,sync,size=100M   0       0
    ramfs   /home/user/.minetest/worlds/test0/map      ramfs   noauto,rw,users,sync,size=2000M        0       0
  • Create folder /home/user/.minetest/worlds/test0/map/
    Then in folder /home/user/.minetest/worlds/test0/ create symbolic link with name "map.sqlite" pointing to file /home/user/.minetest/worlds/test0/map/map.sqlite
  • Make startup script "/etc/init.d/minetest" to run minetest as service, use /etc/init.d/skeleton or https://gist.github.com/MarkTraceur/998260 as base. Use command "/ets/init.d/minetest start" to test it.
  • Create folders to safely save map and players, something like:
    /home/user/.minetest/worlds/test0/players.safe/ and /home/user/.minetest/worlds/test0/map.safe/
  • To keep all in ram, while server is working, and save to disk, when it shuts down, add the following to your startup script:
    In "start" part add lines before daemon start:

    Code: Select all

    mount /home/user/.minetest/worlds/test0/players
    chown user /home/user/.minetest/worlds/test0/players/
    cp -rp /home/user/.minetest/worlds/test0/players.safe/* /home/user/.minetest/worlds/test0/players/
    mount /home/user/.minetest/worlds/test0/map
    chown user /home/user/.minetest/worlds/test0/map/
    cp -rp /home/user/.minetest/worlds/test0/map.safe/map.sqlite /home/user/.minetest/worlds/test0/map/map.sqlite
    In "stop" part add lines after daemon stop:

    Code: Select all

    cp -rp /home/user/.minetest/worlds/test0/players/* /home/user/.minetest/worlds/test0/players.safe/
    umount /home/user/.minetest/worlds/test0/players
    cp -rp /home/user/.minetest/worlds/test0/map/map.sqlite /home/user/.minetest/worlds/test0/map.safe/map.sqlite
    umount /home/user/.minetest/worlds/test0/map
  • Make minetest to always run and gracefully shutdown, i used chkconfig:

    Code: Select all

    apt-get install chkconfig
    chkconfig --add minetest
  • Configure minetest by changing file /home/user/.minetest/minetest.conf
    Add something like this:

    Code: Select all

    server_map_save_interval = 500.0
    sqlite_synchronous = 0
    max_simultaneous_block_sends_server_total = 250
    max_simultaneous_block_sends_per_client = 6
    max_clearobjects_extra_loaded_blocks = 30000
    Also spend some time to configure other options.
  • On your local computer create new map and copy your map files to server. Copy map.sqlite to map.safe folder.
  • Make your server restart regularly, for example every night, so map and players can be saved to disk.
  • Restart server and test if game works now.
  • Add your mods(to folder /usr/share/games/minetest/games/minetest_game/mods/) and improvements, make good game, which people could play happily for long time :)
Notes:
  • I used name "user" for user on my Debian server. For map, i used name "test0", take into account if yours is different.
    I heard that maps can easily grow more than 4GB, be aware.
    Player profiles could take more than 100MB too.
    On Windows you can use ImDisk application to create ramdisks.
    Too much bones do heavy load on mobile clients, either disable bones, or remove them regularly with custom mod.
    It is possible that map generator take some CPU to generate new bloks, after some time map could load faster and with less CPU usage.
Please, write here if you succeed!

Andrey.
Last edited by lag01 on Sat Sep 22, 2018 13:11, edited 5 times in total.

User avatar
Calinou
Moderator
Posts: 3169
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

Re: Guide to create fast minetest server on Linux

by Calinou » Post

Code: Select all

active_block_range = 1
max_block_generate_distance = 6
max_block_send_distance = 6
Saves quite a lot of bandwidth of CPU, for almost no downsides (few people use a high enough viewing range to enjoy high distance block sends anyway).

The first line helps a lot when using mobs or mods that heavily rely on entities and ABMs.

twoelk
Member
Posts: 1482
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

Re: Guide to create fast minetest server on Linux

by twoelk » Post

Calinou wrote:... (few people use a high enough viewing range to enjoy high distance block sends anyway).
....
I for one love to climb to some high place after a long journey and then switching on far view look back at the path I have taken. I have loads of screenshots of such views. I have experimented with how far "far view" actually is on VanessaE's creative server. Turned out at the time to be limited to something like 3500 nodes. At least that was how far away you could be from MichaelEh's space station and still see it. The pictures I made of Mt.Meru on Brandon's Landrush server also made use of a viewing distance of nearly 2000 blocks. Some objects or features are only half the fun if you cannot use "far view" once in a while and have that view display a little more the just the immediate surroundings of the places you have just been. I don't know what the viewing range on the Landrush server was at the time but climbing the mountain to have most of it's blocks displayed and kept in memory was some hell of a job. I don't know if roaming the landscape on the "Just Test" server wouldn't be too difficult if I couldn't see my own feet. ;-P

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: Guide to create fast minetest server on Linux

by lag01 » Post

twoelk wrote:I don't know if roaming the landscape on the "Just Test" server wouldn't be too difficult if I couldn't see my own feet. ;-P
Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.

twoelk
Member
Posts: 1482
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

Re: Guide to create fast minetest server on Linux

by twoelk » Post

lag01 wrote:...
Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.
hehe - so now nobody can see my castle retreat when aproaching the little hill it is perching on and scraping the clouds.

Joz
Member
Posts: 41
Joined: Fri Oct 25, 2013 21:37

Re: Guide to create fast minetest server on Linux

by Joz » Post

If you just use redis, which stores many blocks in RAM per default (or leveldb maybe) I think it is very fast also. Nearly as fast as RamFS and more-often backups.
Minetest world settings for redis: http://wiki.minetest.net/Database_backends#Redis
Redis quick start, also for debian: http://redis.io/topics/quickstart
Alternatively sudo apt-get redis-tools or something like this

cesarpachon
Member
Posts: 11
Joined: Sat Nov 22, 2014 04:00
GitHub: cesarpachon
IRC: hydalgo
In-game: hydalgo
Location: colombia
Contact:

Re: Guide to create fast minetest server on Linux

by cesarpachon » Post

@lag01: thank you for the detailed description! I am new to minetest and had been playing for a couple of days with a amazon ec2 instance and this info is really useful!
just a digital hermit
http://www.cesarpachon.com

amadin
Member
Posts: 549
Joined: Tue Jun 16, 2015 16:23

ssh

by amadin » Post

deleted

TuxerP
Member
Posts: 15
Joined: Mon Jun 29, 2015 13:40

Re: Guide to create fast minetest server on Linux

by TuxerP » Post

@lag01
I see that you are running debian jessie which runs systemd. I am running ubuntu 15.10 which also runs systemd. When I add the mount option before start en after the stop daemon in my /etc/init.d/minetest-server script and restart. Nothing happens.
Can you help out?

This is part of my /etc/init.d/minetest-server file:

# Include defaults if available
if [ -f /etc/default/$NAME ] ; then
. /etc/default/$NAME
fi

mount /var/games/minetest-server/.minetest/worlds/world/players
chown Debian-minetest:games /var/games/minetest-server/.minetest/worlds/world/players/
cp -rp /var/games/minetest-server/.minetest/worlds/world/players.safe/* /var/games/minetest-server/.minetest/worlds/world/players/

minetest_start() {
start-stop-daemon \
--start \
--quiet \
--pidfile $PIDFILE \
--oknodo \
--background \
--exec $BINARY \
--startas $DAEMON \
--make-pidfile --chuid $USER \
-- $DAEMON_OPTS > /dev/null 2>&1 || return 1
return 0
}

minetest_stop() {
start-stop-daemon \
--stop \
--quiet \
--pidfile $PIDFILE \
--oknodo \
--exec $BINARY || return 1
rm -f $PIDFILE
return 0
}

#cp -rp /var/games/minetest-server/.minetest/worlds/world/players/* /var/games/minetest-server/.minetest/worlds/world/players.safe/
#umount /var/games/minetest-server/.minetest/worlds/world/players

case "$1" in
start)
log_begin_msg "Starting $DESC: $NAME"
minetest_start
log_end_msg $?
;;

And my fstab file

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/minetest--vg-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/sda1 during installation
UUID=42d02e8c-e515-4162-b1d1-4e57f77f11fe /boot ext2 defaults 0 2
/dev/mapper/minetest--vg-swap_1 none swap sw 0 0
ramfs /var/games/minetest-server/.minetest/worlds/world/players ramfs defaults 0 0

trukoil
Member
Posts: 145
Joined: Mon Oct 22, 2012 03:32
Location: Minetest

Re: Guide to create fast minetest server on Linux

by trukoil » Post

Don't know how LevelDB runs for a server (goes well in singleplayer) but it's just a changed line in world.mt

By the way:
lag01 wrote: Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.
Didn't thought that block != node.

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: Guide to create fast minetest server on Linux

by lag01 » Post

trukoil wrote:Don't know how LevelDB runs for a server (goes well in singleplayer) but it's just a changed line in world.mt

By the way:
lag01 wrote: Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.
Didn't thought that block != node.
Would be nice to have another good option, instead of randisk... But, as i read in wikipedia about LevelDB, there is nothing about casching in memory and there is no indexing.

"Blocks" and "nodes" terminology is used in minetest mapformat description, so i try to honor that naming convention.

User avatar
ExeterDad
Member
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
In-game: ExeterDad
Location: New Hampshire U.S.A

Re: Guide to create fast minetest server on Linux

by ExeterDad » Post

But, as i read in wikipedia about LevelDB, there is nothing about casching in memory
No need to cache in memory. The database IS in memory. That's why it's so fast. But... you must have enough memory to accommodate the database or you will run out of memory for the server to run on.

TuxerP
Member
Posts: 15
Joined: Mon Jun 29, 2015 13:40

Re: Guide to create fast minetest server on Linux

by TuxerP » Post

I am trying to run a server that you can play from a mobile device. Mobile devices have limited amounts of cpu and memory, so server optimization is a thing that is important. I have tuned my minetest.conf settings so the amount of data send to the mobile device is not to much to have good fps and so little lag as possible.
Running on leveldb, redis or running players and maps in memory will be a faster option than running sqlite I think.
Also I try to run stable mods.

User avatar
ExeterDad
Member
Posts: 1717
Joined: Sun Jun 01, 2014 20:00
In-game: ExeterDad
Location: New Hampshire U.S.A

Re: Guide to create fast minetest server on Linux

by ExeterDad » Post

TuxerP wrote:I am trying to run a server that you can play from a mobile device. Mobile devices have limited amounts of cpu and memory, so server optimization is a thing that is important. I have tuned my minetest.conf settings so the amount of data send to the mobile device is not to much to have good fps and so little lag as possible.
Running on leveldb, redis or running players and maps in memory will be a faster option than running sqlite I think.
Also I try to run stable mods.
From my limited experience using Android on our server. Less is more. Using games or mods heavy in textures/sounds will kill our devices the fastest. And as you said, it boils down to limited memory. All media is downloaded to the devices when logging in. Even textures for nodes that you may never see the entire time you are logged in with the device. My Android, and it's not too shabby... will crash while loading the media when our server is running a BFD instance. It plays a default game map like a dream. But the heavier it gets, the more likely it will crash.

amadin
Member
Posts: 549
Joined: Tue Jun 16, 2015 16:23

Re: Guide to create fast minetest server on Linux

by amadin » Post

I think this guide the developers must put in wiki, because with default minetest.conf - parts of the world loading very-very slow if more than 30 people online on powerfull server.

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: Guide to create fast minetest server on Linux

by lag01 » Post

Right now i am working on improving this ramdisk setup.
Here i attached halfway ready code for using ramdisk.
Also other useful functions, like backup, clear player profiles, keep service up.

To use this install, minetest must be installed, and there must be correct user and world folder names. Also map.sqlite and debug.txt must be present in the right places.

Can somebody help me improve install.sh to automatically add crontab entries (like in crontab.txt)?
Attachments
RAMDISK_setup.tar.gz
(2.51 KiB) Downloaded 298 times

amadin
Member
Posts: 549
Joined: Tue Jun 16, 2015 16:23

Re: Guide to create fast minetest server on Linux

by amadin » Post

Do you can add parameters of your server (processor, hard drive (ssd, sata or ata), ram (ddr or sdram), network speed) or add it to your post here viewtopic.php?p=162449#p162449?

User avatar
Calinou
Moderator
Posts: 3169
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

Re: Guide to create fast minetest server on Linux

by Calinou » Post

If you have spare CPU and bandwidth, you can use this:

Code: Select all

dedicated_server_step = 0.05
This will make player/mob movement appear to be smoother and more accurate (and any other action that occurs every server step).

BBmine
Member
Posts: 3476
Joined: Sun Jul 12, 2015 22:51
GitHub: BBmine
IRC: BBmine
In-game: Baggins
Location: USA

Re: Guide to create fast minetest server on Linux

by BBmine » Post

Nice, lag :)
Any idea on why my server is not announcing to the serverlist, but other people from all over the world can still join?

User avatar
lag01
Member
Posts: 321
Joined: Sun Mar 16, 2014 03:41
GitHub: AndrejIT
IRC: lag01
In-game: lag
Contact:

Re: Guide to create fast minetest server on Linux

by lag01 » Post

BBmine wrote:Nice, lag :)
Any idea on why my server is not announcing to the serverlist, but other people from all over the world can still join?
Thanks Baggins!
Not sure about your server. Did you check minetest.conf?

Code: Select all

server_address = 45.21.218.32
server_announce = true
serverlist_url = servers.minetest.net

BBmine
Member
Posts: 3476
Joined: Sun Jul 12, 2015 22:51
GitHub: BBmine
IRC: BBmine
In-game: Baggins
Location: USA

Re: Guide to create fast minetest server on Linux

by BBmine » Post

lag01 wrote:
BBmine wrote:Nice, lag :)
Any idea on why my server is not announcing to the serverlist, but other people from all over the world can still join?
Thanks Baggins!
Not sure about your server. Did you check minetest.conf?

Code: Select all

server_address = 45.21.218.32
server_announce = true
serverlist_url = servers.minetest.net
Beerholder showed me what it was :D I should've done this:

Code: Select all

curl_timeout = 10000
Apparently the default is too low.

Post Reply

Who is online

Users browsing this forum: No registered users and 19 guests