Improving server performance / Database backends

Post Reply
bark
Member
Posts: 35
Joined: Thu Sep 24, 2015 13:25
In-game: bark

Improving server performance / Database backends

by bark » Post

I've been running my server, Barkhouse, for almost a year now, and haven't experienced any particular issues with lag during this time. In the last month, however, I have been starting to get lag spikes about every 5-10 seconds (mainly when players are moving into unloaded chunks), lasting a couple of seconds to maybe 20 seconds every time. This started happening around the time new regions of the map was explored recently, as far as 9000 nodes from spawn.

Server specs
Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
6 GB RAM DDR3.
OS: Trisquel 7 (ubuntu derivative)

A priority should probably be to upgrade the RAM

But I have noticed that whenever the lag spikes occur, CPU usage peaks to 100% on one of the server cores. So maybe a RAM increase won't even fix this problem on its own?

edit: I have adjusted my minetest.conf according to this thread, that significantly reduced the lag spikes.

Database backends
Switching the database backend of minetest can apparently give the server a huge performance increase. I found the Database backends page in the minetest wiki useful, but also a bit scarce on information.

The default database backend for minetest is SQLite3, which is "Acceptable" in Speed and Reliability. The two other ones, LevelDB and Redis are either "Good" or "Very Good" (see table on wiki page).

So which one to choose? LevelDB seems like the safest way to go, based on
LevelDB is mostly used on servers because of its improved reliability and the ability to use more than 4 GB of space. You need to install it additionally.
But then a search for "leveldb minetest" returns mainly this post from LazyJ's blog, titled "LevelDB in Minetest is Just Not Worth it" which basically says that it's a terrible idea to use it.

In addition, I can't seem to find any documentation on how to install and activate LevelDB for minetest. Any further help and/or pointers on this would be greatly appreciated.

See also: Can SQLite3-databases NOT grow bigger than 4 gb? What would happen if they did?
+ Spoiler
Then there is Redis, which supposedly is even faster, but also is more volatile, because the database is stored in memory only, and written to harddrive occasionally (ccould someone please elaborate on this?)

In addition, a server with Redis database backend can run MTSatellite for a live webmap which is a huge plus in my opinion.

Redis also has more install instructions than the latter, but not much. I did find this thread, Database Backend Questions, extremely useful in this regard, especially the thorough walkthrough on the install reply by joshburt;
+ Spoiler
Feedback on this is greatly appreciated. Which database should I choose? Are there other steps I can take in addition to this?
Last edited by bark on Thu Jun 30, 2016 18:42, edited 6 times in total.

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: Improving server performance / Database backends

by burli » Post

there is now also a Postgresql backend. You should try this. But it is not listed in the wiki. Search in the forum

User avatar
TenPlus1
Member
Posts: 3039
Joined: Mon Jul 29, 2013 13:38
GitHub: tenplus1

Re: Improving server performance / Database backends

by TenPlus1 » Post

To make a leveldb world create a new map then run it once only, then close minetest and edit the world.mt file inside of the world folder, change the following line:

backend = sqlite3

..to..

backend = leveldb

..and run the map, it will now be copied across to a level db database and used from then on.

bark
Member
Posts: 35
Joined: Thu Sep 24, 2015 13:25
In-game: bark

Re: Improving server performance / Database backends

by bark » Post

Thanks for the reply. I guess I made this seem more difficult in my head than it was.

So if I instead want to migrate an existing world I can run (with minetest not running)

minetestserver --migrate leveldb path/to/world?

I have discovered that LevelDB must be installed, and then I have to compile minetest using the DENABLE_LEVELDB=1 flag.
Calinou wrote:In Debian and Ubuntu, LevelDB is available as libleveldb-dev. In Fedora, it is leveldb-devel.

bark
Member
Posts: 35
Joined: Thu Sep 24, 2015 13:25
In-game: bark

Re: Improving server performance / Database backends

by bark » Post

Ok, so I installed libleveldb-dev, and managed to successfully compile minetest from source using the DENABLE_LEVELDB-flag with the help of Calinou's 1-line compile script (Thanks Calinou!)

When I'm trying to migrate my existing map to LevelDB, I get the following error:

Code: Select all

$ minetestserver --migrate leveldb worlds/testmap/
ERROR[main]: Migration to leveldb is not supported
Does this mean that existing sqlite databases simply cannot be converted to leveldb? Or did I do something wrong?

@burli: Thanks for the advise. I have checked out the thread on Postgresql, and got a bit scared from the initial sentence, as I don't have much (actually no) experience with Postgresql or any other database for that matter.
Johnny Joy wrote:The PostgreSQL backend is not for everyone. It requires some experience with
PostgreSQL or a willingness to learn. This is not something for the casual user.
Still might try this eventually.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest