minetestserver consistently crashing on Oracle Free Cloud server

[pupper1337]

Hi! I want to share my experience/challenges with running MineClone 2 on a cloud server, and see if anyone can help me troubleshoot :)

TL;DR: I was able to get Minetest 5.4.1 running with MineClone2 0.76.0. However, I could NOT get Minetest 5.6.0 running with Mineclone2 0.82.0

Context: I want to set up a Minetest server in the cloud for me and my friends to play. I want to use cloud hosting rather than self-hosting on a local device, because my internet connection is spotty, and I want the server to have reliable uptime.

Cloud: I used Oracle's free cloud service, because it's free, and gives you 4 powerful ARM Ampere CPUs, and 24 GB of RAM https://www.oracle.com/cloud/free/. I'm running Ubuntu Linux on this Virtual Private Server (VPS).

Problems: There are two problems with this setup:
1. Minetest doesn't provide a self-contained Minetest zip folder on Linux, unlike Windows. So in order to run a Minetest server, I would have to get an outdated version from the repositories, or compile Minetest myself.
2. I'm starting to suspect that the ARM architecture is causing my problems. I've been able to run a Minetest server on my AMD-based laptop without issues.

Failures and Successes:

After setting up Ubuntu on my Oracle Cloud server, I've tried all of the following to get my Minetest server running:

Failure 1: I tried compiling Minetest 5.6.0 on my Oracle Cloud VPS, and running it with Mineclone2 0.82.0.
I chose these versions because they were listed on this Mineclone2 release thread as compatible: viewtopic.php?t=16407

Result: My friends and I can connect and play Mineclone2, but the server consistently crashes during the 1st night. We checked the debug.txt logs in the Minetest folder, but didn't find anything useful.

Here's an example of the segmentation fault crash that happened. All the crashes mentioned below showed a similar message:

Spoiler

Code: Select all

2023-01-23 10:25:59: ACTION[Server]: [mcl_mobs] Mob mobs_mc:spider despawns at (-5.7,-21.5,39.8) lifetimer ran out
2023-01-23 10:26:01: ACTION[Server]: [mcl_mobs] Mob mobs_mc:skeleton spawns on mcl_core:granite at (-7.0,-5.0,14.0)
2023-01-23 10:26:01: WARNING[Server]: [mcl_mobs] mob mobs_mc:enderman refused to spawn at (-7,-5,14)
2023-01-23 10:26:03: ACTION[Server]: [mcl_mobs] Mob mobs_mc:enderman despawns at (-7.8,-23.1,44.0) lifetimer ran out
2023-01-23 10:26:18: ACTION[Server]: [mcl_mobs] Mob mobs_mc:creeper despawns at (-21.9,1.7,-41.4) lifetimer ran out
2023-01-23 10:26:23: ACTION[Server]: [mcl_mobs] Mob mobs_mc:zombie despawns at (44.2,1.5,38.5) lifetimer ran out
Segmentation fault (core dumped)

Minetest game works without crashing.

Failure 2: I tried compiling Minetest 5.5.1 on my Oracle Cloud VPS, and running it with Mineclone2 0.80.1.
I chose these versions because they were listed on this Mineclone2 release thread as compatible: viewtopic.php?t=16407

Result: Mineclone2 was stable when I played as the only person on the server. But when my friends joined the server, it crashed after about 23 minutes. It ran longer without crashing than with Minetest 5.6.0 and Mineclone2 0.82.0.

Failure 3: I tried compiling Minetest 5.4.1 on my Oracle Cloud VPS.

Result: I didn't even get to run the server because it had build errors.

Failure 4: I tried compiling Minetest 5.6.0 on my Oracle Cloud VPS, and tried to run it with Mineclone2 0.80.1.

Result: The server kept immediately crashing.

Success 1: I tried installing Minetest 5.4.1 via Ubuntu's repositories

Code: Select all

sudo apt install minetest-server

on my Oracle Cloud VPS, and tried to run it with Mineclone2 0.76.0.

Result: My friends and I were able to play without crashing for ~ 7 hours straight.

Success 2: I tried compiling Minetest 5.6.0 on my own laptop, and running it with Mineclone2 0.82.0. I did this as a direct comparison to the Failure 1 (on my Oracle Cloud server).

Result: My friends and I were able to play without crashing for ~ 4 hours straight. This shows that I was following the compiling directions correctly, but it still wasn't working on my Oracle Cloud server for some reason.

Observations:

- When the server does crash, it either crashes immediately, or during the first night. I'm not sure if the crashing has something to do with monsters spawning, but that's a consistent pattern. I posted this here because the server versions that crashed when running with Mineclone2 seemed to work fine with Minetest game.
- I was able to get Minetest 5.6.0 and Mineclone2 0.82.0 running without crashes on my personal laptop. These same versions consistently crashed on my Oracle Cloud server.

Please help:

I know my setup is pretty niche (running on an ARM-based cloud server), but I was so excited to get my friends off Minecraft and onto the FOSS Minetest, and I want to be able to play with the latest version and feature updates to Minetest and Mineclone2.

I'd really appreciate your help! :)

[sfan5]

Can you show the output of minetestserver --version?
There are known crash issues with LuaJIT on ARM64 where you may have to build it from source to get a working binary.

[Festus1965]

Observations:

- When the server does crash, it either crashes immediately, or during the first night. I'm not sure if the crashing has something to do with monsters spawning, but that's a consistent pattern. I posted this here because the server versions that crashed when running with Mineclone2 seemed to work fine with Minetest game.

Why didn't you just disable the mods you think are responsible and let run again? Soon you have an answer if it is as of them, and check the debug.txt file, withing the last around 10 lines there is a crash report included, pointing nice on the causing problem.

Two way testing:
* disable all mods and add one by one
* disable one by one alone to see the reaction
with run each version 24 hours - if not crash before.

[pupper1337]

Can you show the output of minetestserver --version?
There are known crash issues with LuaJIT on ARM64 where you may have to build it from source to get a working binary.

Here's the minetestserver --version for the Minetest 5.6.0 build I compiled, that crashed when playing with friends:

Code: Select all

Minetest 5.6.0 (Linux)
Using LuaJIT 2.1.0-beta3
BUILD_TYPE=Release
RUN_IN_PLACE=1
USE_CURL=1
STATIC_SHAREDIR="."
STATIC_LOCALEDIR="locale"

Here's the minetestserver --version for the Minetest 5.6.0 build I installed from apt that works fine when playing with friends:

Code: Select all

Minetest 5.4.1 (Linux)
Using Lua 5.1.5
BUILD_TYPE=Release
RUN_IN_PLACE=0
USE_CURL=1
STATIC_SHAREDIR="/usr/share/games/minetest"
STATIC_LOCALEDIR="/usr/share/locale"

Are you referring to these Github issues where you responded?
https://github.com/minetest/minetest/issues/12864
https://github.com/minetest/minetest/is ... 1276255538

Why didn't you just disable the mods you think are responsible and let run again? Soon you have an answer if it is as of them, and check the debug.txt file, withing the last around 10 lines there is a crash report included, pointing nice on the causing problem.

Two way testing:
* disable all mods and add one by one
* disable one by one alone to see the reaction
with run each version 24 hours - if not crash before.

Here are the last few lines from debug.txt

Code: Select all

2023-01-26 03:36:19: ERROR[Main]: ServerError: AsyncErr: Lua: Runtime error from mod '??' in callback luaentity_Step(): .../games/mineclone2/mods/ENTITIES/mcl_mobs/effects.lua:301: bad argument #1 to 'random' (interval is empty)
2023-01-26 03:36:19: ERROR[Main]: stack traceback:
2023-01-26 03:36:19: ERROR[Main]:       [C]: in function 'random'
2023-01-26 03:36:19: ERROR[Main]:       .../games/mineclone2/mods/ENTITIES/mcl_mobs/effects.lua:301: in function 'who_are_you_looking_at'
2023-01-26 03:36:19: ERROR[Main]:       .../games/mineclone2/mods/ENTITIES/mcl_mobs/effects.lua:328: in function 'check_head_swivel'
2023-01-26 03:36:19: ERROR[Main]:       ...n/../games/mineclone2/mods/ENTITIES/mcl_mobs/api.lua:360: in function 'on_step_old'
2023-01-26 03:36:19: ERROR[Main]:       ...mineclone2/mods/ENVIRONMENT/mcl_void_damage/init.lua:24: in function <...mineclone2/mods/ENVIRONMENT/mcl_void_damage/init.lua:23>

It does seem like the error has to do with Lua, and involves mineclone2 assets...

However, I don't want to give up on the mineclone2 game, since it's so close to the Minecraft experience that my friends and I really like it.

[Blockhead]

This stack trace leads to some very interesting findings. This part of the code decides whether a mob will swivel its head to look at the player due to being idle.

I can tell you are using PUC Lua (the one from the official Lua project, or rather, the one based on it that ships with Minetest) and not LuaJIT because only PUC Lua's implementation of math.random has the "interval is empty". There are two requirements there: that the first argument must be >= the second, and if there is only one argument, the argument shall not be less than 1.

The equation is 20/self.curiosity. A division occurs on self.curiosity, a stat that is constant for a kind of mob. By searching for curiosity figures, I found that the wither skeleton has a curiosity of 60, which is greater than 20, and so will yield a result less than 1. Under LuaJIT, any value less than 1 will just act as if it were the value 1.

I placed a heap of wither skeletons with a spawn egg on my usual LuaJIT version. I was in creative to make sure the withers would be idle and not attack me. No crashes. I compiled 5.6.1 with Minetest builtin Lua and loaded the same world. Instant crash. I have posted a bug report over on Minetest Land where Mineclone2 development happens.

[sfan5]

Here's the minetestserver --version for the Minetest 5.6.0 build I compiled, that crashed when playing with friends:
[...]

Here's the minetestserver --version for the Minetest 5.6.0 build I installed from apt that works fine when playing with friends:
[...]

Are you referring to these Github issues where you responded?
https://github.com/minetest/minetest/issues/12864
https://github.com/minetest/minetest/is ... 1276255538

Yes, that's it.
The last released version - LuaJIT 2.1.0-beta3 - is itself several years old and also affected so you should try to compile LuaJIT git and then Minetest 5.6.1 from source.