[mod] FFi_accel, accelerator for map generators [ffi_accel]

New member
Posts: 3
Joined: Fri May 03, 2019 17:46

[mod] FFi_accel, accelerator for map generators [ffi_accel]

by Tikilou » Sat May 04, 2019 07:31

Since this thread was created after the last backup, here there is the fork of FFIopt (latest post on the thread before crash), with optimizations and better security (thanks to pgimeno) :

Description :

LuaJIT FFI-based accelerator for map generators

This mod is an accelerator for Minetest map generators written in Lua. It accelerates the memory-intensive functions in LuaVoxelManip and PerlinNoiseMap, by adding an emulation layer on top of them that allows these functions to run faster.

It additionally prevents the usage of the extra LuaJIT memory needed to hold the data, leaving it in the C++ memory area, making it less likely that LuaJIT crashes due to memory problems.

There are cons, too. First, it works by implementing a sort of emulation layer, making the mod believe it's writing to a Lua table when it's actually writing to the C++ memory directly. In some circumstances, this could not work in the way it's designed to work, potentially leading to mod misbehaviour or even crashes (e.g. if the table is written to after removing the PerlinNoiseMap or VoxelManip object). Fortunately, these cases are expected to be rare. But be careful to only use mods that you trust, since that kind of access could be exploitable by a rogue mod.

Second, while it accelerates the functions that read and write the data back (by replacing these operations with just a simple setup of a metatable), the reading and writing of the table data will be slower, because it's redirected through a function that performs the access to the C++ memory after doing bounds checks.

Finally, for some mods like WorldEdit there may be no acceleration at all.

Result is :

Faster data retreiving and storing ;
Slower table access (essencially due to use of metatable avoiding mods code change) ;
Lower Lua memory footprint (avoiding many OOM errors) ;

ffi_accel => https://notabug.org/pgimeno/ffi_accel

If you want to patch the minetest 5.0.1 source code, and compile/install it on Ubuntu and Debian-based distros (don't forget dependencies) :

Code: Select all
git clone -b stable-5 https://github.com/minetest/minetest.git

Code: Select all
cd minetest/games/

Code: Select all
git clone https://github.com/minetest/minetest_game.git

Code: Select all
cd ../

Code: Select all
wget https://notabug.org/pgimeno/ffi_accel/raw/master/ffi_accel.patch

Code: Select all
patch -p1 < ffi_accel.patch

Code: Select all

Code: Select all
make -j$(grep -c processor /proc/cpuinfo)

Code: Select all
sudo make install

Code: Select all
sudo ln -s /opt/minetest-server/5.0.1-ffi_accel/bin/minetestserver /usr/bin/minetestserver_5.0.1-ffi_accel

And for launching it :

Code: Select all
sudo minetestserver_5.0.1-ffi_accel --config /etc/minetest/minetest_5.0.1-ffi_accel.conf --logfile /var/log/minetest/minetest_5.0.1-ffi_accel.log

After that you can make a service :

Code: Select all
sudo adduser --quiet --system --home /var/games/minetest-server/5.0.1-ffi_accel --no-create-home --gecos "Minetest ffi_accel dedicated server 5.0.1" --ingroup games --force-badname Ubuntu-minetest

Code: Select all
sudo nano /lib/systemd/system/minetest-server_5.0.1-ffi_accel.service

Code: Select all
Description=Minetest multiplayer server minetest.conf server config

ExecStart=/usr/bin/minetestserver_5.0.1-ffi_accel --config /etc/minetest/minetest_5.0.1-ffi_accel.conf --logfile /var/log/minetest/minetest_5.0.1-ffi_accel.log


Make this file too :

Code: Select all
sudo nano /etc/init.d/minetest-server_5.0.1-ffi_accel

Code: Select all
#! /bin/sh
# Provides:          minetest-server
# Required-Start:    $remote_fs $network
# Required-Stop:     $remote_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start minetest network game server
# Description:       dedicated game server for Minetest

DESC="minetest network game server"

test -x $DAEMON || exit 0

. /lib/lsb/init-functions

# Include defaults if available
if [ -f /etc/default/$NAME ] ; then
    . /etc/default/$NAME

minetest_start() {
    start-stop-daemon \
        --start \
        --quiet \
        --pidfile $PIDFILE \
        --oknodo \
        --background \
        --exec $BINARY \
        --startas $DAEMON \
        --make-pidfile --chuid $USER \
        -- $DAEMON_OPTS > /dev/null 2>&1 || return 1
    return 0

minetest_stop() {
    start-stop-daemon \
        --stop \
        --quiet \
        --pidfile $PIDFILE \
        --oknodo \
        --exec $BINARY || return 1
    rm -f $PIDFILE
    return 0

case "$1" in
        log_begin_msg "Starting $DESC: $NAME"
        log_end_msg $?
        log_begin_msg "Stopping $DESC: $NAME"
        log_end_msg $?
        log_begin_msg "Restarting $DESC: $NAME"
        minetest_stop && sleep 1 && minetest_start
        log_end_msg $?
    status_of_proc -p "$PIDFILE" "$DAEMON" "$NAME" && exit 0 || exit $?
    # echo "Usage: $0 {start|stop|restart|reload|force-reload}" >&2
    echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2
    exit 1

exit 0

Then enable the service :

Code: Select all
sudo systemctl enable minetest-server_5.0.1-ffi_accel

And start it :

Code: Select all
sudo systemctl start minetest-server_5.0.1-ffi_accel

You can see logs with that command :

Code: Select all
tail -f /var/log/minetest/minetest_5.0.1-ffi_accel.log

Conf server file is inside /etc/minetest/minetest_5.0.1-ffi_accel.conf and mods and worlds will be inside /var/game/minetest-server/5.0.1-ffi_accel/.minetest

After that, don't forget to get the mod too, installing it inside the "mods" folder, and enabling it inside the world.mt file.

Return to WIP Mods

Who is online

Users browsing this forum: No registered users and 6 guests