[mod] FFi_accel, accelerator for map generators [ffi_accel]

Tikilou
New member
 
Posts: 3
Joined: Fri May 03, 2019 17:46

[mod] FFi_accel, accelerator for map generators [ffi_accel]

by Tikilou » Sat May 04, 2019 07:31

Since this thread was created after the last backup, here there is the fork of FFIopt (latest post on the thread before crash), with optimizations and better security (thanks to pgimeno) :

Description :

LuaJIT FFI-based accelerator for map generators

This mod is an accelerator for Minetest map generators written in Lua. It accelerates the memory-intensive functions in LuaVoxelManip and PerlinNoiseMap, by adding an emulation layer on top of them that allows these functions to run faster.

It additionally prevents the usage of the extra LuaJIT memory needed to hold the data, leaving it in the C++ memory area, making it less likely that LuaJIT crashes due to memory problems.

There are cons, too. First, it works by implementing a sort of emulation layer, making the mod believe it's writing to a Lua table when it's actually writing to the C++ memory directly. In some circumstances, this could not work in the way it's designed to work, potentially leading to mod misbehaviour or even crashes (e.g. if the table is written to after removing the PerlinNoiseMap or VoxelManip object). Fortunately, these cases are expected to be rare. But be careful to only use mods that you trust, since that kind of access could be exploitable by a rogue mod.

Second, while it accelerates the functions that read and write the data back (by replacing these operations with just a simple setup of a metatable), the reading and writing of the table data will be slower, because it's redirected through a function that performs the access to the C++ memory after doing bounds checks.

Finally, for some mods like WorldEdit there may be no acceleration at all.

Result is :

Faster data retreiving and storing ;
Slower table access (essencially due to use of metatable avoiding mods code change) ;
Lower Lua memory footprint (avoiding many OOM errors) ;



ffi_accel => https://notabug.org/pgimeno/ffi_accel

If you want to patch the minetest 5.0.1 source code, and compile/install it on Ubuntu and Debian-based distros (don't forget dependencies) :

Code: Select all
git clone -b stable-5 https://github.com/minetest/minetest.git


Code: Select all
cd minetest/games/


Code: Select all
git clone https://github.com/minetest/minetest_game.git


Code: Select all
cd ../


Code: Select all
wget https://notabug.org/pgimeno/ffi_accel/raw/master/ffi_accel.patch


Code: Select all
patch -p1 < ffi_accel.patch


Code: Select all
cmake . -DENABLE_GETTEXT=1 -DENABLE_FREETYPE=1 -DENABLE_LEVELDB=1 -DENABLE_REDIS=1 -DBUILD_CLIENT=0 -DCMAKE_INSTALL_PREFIX:PATH=/opt/minetest-server/5.0.1-ffi_accel


Code: Select all
make -j$(grep -c processor /proc/cpuinfo)


Code: Select all
sudo make install


Code: Select all
sudo ln -s /opt/minetest-server/5.0.1-ffi_accel/bin/minetestserver /usr/bin/minetestserver_5.0.1-ffi_accel


And for launching it :

Code: Select all
sudo minetestserver_5.0.1-ffi_accel --config /etc/minetest/minetest_5.0.1-ffi_accel.conf --logfile /var/log/minetest/minetest_5.0.1-ffi_accel.log



After that you can make a service :

Code: Select all
sudo adduser --quiet --system --home /var/games/minetest-server/5.0.1-ffi_accel --no-create-home --gecos "Minetest ffi_accel dedicated server 5.0.1" --ingroup games --force-badname Ubuntu-minetest


Code: Select all
sudo nano /lib/systemd/system/minetest-server_5.0.1-ffi_accel.service


Code: Select all
[Unit]
Description=Minetest multiplayer server minetest.conf server config
Documentation=man:minetestserver(6)
After=network.target
RequiresMountsFor=/var/games/minetest-server/5.0.1-ffi_accel/

[Service]
Restart=on-failure
User=Ubuntu-minetest
Group=games
ExecStart=/usr/bin/minetestserver_5.0.1-ffi_accel --config /etc/minetest/minetest_5.0.1-ffi_accel.conf --logfile /var/log/minetest/minetest_5.0.1-ffi_accel.log
StandardOutput=null

[Install]
WantedBy=multi-user.target


Make this file too :

Code: Select all
sudo nano /etc/init.d/minetest-server_5.0.1-ffi_accel


Code: Select all
#! /bin/sh
### BEGIN INIT INFO
# Provides:          minetest-server
# Required-Start:    $remote_fs $network
# Required-Stop:     $remote_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start minetest network game server
# Description:       dedicated game server for Minetest
### END INIT INFO

PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
NAME="minetestserver_5.0.1-ffi_accel"
DAEMON="/usr/bin/$NAME"
DESC="minetest network game server"
PIDFILE="/var/run/$NAME.pid"
BINARY="/usr/bin//$NAME"
USER="Ubuntu-minetest"

test -x $DAEMON || exit 0

. /lib/lsb/init-functions

# Include defaults if available
if [ -f /etc/default/$NAME ] ; then
    . /etc/default/$NAME
fi

minetest_start() {
    start-stop-daemon \
        --start \
        --quiet \
        --pidfile $PIDFILE \
        --oknodo \
        --background \
        --exec $BINARY \
        --startas $DAEMON \
        --make-pidfile --chuid $USER \
        -- $DAEMON_OPTS > /dev/null 2>&1 || return 1
    return 0
}

minetest_stop() {
    start-stop-daemon \
        --stop \
        --quiet \
        --pidfile $PIDFILE \
        --oknodo \
        --exec $BINARY || return 1
    rm -f $PIDFILE
    return 0
}

case "$1" in
    start)
        log_begin_msg "Starting $DESC: $NAME"
        minetest_start
        log_end_msg $?
    ;;
    stop)
        log_begin_msg "Stopping $DESC: $NAME"
        minetest_stop
        log_end_msg $?
    ;;
    restart|force-reload)
        log_begin_msg "Restarting $DESC: $NAME"
        minetest_stop && sleep 1 && minetest_start
        log_end_msg $?
    ;;
    status)
    status_of_proc -p "$PIDFILE" "$DAEMON" "$NAME" && exit 0 || exit $?
    ;;
    *)
    # echo "Usage: $0 {start|stop|restart|reload|force-reload}" >&2
    echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2
    exit 1
    ;;
esac

exit 0


Then enable the service :

Code: Select all
sudo systemctl enable minetest-server_5.0.1-ffi_accel


And start it :

Code: Select all
sudo systemctl start minetest-server_5.0.1-ffi_accel


You can see logs with that command :

Code: Select all
tail -f /var/log/minetest/minetest_5.0.1-ffi_accel.log


Conf server file is inside /etc/minetest/minetest_5.0.1-ffi_accel.conf and mods and worlds will be inside /var/game/minetest-server/5.0.1-ffi_accel/.minetest

After that, don't forget to get the mod too, installing it inside the "mods" folder, and enabling it inside the world.mt file.
 

Return to WIP Mods



Who is online

Users browsing this forum: No registered users and 5 guests