[TOOL]mtmediasrv[mtmediasrv] - Minetest Remote Media Server

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

[TOOL]mtmediasrv[mtmediasrv] - Minetest Remote Media Server

by sofar » Wed Apr 19, 2017 07:47

A Minetest Remote Media server implementation (as fcgi server).

This program is useful to distribute minetest media (textures, models, sounds) to minetest clients for multiplayer server owners that wish to have their media hosted on a `remote media server` URL. Doing this as a separate download removes some of the download bandwidth from the actual game server and offloads it to a different HTTP server. This will work for clients that have cURL support enabled.


Requirements

  • nginx or apache, with fcgi
  • webserver must be able to access file sockets in /run/mtmediasrv
  • systemd for controlling the service startup
  • (optional) go to build the service(there are binary builds available for Linux)

This program is intended to run as fcgi process and handle POST requests for the `/index.mth` URI. It listens on a local unix domain socket, and needs to read the media files in the media folder to create sha1 hashes. It creates a hash list of files it has available and keeps this in memory.

At startup, the program can optionally scan mods and subgames to find and copy or hardlink (the default) all the assets into the webroot. The hardlink method is better for space, but may fail if the media is not on the same filesystem as the webroot.

When a client connects, the client POSTS their list of known sha1 hashes of files they need.

The program verifies for each needed hash that the server has the hash listed, and returns the needed hashes only, and only hashes that it has the files for.

The actual file content is not served by this program, for this you need to have your web server serve that content as static files.


Building

Run `go build` in this folder to create the binary `mtschemsrv`.

Please note the currently hardcoded values in the binary and example configuration files.


Installation

Several example config and service units are provided as an example on how to deploy this service.

Once you have configured your web server properly so that it serves up the static content files by hash, point the mtmediasrv to the same folder and it will talk to minetest clients that request media automatically. If the content changes, you need to restart the program.

You should not have a file called `index.mth` in the media folder, although this will not break anything, it will probably be confusing.

opy and edit the `mtmediasrv.yaml` file and point it at the proper webroot, socket path, and mediapath entries. Place it in `/etc/` or `~/.config/`.

logging

The program logs all clients and writes out the following data to the log output (journal/syslog):

  • remote address
  • user agent
  • hashes needed/hashes given
  • content-length of sent bytes

Example output:

Code: Select all
mtmediasrv: 5.4.3.2:34567 'Minetest/0.4.15 (Linux/4.10.1 x86_64)' 64/64 1286



Github Project: https://github.com/sofar/mtmediasrv
License: AGPL-3.0
Download: (binaries!) https://github.com/sofar/mtmediasrv/releases/tag/v1
Download: (source!) https://github.com/sofar/mtmediasrv/archive/master.zip
 

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest fcgi Media Ser

by sofar » Wed Apr 19, 2017 07:48

TODO: currently this still requires web servers to provide a file of hashed-named-files in the media folder. I intend to make this easier and make the daemon create those files for the hoster automatically so you can just point the daemon at your mod folders.
 

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest fcgi Media Ser

by sofar » Wed Apr 19, 2017 18:31

DONE: the program is now entirely self-sufficient. You will not need any shell script or external tools to collect and serve remote media. Just edit and put the config file in place and point it at your webroot and asset folders.
 

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest Remot Media Se

by sofar » Thu Apr 27, 2017 05:39

Changed the topic to be a bit more specific - "Minetest Remote Media Server"
 

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest Remot Media Se

by sofar » Thu May 11, 2017 04:59

I will post binary builds so this is even easier to deploy. I've added 32bit and 64bit binaries for Linux on the `releases` page.
 

User avatar
texmex
Member
 
Posts: 580
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: texmex

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest Remot Media Se

by texmex » Thu May 11, 2017 08:41

Hi sofar! This sounds great for large server admins. Is there a gain to be had from this from small servers to or is it not worth the hassle?
 

sofar
Developer
 
Posts: 909
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [TOOL]mtmediasrv[mtmediasrv] - A Minetest Remot Media Se

by sofar » Thu May 11, 2017 17:19

texmex wrote:Hi sofar! This sounds great for large server admins. Is there a gain to be had from this from small servers to or is it not worth the hassle?


It should always be beneficial, even for smaller servers, because the HTTP downloads are async in the client, which means that while the main server thread is running (working to send already logged in players game data stuff) and sending blocks, the client can download media over HTTP at the same time, and HTTP downloads are very small and efficient.

Of course, a second benefit is that the remote media server can run on a different server, thus offloading bandwidth to a completely different IP address.
 

User avatar
texmex
Member
 
Posts: 580
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: texmex
 


Return to Minetest-related projects



Who is online

Users browsing this forum: No registered users and 1 guest