Public Remote Media Server Project

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Public Remote Media Server Project

by sofar » Wed Dec 06, 2017 21:58

I've created a fully public remote media server, and it is available for all minetest servers.

This remote media server provides clients with media for your servers, so that your server does not have to provide this download to your players when they connect. This generally speeds up connections and lowers the pressure on your server, and so it's a good thing!

I've taken the "beta" label off this server as I don't foresee any major changes to the project - You should be reminded to check this thread regularly to make sure we haven't announced any changes. As always, this is a volunteer project and we may change policies, stop it, and/or do other things with it if needed. I invite everyone to come help out and use the server.

How do I use this remote media server?

If you're a player, you don't need to do anything.

If you're a server operator, do these things:

1. Put the following entry in your minetest server configuration:

Code: Select all
remote_media = https://minetestmedia.foo-projects.org/


(http is also supported, but you should prefer https)

2. Restart your minetest server.

3. (optional) Post in this thread that you've done this on your server!

4. (optional, but recommended) Post mods by github/gitlab/etc. project URL in this thread, so we can add these mods to the media server and keep them up to date automatically.

5. (required) Check back in regularly about the progress of this project, by reading this thread once in a while. We'll post updates, statistics and mod changes in here.

This project is using mtmediasrv https://forum.minetest.net/viewtopic.php?f=14&t=17411 to serve the media content to clients - this is an efficient implementation that allows the media server to host thousands of files without any problems and not overloading your clients with unneeded data.

A note on mod requirements: mods and assets should be under a freely redistributable license or otherwise free of restrictions for commercial use and allow derivatives. We may at our discretion choose to reject the inclusion of mods for any reason.

You can get a list of currently included mods from this URL: https://minetestmedia.foo-projects.org/remotes. You can use this to check if your favorite mods are already included or not.

PS clients will need to have been built with LibCurl and have
Code: Select all
enable_remote_media_server = true
set in the settings for this protocol to be used.
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Hosted Mod List

by sofar » Wed Dec 06, 2017 22:48

This list is descriptive - it would be too long to describe all the mods by name. If you spot a mod that you think is missing, please ask, and we'll make sure it's included.

Hosted mods:

1. Minetest Game (minetest_game).
2. All minetest-mods team mods.
3. Dreambuilder modpack.
4. 3d_armor
5. playereffects
6. pep
7. hardenedclay
8. nether
9. nssb+nssm
10. mobs_redo|animal|horse|npc|monster
11. unified_inventory
12. multiskin

More may be added. Read the thread below to check or ask. If you want to suggest a mod, please link the github project page or forum page.
 

User avatar
Lejo
Member
 
Posts: 533
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project (BETA)

by sofar » Thu Dec 07, 2017 16:58

Lejo wrote:I use it for my server: 217.160.13.85
Could you add these mods?


Added! Enjoy, don't forget to change your server configs and restart!
 

User avatar
Lejo
Member
 
Posts: 533
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo
 

TommyTreasure
Member
 
Posts: 21
Joined: Tue Nov 15, 2016 01:37
IRC: TommyTreasure
In-game: TommyTreasure

Re: Public Remote Media Server Project (BETA)

by TommyTreasure » Sat Dec 09, 2017 23:35

I'm not sure my server will qualify for your offer.

1. Its not going to be on the announce list
2. Players will have access only by invitation or referral.
3. The mods I would be interested for remote media are quite large, and not used very much on production servers.

I'm running on a port forwarded system, and even if I don't qualify, I think this is a fantastic offer to all admins.

One of reasons I'm going to be limiting my player base, is because of the extremely large media size of NSSM and NSSB.
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project (BETA)

by sofar » Sun Dec 10, 2017 04:00

TommyTreasure wrote:I'm not sure my server will qualify for your offer.

1. Its not going to be on the announce list
2. Players will have access only by invitation or referral.
3. The mods I would be interested for remote media are quite large, and not used very much on production servers.

I'm running on a port forwarded system, and even if I don't qualify, I think this is a fantastic offer to all admins.

One of reasons I'm going to be limiting my player base, is because of the extremely large media size of NSSM and NSSB.


You are still welcome to use the remote media server, all the same. Even if you do not ask me to include more mods, your players will still benefit from the remote media server. Additionally, you can always *ask* me to include specific mods to help your players and your servers, even if the server is private or invite only. Note: The top post explains what parts are optional.
 

User avatar
Lejo
Member
 
Posts: 533
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project (BETA)

by sofar » Sun Dec 10, 2017 20:04

Lejo wrote:I found some more mods in my worldmods folder.
Could you add these:


Added both! Cheers.
 

TommyTreasure
Member
 
Posts: 21
Joined: Tue Nov 15, 2016 01:37
IRC: TommyTreasure
In-game: TommyTreasure

Re: Public Remote Media Server Project (BETA)

by TommyTreasure » Mon Dec 11, 2017 07:16

sofar, I think your efforts on this will be great for those of us with port forwarded servers! Thanks.

I've added the lines in my minetest.conf, but how do I verify that its working correctly? I don't see anything new in my debug.txt on server start.

The 2 mods that are impacting my systems the hardest are the NSSM and NSSB.

https://github.com/NPXcoot/nssm.git
https://github.com/NPXcoot/nssb.git
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project (BETA)

by sofar » Mon Dec 11, 2017 07:38

TommyTreasure wrote:sofar, I think your efforts on this will be great for those of us with port forwarded servers! Thanks.

I've added the lines in my minetest.conf, but how do I verify that its working correctly? I don't see anything new in my debug.txt on server start.


You won't see anything on your server. Even the client won't see anythng in the logs, however, you should be able to see it on a firewall that has monitoring or proxy services enabled. Only the remote media server gets to see the clients connect in an easy way and fetch the media files.

TommyTreasure wrote:The 2 mods that are impacting my systems the hardest are the NSSM and NSSB.

https://github.com/NPXcoot/nssm.git
https://github.com/NPXcoot/nssb.git


Added! Enjoy!
 

User avatar
BBmine
Member
 
Posts: 3474
Joined: Sun Jul 12, 2015 22:51
Location: USA
GitHub: BBmine
IRC: BBmine
In-game: Baggins

Re: Public Remote Media Server Project (BETA)

by BBmine » Fri Dec 15, 2017 14:56

Wouldn't it be better for the server to upload media to the PRMS? And then the clients get it from the PRMS. Many admins edit textures and make their own textures, so using links to the official mods would be a pain. I know you can upload your changes to your own github repo, but seriously, who wants to always push changes for every texture/sound change?
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project (BETA)

by sofar » Fri Dec 15, 2017 18:53

BBmine wrote:Wouldn't it be better for the server to upload media to the PRMS? And then the clients get it from the PRMS. Many admins edit textures and make their own textures, so using links to the official mods would be a pain. I know you can upload your changes to your own github repo, but seriously, who wants to always push changes for every texture/sound change?


Yes and no.

There is no problem with missing textures - the system is entirely failsafe, and even if the Remote Media server is offline, clients won't have any issues with missing bits and will just fetch those few images from the server directly, without any interruption.

The goal isn't to provide 100% media coverage, but a significantly large amount. There will always be custom textures that the server operator changes too often, and there's no reason to be all crazy about getting 100% of the media from the remote public media server. If server owners really want that, they can, and should, run their own remote media server (like I do for Inside the Box, for instance).

Uploading would require me to grant privileges to server operators, and this is simply not acceptable in today's security environment. The risks of abuse are far too great.

An alternative to uploading would be to download from the servers to the remote media server, however this would impose a significant burden on servers as now they are going to get frequent login requests that may cause other issues, again not a very sensible and nice thing.

If server operators use mod XX and have minor modifications to only a few textures in said mod, then they can still use the remote media server and get a huge benefit from it, and it's simpler for everyone if we just let the redundancy in the system take care of those special few textures. They just have to ask me to include the upstream mod XX to get the most benefit.

To comment on the efficiency of this model, I can share some really early statistics (these are very early statistics, we're barely running for a week now with very few servers pointing to the PRMS): over 70% of requested media was served by the remote media server. With more mods in the collection, this number will only slowly increase. Most clients get 80%+ of the media files from the PRMS. Repeat connections of clients obviously will often get misses, but this is at no cost to the client.
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project

by sofar » Wed Dec 20, 2017 07:08

I'm taking the "beta" tag off this post. The server is running stable and serving media requests reliably and as far as I can see, smoothly. Please use this media server as much as you can for every server you have!
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project

by sofar » Wed Dec 20, 2017 07:17

Some quick statistics: only very few servers are actually using this service. We're getting somewhere between 5-15 connections a day. The hit rate is well over 70%, and clients are getting sometimes nearly 1000 media files from this service. On average, the service is only pushing out somewhere between 20-30mb per day. It's pretty much entirely idle.
 

User avatar
Lejo
Member
 
Posts: 533
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo

Re: Public Remote Media Server Project

by Lejo » Thu Dec 21, 2017 12:47

sofar wrote:Please use this media server as much as you can for every server you have!

May you should change the title to something, which everyone can understand.
If you here Public Remote Media Server Project, you don't think it is sth. to decrease your network using.
 

TommyTreasure
Member
 
Posts: 21
Joined: Tue Nov 15, 2016 01:37
IRC: TommyTreasure
In-game: TommyTreasure

Re: Public Remote Media Server Project

by TommyTreasure » Fri Dec 22, 2017 02:20

sofar, So far (pun intended) this seems to be working well for me. Except for my local network client. I had to edit my hosts file to connect to my own media.

Thank you for the effort. I think this will be great for the smaller, and especially the home based port forwarded servers.
 

User avatar
Lejo
Member
 
Posts: 533
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo

Re: Public Remote Media Server Project

by Lejo » Fri Dec 22, 2017 09:33

TommyTreasure wrote:I think this will be great for the smaller, and especially the home based port forwarded servers.

It is for all servers great, becouse big servers can also ask to add some more mods.
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project

by sofar » Fri Dec 22, 2017 19:14

Lejo wrote:It is for all servers great, becouse big servers can also ask to add some more mods.


Even people running small servers can ask for certain mods to be included, so, this will benefit everyone who runs a server, large or small.
 

twahm
Member
 
Posts: 14
Joined: Tue Sep 12, 2017 18:11
GitHub: twahm
IRC: twahm
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project

by sofar » Sat Dec 23, 2017 06:29

twahm wrote:What is this?


The topic (OP) describes quite well what this is. Please read it again. Is there a more specific thing that is unclear? If you ask a more specific question, then maybe I can answer it.
 

User avatar
Aerozoic
Member
 
Posts: 46
Joined: Mon Jul 24, 2017 15:04
Location: USA
GitHub: aerozoic
IRC: aerozoic
In-game: aerozoic

Re: Public Remote Media Server Project

by Aerozoic » Thu Dec 28, 2017 17:16

How does it work for servers with modified textures? It automatically knows to get the modified texture from the server instead?
“For once you have tasted flight, you will forever walk the earth with your eyes turned skyward, for there you have been, and there you will always long to return.”
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Public Remote Media Server Project

by sofar » Thu Dec 28, 2017 22:41

Aerozoic wrote:How does it work for servers with modified textures? It automatically knows to get the modified texture from the server instead?


Correct. The client will ask the remote media server for everything that the client needs. The remote media server then tells the client "I have these, but not those". The client then fetches "these" from the remote media server HTTP location, and "those" from the minetest server directly.
 

User avatar
Aerozoic
Member
 
Posts: 46
Joined: Mon Jul 24, 2017 15:04
Location: USA
GitHub: aerozoic
IRC: aerozoic
In-game: aerozoic

Re: Public Remote Media Server Project

by Aerozoic » Sat Dec 30, 2017 13:55

Could you add all of tenplus1's mobs mods:

https://github.com/tenplus1/mobs_redo <-- Mobs Redo API
https://github.com/tenplus1/mobs_animal <-- Farm Animals
https://github.com/tenplus1/mobs_monster <-- Monsters
https://github.com/tenplus1/mobs_npc <-- Simple NPC and Trader
https://github.com/tenplus1/mob_horse <-- Ridable Horses with horseshoe upgrades

unified_inventory: https://github.com/minetest-technic/unified_inventory

and multiskin: https://github.com/stujones11/multiskin

Thanks
“For once you have tasted flight, you will forever walk the earth with your eyes turned skyward, for there you have been, and there you will always long to return.”
 

sofar
Developer
 
Posts: 2001
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar
 

Next

Return to Minetest-related projects



Who is online

Users browsing this forum: No registered users and 1 guest