Public Remote Media Server Project
- lag01
- Member
- Posts: 321
- Joined: Sun Mar 16, 2014 03:41
- GitHub: AndrejIT
- IRC: lag01
- In-game: lag
- Contact:
Re: Public Remote Media Server Project
As my server works from WiFi now, i added this service to configuration. I wonder if media can be fetched from multiple server simultaneously by client...
Re: Public Remote Media Server Project
It could, or, you can deploy the media server on a CDN, and automatically do this on the server side. Not that it matters much at this point, since the throughput needed to host the PRMS is trivial, and the client download rate is the limiting factor.lag01 wrote:As my server works from WiFi now, i added this service to configuration. I wonder if media can be fetched from multiple server simultaneously by client...
Re: Public Remote Media Server Project
Quick stats on the server:
last full week: 11gb total, 3k unique vsits, 1.5 million files requested.
Please keep suggesting mods, especially if they have moved.
last full week: 11gb total, 3k unique vsits, 1.5 million files requested.
Please keep suggesting mods, especially if they have moved.
- texmex
- Member
- Posts: 1753
- Joined: Mon Jul 11, 2016 21:08
- GitHub: tacotexmex
- In-game: tacotexmex
Re: Public Remote Media Server Project
texmex wrote:I suggest you add the bgm mod since it bandwidth heavy!
https://gitlab.com/timnxx20/bgm
Sorry, I won't add this, especially since it's both large and NC licensed. Servers that force players to download these media files should have to bear the burden themselves to serve these to their players.The included tracks are taken from the album "Resolute" by "Blue Dot Sessions" and are released under a CC BY-NC 4.0 license.
Re: Public Remote Media Server Project
I've updated the topic to include a little blurb that states that mods should be freely distributable, as well as a quick statement that we can refuse mods for various reasons. Most of the time it's as simple as "this mod already contains all these assets" or "this is an obsolete mod" but sometimes it can be content or license related. For instance, I'll likely not add things like nudity, etc.
- BuckarooBanzay
- Member
- Posts: 437
- Joined: Tue Apr 24, 2018 05:58
- GitHub: BuckarooBanzay
- IRC: BuckarooBanzai
- In-game: BuckarooBanzai
Re: Public Remote Media Server Project
Can you add the following mods/packs:
- display_modpack viewtopic.php?f=11&t=19365
- Botic font for display_modpack: https://github.com/pyrollo/font_botic
Re: Public Remote Media Server Project
Contains NC material. Maybe replace it with a font that's not NC?BuckarooBanzay wrote:Can you add the following mods/packs:
- display_modpack viewtopic.php?f=11&t=19365
The font is SIL, so I've added that (it's also tiny, really).BuckarooBanzay wrote:[*] Botic font for display_modpack: https://github.com/pyrollo/font_botic[/list]
)
- rubenwardy
- Moderator
- Posts: 6978
- Joined: Tue Jun 12, 2012 18:11
- GitHub: rubenwardy
- IRC: rubenwardy
- In-game: rubenwardy
- Location: Bristol, United Kingdom
- Contact:
Re: Public Remote Media Server Project
Hi, please may you add capturetheflag and ctf_pvp_engine?
https://github.com/MT-CTF/capturetheflag
https://github.com/MT-CTF/ctf_pvp_engine
Also, any idea why normal MTG resources aren't being picked up? I get a lot of messages like:
2018-12-31 17:00:06: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/d ... esword.png not found (HTTP response code said error) (response code 404)
2018-12-31 17:00:27: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/wool_pink.png not found (HTTP response code said error) (response code 404)
https://github.com/MT-CTF/capturetheflag
https://github.com/MT-CTF/ctf_pvp_engine
Also, any idea why normal MTG resources aren't being picked up? I get a lot of messages like:
2018-12-31 17:00:06: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/d ... esword.png not found (HTTP response code said error) (response code 404)
2018-12-31 17:00:27: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/wool_pink.png not found (HTTP response code said error) (response code 404)
Re: Public Remote Media Server Project
Done, added.rubenwardy wrote:Hi, please may you add capturetheflag and ctf_pvp_engine?
https://github.com/MT-CTF/capturetheflag
https://github.com/MT-CTF/ctf_pvp_engine
That's a bug in the client? What client version are you seeing this in?rubenwardy wrote: Also, any idea why normal MTG resources aren't being picked up? I get a lot of messages like:
2018-12-31 17:00:06: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/d ... esword.png not found (HTTP response code said error) (response code 404)
2018-12-31 17:00:27: ERROR[CurlFetch]: http://minetestmedia.foo-projects.org/wool_pink.png not found (HTTP response code said error) (response code 404)
The client should never ask for "wool_ping.png" but instead ask for "<prms_url>/<sha1ofresource>". But wait, it should ONLY ask for those URL's if the PRMS even says that "I have that sha1sum".
Somehow there are clients out there that (1) completely barf the remote media protocol and then (2) ask for the plain file instead of the sha1 from the media server.
I've seen this on the server side, but I don't get this on my own clients and I've never been able to reproduce it, because it is specific to some clients somehow. I've wanted to figure out why, but all I can see from the protocol implementation is that my PRMS side is correct and it had those files properly to begin with.
FYI, the PRMS has these resources and will share them by sha1sum properly:
Code: Select all
$ find . -name wool_pink.png | xargs sha1sum
c431049287c68a56bf08f933ccb3b59e0bf3c9ee ./MineClone2/mods/ITEMS/mcl_wool/textures/wool_pink.png
96d5fa1509960522cf20bbef370e31775d5deed4 ./minetest_game/mods/wool/textures/wool_pink.png
c71c9c8a2178c7945e42656e312b69324ac33732 ./capturetheflag/mods/shooter/textures/wool_pink.png
96d5fa1509960522cf20bbef370e31775d5deed4 ./capturetheflag/mods/wool/textures/wool_pink.png
$ find . -name default_tool_bronzesword.png | xargs sha1sum
476d7d4497c2d5f7f42a3aceefa6511f934e5f26 ./minetest_game/mods/default/textures/default_tool_bronzesword.png
476d7d4497c2d5f7f42a3aceefa6511f934e5f26 ./capturetheflag/mods/default/textures/default_tool_bronzesword.png
...
$ curl http://minetestmedia.foo-projects.org/c431049287c68a56bf08f933ccb3b59e0bf3c9ee -O
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 203 100 203 0 0 50750 0 --:--:-- --:--:-- --:--:-- 50750
$ file c431049287c68a56bf08f933ccb3b59e0bf3c9ee
c431049287c68a56bf08f933ccb3b59e0bf3c9ee: PNG image data, 16 x 16, 4-bit colormap, non-interlaced
$ sha1sum c431049287c68a56bf08f933ccb3b59e0bf3c9ee
c431049287c68a56bf08f933ccb3b59e0bf3c9ee c431049287c68a56bf08f933ccb3b59e0bf3c9ee
Re: Public Remote Media Server Project
And what about only providing the raw file and not the sha1sum?sofar wrote: That's a bug in the client? What client version are you seeing this in?
The client should never ask for "wool_ping.png" but instead ask for "<prms_url>/<sha1ofresource>". But wait, it should ONLY ask for those URL's if the PRMS even says that "I have that sha1sum".
I‘m using this way and it worked for my client.
Re: Public Remote Media Server Project
That would break if mod A and mod B both wanted to provide different versions of the same file name. Which happens, a lot.Lejo wrote: And what about only providing the raw file and not the sha1sum?
I‘m using this way and it worked for my client.
The above method is broken by design. The PRMS has many files that are duplicate filenames but a different sha1sum. Only this way will clients be sure to get the actual content they need, and not something randomly picked.
Re: Public Remote Media Server Project
[EDIT] this experiment is cancelled now [/EDIT]
I'm running an experiment on the PRMS - the PRMS will be sending a custom error image to clients that ask for the wrong files and do not honor the remote media protocol. The image contains the text "game client is out of date. Get it without ads for free: minetest.net".
It's possible that newer clients might see this texture somehow, but I can't figure out from the code why. If you see this texture appear in 0.4.17 versions of minetest, please send me a screenshot and the server address so I can try to reproduce the issue.
I'm running an experiment on the PRMS - the PRMS will be sending a custom error image to clients that ask for the wrong files and do not honor the remote media protocol. The image contains the text "game client is out of date. Get it without ads for free: minetest.net".
It's possible that newer clients might see this texture somehow, but I can't figure out from the code why. If you see this texture appear in 0.4.17 versions of minetest, please send me a screenshot and the server address so I can try to reproduce the issue.
- Linuxdirk
- Member
- Posts: 3219
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: Public Remote Media Server Project
So you intercept and modify the payload of your "customers"? The idea behind this is actually pretty great and hopefully will help in fighting malicious clients but two wrongs don't make a right.sofar wrote:I'm running an experiment on the PRMS - the PRMS will be sending a custom error image to clients that ask for the wrong files and do not honor the remote media protocol. The image contains the text "game client is out of date. Get it without ads for free: minetest.net".
Re: Public Remote Media Server Project
These clients are not compatible with the remote media protocol (largely, 0.4.11 and 0.4.13) and they should have not been built with curl support. They ask for resources by file name which means that if your remote media server has 25 copies of default_grass.png, the client can't ask for a specific one anyway. This is why the remote media protocol works by content sha1sums, and it should only ask for the sha1sums that the MT server told it to get. Note that throughout all this, the client can just discard the placeholder texture that the PRMS sends if a client does this, because the MT server told it that it needed sha1sum X, and the PRMS sends it the placeholder texture which does not have sha1sum X, but some other value.Linuxdirk wrote:So you intercept and modify the payload of your "customers"? The idea behind this is actually pretty great and hopefully will help in fighting malicious clients but two wrongs don't make a right.sofar wrote:I'm running an experiment on the PRMS - the PRMS will be sending a custom error image to clients that ask for the wrong files and do not honor the remote media protocol. The image contains the text "game client is out of date. Get it without ads for free: minetest.net".
So it's not as black and white as "a double wrong". These clients allow themselves to be passed malicious content by a MITM, effectively.
Anyway, it's an experiment, and I'm going to make sure this isn't a bigger nuisance or causes other problems. If you spot issues, please let me know ASAP.
Re: Public Remote Media Server Project
Well, after compiling 0.4.13 and attempting to see if the placeholder image actually shows up in-game, I've stopped the experiment. The 0.4.13 MT client code does download the placeholder, but then properly checks the checksum and does not use the placeholder image in-game, and so it's more efficient for the PRMS to return a 404 instead of a placeholder image.
I'm not evil enough to generate sha1 collisions on the fly :).
I'm not evil enough to generate sha1 collisions on the fly :).
- Linuxdirk
- Member
- Posts: 3219
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: Public Remote Media Server Project
Now it makes absolute sense to me.sofar wrote:These clients are not compatible with the remote media protocol (largely, 0.4.11 and 0.4.13) and they should have not been built with curl support. They ask for resources by file name which means that if your remote media server has 25 copies of default_grass.png, the client can't ask for a specific one anyway. This is why the remote media protocol works by content sha1sums, and it should only ask for the sha1sums that the MT server told it to get. Note that throughout all this, the client can just discard the placeholder texture that the PRMS sends if a client does this, because the MT server told it that it needed sha1sum X, and the PRMS sends it the placeholder texture which does not have sha1sum X, but some other value.
Re: Public Remote Media Server Project
The following mods were *removed* due to no longer being available for updates:
git://github.com/MT-Modder/hydroponics
git://github.com/bell07/minetest-skinsdb
git://github.com/tenplus1/mob_horse
git://github.com/tenplus1/mobs_animal
git://github.com/tenplus1/mobs_monster
git://github.com/tenplus1/mobs_npc
git://github.com/tenplus1/farming.git
If people want these still hosted at the PRMS, send me the proper git locations, and I can add them back.
git://github.com/MT-Modder/hydroponics
git://github.com/bell07/minetest-skinsdb
git://github.com/tenplus1/mob_horse
git://github.com/tenplus1/mobs_animal
git://github.com/tenplus1/mobs_monster
git://github.com/tenplus1/mobs_npc
git://github.com/tenplus1/farming.git
If people want these still hosted at the PRMS, send me the proper git locations, and I can add them back.
-
- Member
- Posts: 115
- Joined: Sun Sep 06, 2015 12:23
- GitHub: Thomas--S
- IRC: Thomas-S
- In-game: Thomas-S
- Location: Bavaria, Germany
Re: Public Remote Media Server Project
As far as the tenplus1 mods are concerned, this should be the new git locations:
https://notabug.org/TenPlus1/mob_horse.git
https://notabug.org/TenPlus1/mobs_animal.git
https://notabug.org/TenPlus1/mobs_monster.git
https://notabug.org/TenPlus1/mobs_npc.git
https://notabug.org/TenPlus1/farming.git
https://notabug.org/TenPlus1/mob_horse.git
https://notabug.org/TenPlus1/mobs_animal.git
https://notabug.org/TenPlus1/mobs_monster.git
https://notabug.org/TenPlus1/mobs_npc.git
https://notabug.org/TenPlus1/farming.git
My mods: ts_furniture | ts_doors
Re: Public Remote Media Server Project
WHY does notabug not allow/provide git:// support? WHY?Thomas-S wrote:As far as the tenplus1 mods are concerned, this should be the new git locations:
https://notabug.org/TenPlus1/mob_horse.git
https://notabug.org/TenPlus1/mobs_animal.git
https://notabug.org/TenPlus1/mobs_monster.git
https://notabug.org/TenPlus1/mobs_npc.git
https://notabug.org/TenPlus1/farming.git
sigh, added. With grumbling.
Re: Public Remote Media Server Project
I have added this service to my five servers
add server running under KNOPPIX ( Debian ) with minetest-server 0.4.17.1
and the minetest-game 0.4.17.1 or the mineclone2 0.45.1 subgame
minetest can be found here: https://repo.or.cz/MineClone/MineClone2.git
the list of used 3rd person mods by servers:
P.S. sorry for my bad english, i am from german ^^
add server running under KNOPPIX ( Debian ) with minetest-server 0.4.17.1
and the minetest-game 0.4.17.1 or the mineclone2 0.45.1 subgame
minetest can be found here: https://repo.or.cz/MineClone/MineClone2.git
the list of used 3rd person mods by servers:
Spoiler
--- German-Mineclone2-Night-server ( Port 30000 )
gameid = MineClone2
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = false
load_mod_sethome = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_xban2 = true
--- German-MineClone2-server ( Port 30001 )
gameid = MineClone2
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = false
load_mod_sethome = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_xban2 = true
--- German-Vanilla-server ( Port 30002 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = true
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
--- German-Underground-server ( Port 30003 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = true
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
--- German-Trash-server ( Port 30004 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = false
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
gameid = MineClone2
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = false
load_mod_sethome = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_xban2 = true
--- German-MineClone2-server ( Port 30001 )
gameid = MineClone2
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = false
load_mod_sethome = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_xban2 = true
--- German-Vanilla-server ( Port 30002 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = true
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
--- German-Underground-server ( Port 30003 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = true
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
--- German-Trash-server ( Port 30004 )
gameid = minetest
backend = sqlite3
creative_mode = false
enable_gemage = true
player_backend = files
server_announce = true
load_mod_areas = true
load_mod_protector = true
load_mod_sethome = true
load_mod_gravelsieve = true
load_mod_bonemeal = true
load_mod_throwing = true
load_mod_3d_armor = true
load_mod_farming = true
load_mod_shields = true
load_mod_hazmat_suit = true
load_mod_mtg_i18n = true
load_mod_hud = true
load_mod_mobs = true
load_mod_technic_armor = true
load_mod_3d_armor_stand = true
load_mod_hunger = true
load_mod_anvil = false
load_mod_inventorybags = false
load_mod_intllib = true
load_mod_mobs_mc = true
load_mod_wieldview = true
load_mod_mobs_monster = true
load_mod_mobs_animal = true
load_mod_invisible = true
load_mod_invhack = true
load_mod_unified_inventory = true
load_mod_unified_inventory_plus = true
load_mod_xban2 = true
My YouTube-Site with Minetest-LetsPlays and some of my Servers: Carpathian Ethereal Vanilla Creative MineClone2 --- Discord
Re: Public Remote Media Server Project
I can't do anything with this data.Walker wrote: load_mod_areas = true
Please list only new mods that were not added before, and give me the git URL for them. If you say "load_mod_areas" then I have no idea what git URL that is, or whether you use a fork of it at a different URL or not.
For reference, a git url starts with "git://"
Re: Public Remote Media Server Project
Update:
You can get a list of currently included mods from this URL: http://media.foo-projects.org/remotes. You can use this to check if your favorite mods are already included or not. Or send corrections and updates.
You can get a list of currently included mods from this URL: http://media.foo-projects.org/remotes. You can use this to check if your favorite mods are already included or not. Or send corrections and updates.
Re: Public Remote Media Server Project
Hello. Thank you for the list; if you're interested I'm using this for git://github.com/jastevenson303/Glitchtest.git --
Particularly FaceDeer's dfcaverns modpack @ git://github.com/facedeer/dfcaverns.git (LICENSE.txt)
Thank you for your consideration
Particularly FaceDeer's dfcaverns modpack @ git://github.com/facedeer/dfcaverns.git (LICENSE.txt)
Thank you for your consideration
Re: Public Remote Media Server Project
Added both. They'll be included in a few hours as usual.jas wrote:Hello. Thank you for the list; if you're interested I'm using this for git://github.com/jastevenson303/Glitchtest.git --
Particularly FaceDeer's dfcaverns modpack @ git://github.com/facedeer/dfcaverns.git (LICENSE.txt)
Who is online
Users browsing this forum: No registered users and 5 guests