[Mod] Exchange stuff between servers [Interkom]

[Gundul]

! CAUTION !

This mod is wip and in development.
Use at your own risk.
No garantees, no support, I am not responsible for any loss. :D

Dependencies: none
License:

+ Spoiler

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004

Copyright (C) 2004 Sam Hocevar
14 rue de Plaisance, 75014 Paris, France
Everyone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

0. You just DO WHAT THE FUCK YOU WANT TO.

This mod works, proofed on the following connected servers:
"Jungle"
"Lilly in the valley"
"LouisZ" (not in serverlist)

You need to setup a synchronized folder if you want to connect servers.
By default this folder must be in your world directory named "Lilly"
for example ~/.minetest/worlds/myworld/Lilly

With this mod it is possible to transfer stuff from one server to an other. It also works
on one server only, but /give command is a bit faster :)
All can be send, except tools or nodes containing metadata. You are free to use what ever
mods you want on each server because interkom will know what is known and which nodes
not. Unknown things will return to sender.


Sorry no pictures as this mod only works with chatcommands so far:

/interkom
This will show you a chart of all connected servers with its players

/pm player,server,message
Will send a private message to player@server

/stuff player,server,stuff
Will send "stuff" to player@server, where stuff is an itemstack like "default:stone 25"

/ok
Will accept what a player wants to send you and puts him on the whitelist

/bl player@server
puts player@server on the blacklist again

So far it works on above servers, I will add other features in the future and maybe a
gui too. Any suggestions post here please.

download here: https://github.com/berengma/interkom/archive/master.zip
Sourcecode here: https://github.com/berengma/interkom


Gundul

[sofar]

This code looks super iffy. A fun read to find input validation issues so people can send arbitrary stuff they don't have to players on other servers. Are you sure you're checking all the input correctly? Did you protect against races and properly lock the transfer files?

You should post the dependencies and license in the forum thread. See sticky topic.

[Gundul]

This code looks super iffy.

just for the record:

-I do not post something "iffy" in mods section, this works for months without iussues on my two public servers
-We are here in wip mods section.Wip means: work in progress.
Do not expect something here what would not needed to be worked on.

A fun read to find input validation issues so people can send arbitrary stuff they don't have to players on other servers. Are you sure you're checking all the input correctly? Did you protect against races and properly lock the transfer files?

Exaggerated a bit ?
But I am always happy other people have fun with my developments. Any proove, example maybe ?
I am repeating myself, this mod is running for months on my servers without trouble.

[sofar]

This code looks super iffy.

just for the record:

-I do not post something "iffy" in mods section, this works for months without iussues on my two public servers
-We are here in wip mods section.Wip means: work in progress.
Do not expect something here what would not needed to be worked on.

A fun read to find input validation issues so people can send arbitrary stuff they don't have to players on other servers. Are you sure you're checking all the input correctly? Did you protect against races and properly lock the transfer files?

Exaggerated a bit ?
But I am always happy other people have fun with my developments. Any proove, example maybe ?
I am repeating myself, this mod is running for months on my servers without trouble.

If I reply with concerns then it means (1) I generally like the code/mod/idea, and (2) I care enough to voice my concerns in case that I think it's warranted. It's not a personal attack, obviously, I never said "Oh gundul has the nerve to tickle a wombat!", instead I said "this code looks suspect (iffy)" with the intent of making you critically think about whether this is actually secure or not.

It obviously works, I don't doubt it. What I doubt is that if players send requests at the same time that both actually arrive intact. Since the receiving server can't verify that the items are in the senders' inventory, it's not hard to think of a possible exploit to generate some message that gets the receiver lots of items that the sender never had, or just corrupt the messages. There's also some splitting of user input going on without escaping, so there's a possibility of people throwing weird characters at you that you didn't think of ("\n") and may end up in your transfer file.

It's not my code, if you don't want to secure it more, that's your perogative. But now that you've posted the code, if there is an exploit possible, anyone can find it.

[Krock]

I wondered how "iffy" the code actually is and only figured out some minor issues:

1) Unreliable file writing. I'd rather use `minetest.safe_write_file` to ensure everything is written correctly in case the server is not shut down correctly.
2) Writing to the file at the same time: If server 1 writes to the file whereas server 2 deletes an entry from the file, there may be a file corruption. Not sure though, how this could be solved.
3) If you ever allow metadata: New lines and commas would break the system. There's no escape handling in your custom string.split implementation and new lines would destroy the format. Luckily you treat items with metadata like tools (disallowed), even though they could be stackable.

It was however interesting to see this mix of tabs and spaces. I think you're using four spaces, but the editor does not convert existing tabs when you indent the code.

[Gundul]

If I reply with concerns then it means (1) I generally like the code/mod/idea, and (2) I care enough to voice my concerns in case that I think it's warranted. It's not a personal attack, obviously, I never said "Oh gundul has the nerve to tickle a wombat!", instead I said "this code looks suspect (iffy)" with the intent of making you critically think about whether this is actually secure or not.

Sorry for getting you wrong, engish is not my native language and "iffy" is not the word we would use in a situation like this.

It obviously works, I don't doubt it. What I doubt is that if players send requests at the same time that both actually arrive intact. Since the receiving server can't verify that the items are in the senders' inventory, it's not hard to think of a possible exploit to generate some message that gets the receiver lots of items that the sender never had, or just corrupt the messages. There's also some splitting of user input going on without escaping, so there's a possibility of people throwing weird characters at you that you didn't think of ("\n") and may end up in your transfer file.

The security is always at my mind. But it is only about sharing some stuff between servers. Worst that could happen is someone getting lots of stuff he did not have. And I get the debug for free and can work with that. :)
I already updated some parts of the code checking more detailed for nodenames.

It's not my code, if you don't want to secure it more, that's your perogative. But now that you've posted the code, if there is an exploit possible, anyone can find it.

I will make it as secure as I am able to do, that is why I post it now and not already did when I first put the mod
online. That is also why I put the warning caution and do not describe more in detail how to synchronize folders.
Everybody must decide this for his own. Not to forget that exploits also can have positve effects when server and
running software are well monitored.

[Gundul]

I wondered how "iffy" the code actually is and only figured out some minor issues:

1) Unreliable file writing. I'd rather use `minetest.safe_write_file` to ensure everything is written correctly in case the server is not shut down correctly.

Thank you, I did not know that exists in the minetest api.

2) Writing to the file at the same time: If server 1 writes to the file whereas server 2 deletes an entry from the file, there may be a file corruption. Not sure though, how this could be solved.

I had this in my mind. But I must confess I have no solution for this, too. I was thinking that the software which synchs the folders would take care of this maybe.

3) If you ever allow metadata: New lines and commas would break the system. There's no escape handling in your custom string.split implementation and new lines would destroy the format. Luckily you treat items with metadata like tools (disallowed), even though they could be stackable.

I do not plan to allow metadata. This is not wanted now and in the near future.
Maybe all this can be done with a gui in the future, but only if this really runs reliable.

Items with metadata could be stackable, yes. But in that case the metadata would be lost if sent to an other server.

It was however interesting to see this mix of tabs and spaces. I think you're using four spaces, but the editor does not convert existing tabs when you indent the code.

Sorry, which tabs and spaces you mean ?

[Krock]

Items with metadata could be stackable, yes. But in that case the metadata would be lost if sent to an other server.

Ah okay. I didn't look at the code for a too long time to notice that behaviour.

Sorry, which tabs and spaces you mean ?

These particular lines are interesting: https://github.com/berengma/interkom/bl ... #L233-L239
space-only indents: 3/7 lines
1x tab, spaces: 1/7 lines
tab-only indents: 3/7 lines

Not important in terms of functionality but looks a bit as if you copies&pasted multiple mods into this one.

[sofar]

My initial thoughts on this mod are to secure the wire protocol and not use files as they are obviously prone to locking issues (as lua does none) and 2 servers could write the same file at the same time. A better implementation would be to use sockets to do all the communication, where each server has a "receive" socket, and the sender transmits by sending some json encoded structural content.

The chatcommands and parsing of input are difficult things to get right (does it handle utf8 properly?) so I would think about replacing the chatcommands with a properly secured formspec where players need to put the actual items in a send slot, which also takes the trouble of knowing the item names away for players. You could use the sockets to send player lists to each server so that players know who is online or not and have the form display this, or maintain a friend list or something like that.

[Lone_Wolf]

2) Writing to the file at the same time: If server 1 writes to the file whereas server 2 deletes an entry from the file, there may be a file corruption. Not sure though, how this could be solved.

I had this in my mind. But I must confess I have no solution for this, too. I was thinking that the software which synchs the folders would take care of this maybe.

I'm not sure of how everything works in your system but you might be able to create a 'status file' for each server in the folder and set it up so that before a server writes anything it checks the other server's status files.
If none of the other servers is writing have it set the contents of its status file to something that says it's writing to a file.
Make it write to the file it was originally going to write to, and when it finishes you can make it rewrite it's status file to something saying it's not writing

I hope that's readable...

[sofar]

I'm not sure of how everything works in your system but you might be able to create a 'status file' for each server in the folder and set it up so that before a server writes anything it checks the other server's status files.
If none of the other servers is writing have it set the contents of its status file to something that says it's writing to a file.
Make it write to the file it was originally going to write to, and when it finishes you can make it rewrite it's status file to something saying it's not writing

I hope that's readable...

Now you just created a second locking problem with the state file. :)

[Gundul]

Make it write to the file it was originally going to write to, and when it finishes you can make it rewrite it's status file to something saying it's not writing

I hope that's readable...

Thank you it was readable :)
But I think sofar already answered this. :)

[Gundul]

My initial thoughts on this mod are to secure the wire protocol and not use files as they are obviously prone to locking issues (as lua does none) and 2 servers could write the same file at the same time. A better implementation would be to use sockets to do all the communication, where each server has a "receive" socket, and the sender transmits by sending some json encoded structural content.

The chatcommands and parsing of input are difficult things to get right (does it handle utf8 properly?) so I would think about replacing the chatcommands with a properly secured formspec where players need to put the actual items in a send slot, which also takes the trouble of knowing the item names away for players. You could use the sockets to send player lists to each server so that players know who is online or not and have the form display this, or maintain a friend list or something like that.

You are right with the formspecs. I chose the chatcommands because experience from my servers says
players are lazy in writing. That way I could test my mod online on my servers more relaxed in the beginning.

I must confess I have very little knowledge about sockets, but that can change. Always happy to learn something.
A quick search returned a "lua socket api" for lua 5.1. No idea if that could help. If someone knows any good source
or book please feel free to post/tell me.

The next steps for interkom will be:

- use minetest.safe_write_file if possible
- restrict use of chatcommands to players with interkom(server ?) priv
- add formspecs to handle items and send them to connected servers
- find a way to use sockets

These steps do not have to be static. I am happy for any serious idea someone could have.

[Gundul]

created a new branch using minetest.safe_file_write
https://github.com/berengma/interkom/tree/savewrite