[discussion]Technical notes on how to defeat(*) oredetect

User avatar
paramat
Developer
Posts: 3700
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat
IRC: paramat
Location: UK

Re: [discussion]Technical notes on how to defeat(*) oredetec

by paramat » Post

Because the vast majority of MT users cannot break styrofoam.

CSM flavours were added to 0.5.0dev (as it is called now) as a way to restrict CSM.
Note that old clients can bypass flavour restrictions, so these will only be enforceable in 0.5.0 where all clients must be 0.5.0 also. Another good reason to update to 0.5.0 as soon as possible.

sofar
Developer
Posts: 2146
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [discussion]Technical notes on how to defeat(*) oredetec

by sofar » Post

Linuxdirk wrote:
sofar wrote:And all of the above options can be defeated trivially by anyone with a compiler.
We all know that.
The options referred to that I commented about were NOT previously discussed in this thread, and therefore I find it highly appropriate to state this for THE FIRST TIME in the discussion.

User avatar
duane
Member
Posts: 1715
Joined: Wed Aug 19, 2015 19:11
GitHub: duane-r
Location: Oklahoma City
Contact:

Re: [discussion]Technical notes on how to defeat(*) oredetec

by duane » Post

paramat wrote:Because the vast majority of MT users cannot break styrofoam.
While that's certainly true, the majority of minetest users aren't interested in cheating, and it only takes one cheater on a given server to ruin the experience for everyone.

Minetest just isn't suitable for most competitive games -- by design.

Your only real options are to either remove the competitive element entirely or to base the competition on things that cannot be cheated (see Inside the Box). It's not practical to change the data that's sent to the client, and as long as it's possible to recompile the client, you will get a cheater eventually. And once someone sees someone else doing it, they're motivated to learn how to do it themselves.

Of course, motivating someone to learn to code is a worthwhile effort in itself, so... there's your silver lining.
Believe in people and you don't need to believe anything else.

User avatar
rubenwardy
Moderator
Posts: 6969
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: [discussion]Technical notes on how to defeat(*) oredetec

by rubenwardy » Post

duane wrote:
Linuxdirk wrote:
sofar wrote:And all of the above options can be defeated trivially by anyone with a compiler.
We all know that. You don't need to justify not having any CSM security by repeating this over and over again.
What's the point of easily defeated security? Would you use a padlock made of styrofoam?
Wish I could like posts.


All of the above suggestions are redundant as the server can disable reading nodes using CSM limits. To improve information security, the server needs to not send any sensitive information to the client.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
AiTechEye
Member
Posts: 1000
Joined: Fri May 29, 2015 21:14
GitHub: AiTechEye
Location: Sweden

Re: [discussion]Technical notes on how to defeat(*) oredetec

by AiTechEye » Post

non-transferable variable?

limit the function's distance to the players currently rage?

only find viewable nodes?

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Linuxdirk » Post

sofar wrote:The options referred to that I commented about were NOT previously discussed in this thread, and therefore I find it highly appropriate to state this for THE FIRST TIME in the discussion.
In a discussion about clean air you don’t need to mention that we need oxygen to survive. It is common sense for everyone discussing the topic.

sofar
Developer
Posts: 2146
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: [discussion]Technical notes on how to defeat(*) oredetec

by sofar » Post

I'm going to start deleting posts to this thread that do not honor the topic guidelines that ARE CLEARLY POSTED IN THE ORIGINAL POST.

If you can't stay on topic, please stop posting in here.
This is a discussion thread to discuss technical measures that can be taken by server operators to defeat CSM mods like oredetect, but without killing oredetect or adding "restrictions".

This is not a rant thread to discuss CSM. Discussions about CSM and other stuff are not on-topic and will be deleted.

The goal is to come up with elegant and creative solutions that are constructive, and not destructive.
Again, the goal isn't to avoid CSM discussions, just DO THEM IN A DIFFERENT THREAD. This thread is to discuss options that leave CSM enabled and fully functional, without restrictions, and without server side detection of clients using CSM.

As I've explained before in the thread, there are definitely options that would defeat oredetect, even though some moan loudly that they hate it, a fact is a fact, it would 100% work. Given that there is a workable solution already, I am looking for ideas that improve on those ideas.

If you are still thinking about restricting CSM at this point, and have read this far, you're not actually reading what I am writing.

User avatar
Wuzzy
Member
Posts: 4781
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Wuzzy » Post

I) remove specific ore nodes altogether and make e.g. plain stone randomly drop gold
This solution would completely defeat oredetect, but it's destructive.
This would be universally hated. This reduces mining to totally mindless and aimless grinding and would be quite boring. It's common sense.
We can do better than this.
II) have normal stone and ore stone, and ore stone randomly drops balanced distributed rare items so that players get a certain amount of iron, gold, diamonds etc.
Useless since oredetect just needs to detect the ore stone.
III) decouple digging ore from obtaining rare resources: e.g. in a mining union, the player gets paid for the amount of work done, so one could reward a player working in a public mine that has mined 1000 stone equally to someone who has mined 50 ore nodes.
This has the same problem as II) because oredetect just detects the ore nodes.

I also think ideas I-III are destructive because you lose the classic ore blocks and get replaced by something boring. They restrict a lot of creative freedom of the subgame author. If you really do want the classic ore nodes in your subgame, these ideas are not helping.



The problem with oredetect is pretty simple: Oredetect works because the Minetest client already knows where the ores are. The server happily gives you this valuable information. You just need some code to extract this information from YOUR client. This code does not even need to be CSM. A modified client works just fine.
The current ideas seem to completely ignore this, and I think they all fall flat. We need ideas based on this premise. That we should not give clients all crucial information on a silver platter.

I came up with a few ideas on my own.


Idea 1:
- Reduce the block send distance

Pros:
- Already possible and simple
- Reduces server load

Cons:
- Does not defeat ore detect at all, only limits it
- Reduces view distance
- Useless if oredetect takes the seed into account

Idea 2:
- Allow modders to add a “disguise” for certain nodes. This is a node attribute which sets another node as the disguise of itself. E.g. a gold ore can be disguised as stone
- This is NOT a texture change!
- Example: The Minetest server will pretend that gold ore is actually stone as long the node is enclosed by stone and/or is far away from the player.
- There are no changes to the map format, or the network protocol required. The server still knows perfectly well where the ores are, it just lies about the itemstring
- For the client, enclosed ores are internally stored as node Y, so oredetect is useless
- When any of the sides has been made visible (e.g. air), the server tells the client that the node at position P is actually a gold ore (or whatever)

Pro:
- Guaranteed to defeat oredetect. There's nothing to detect
- No gameplay changes required
- Completely transparent to the client

Con:
- Requires a round-trip when an ore is revealed
- After an ore is revealed, it will first briefly look like a normal stone. This messes with client predictions
- Useless if oredetect takes the seed into account

Idea 3:
- Only send in “visible” nodes. These are nodes NOT enclosed by fully opaque nodes
- Internally, the client stores hidden (enclosed by a solid block) nodes as a special node type “hidden”
- When the player digs out a block and other blocks get revealed, the server sends an update for the new, revealed blocks

Pros:
- Oredetect can only detect ores which are visible anyway
- No gameplay changes required

Cons:
- Does not defeat oredetect 100%. It can detect ores in a neighbor cave which the player doesn't see
- Complex
- Requires a round-trip for each dig
- There's a delay before you see the revealed node after a dig
- Useless if oredetect takes the seed into account


Idea 4:
- Same as idea 3, but send more nodes: Not only the visible nodes themselves, but also all the nodes directly behind them

Pros:
- Oredetect is made (mostly) useless
- No gameplay changes required
- The dig prediction is a bit better than in idea 3

Cons:
- Visible ores and ores hidden directly behind stone can be detected
- Complex
- Useless if oredetect takes the seed into account


All ideas so far are vulnerable to using the world seed and using it to (pre-)calculate the positions of ore spawns. This can be very powerful as you can find lots of ores long before the server even generated them.

I have 2 ideas to defeat this:
Idea A: Keep the world seed secret from clients
Idea B: Make ore spawns independent from the world seed


Idea 5:
- Generate only stone, but keep the ore nodes (as definitions)
- After digging stone, randomly decide to add an ore cluster in a patch of hidden stone close to the player. If the algorithm decided to not add stone, mark a patch of stone as “tried”
- The randomness and ore types are based on height. Existing ore definitions can be recycled.
- Stone needs a distinction between naturally generated and player-placed (like leaves)
- This algorithm is only allowed to add ores in naturally generated stone and stone which has not been marked as “tried”.

Pros:
- Pretty much destroys oredetect

Cons:
- Additional round-trips required (but you might be clever about it to hide it from the player)
- No ores at cave walls. You have to dig in order to reveal ores
- Hard to implement advanced ore distribution stuff. The actual ore distribution (as seen by players) will be very different

Note: This is similar to idea I, but without the frustration of having no visible ores.


Sadly, none of my ideas are without problems. But maybe you have an idea on how to improve on them.

AntiMS
New member
Posts: 2
Joined: Sat Nov 28, 2015 17:50
GitHub: AntiMS
In-game: Sawag

Re: [discussion]Technical notes on how to defeat(*) oredetec

by AntiMS » Post

Hi, folks. First time poster, long time Minetest player, here.

This topic is interesting to me and I wanted to add my own idea.

So, in this idea, the mapgen doesn't deal with "ores". The mapgen just makes stone.

The server has two new per-world configuration values in env_meta.txt named something like "ore_visibility_length" and "secret_key".

ore_visibility_length should default to some sane value on world creation. secret_key should be a securely generated random value also created on world creation.

Any time a stone node which is adjacent to air is sent to the client, a single "key" is sent to the client for the node. Any time a node which is not adjacent to air is interacted with (i.e. "dug"), 27 keys are generated and sent to the client (either in the TOCLIENT_BLOCKDATA message or in the the TOCLIENT_REMOVENODE message).

The following god-awful pseudocode illustrates a rough idea how to generate these 27 keys:

Code: Select all

function generate_single_key(xf, yf, zf, ore_visibility_length, secret_key)
	return string_concatenate(xf, ":", yf, ":", zf, ":",
		hash(
			string_concatenate(
				tostring(xf),
				":",
				tostring(yf),
				":",
				tostring(zf),
				":",
				tostring(secret_key)
			)
		)
	)
end_function

function keys_for_interacted_node(x, y, z, ore_visibility_length, secret_key)
	out = []
	xf = floor(x / ore_visibility_length)
	yf = floor(y / ore_visibility_length)
	zf = floor(z / ore_visibility_length)
	for xd in [-1, 0, 1]
		for yd in [-1, 0, 1]
			for zd in [-1, 0, 1]
				append(out, generate_single_key(xf+xd, yf+yd, zf+zd, ore_visibility_length, secret_key))
			end_for
		end_for
	end_for
	return out
end_function
Where "hash" is a one-way, cryptographic hash function like sha256 or some such. Crucially, it's important that the secret_key not be deriveable from the keys.

These 27 keys need not be kept by the server. The server can generate them again later. Though some caching server-side might improve performance. The client needs only keep them in memory (probably in a hash table keyed on the "xf", "yf", and "zf" values). Another possible performance improvement would be a simple mechanism for the client to let the server know which keys it's already got so as to allow the server not to re-send those keys the client already has.

(For this conversation, I'm assuming the keys are strings, but in practice, they should probably be binary values to save space and bandwidth. Each key could probably be around 512 bits or less depending on the hash function. 27 of those in a "TOCLIENT_REMOVENODE" is only about 1K of information added to the packet. If you used a value like 4 for ore_visibility_length, worst case it would add around 4K to the TOCLIENT_BLOCKDATA packet.)

The client and server must share a common function called "ore_type" which accepts parameters x, y, z (the coordinates of a node), map_seed (the map seed), and key. It will decide whether the indicated stone node is *actually* an ore node and if so what kind of ore node based only on those parameters. (Its behavior needn't be defined for non-stone nodes.)

Any time the client displays the face of a stone node to the user, it runs the ore_type function, passing it x, y, z, map_seed, and the key prefixed with xf=floor(node_x/ore_visibility_length), yf=floor(node_y/ore_visibility_length), and zf=floor(node_z/ore_visibility_length). Based on the result, it can display the block as the proper ore node or as plain old stone as need be.

When the user digs a node that the server thinks is stone, the server runs the ore_type function, passing it the required parameters, including the corresponding key. If the ore_type function indicates the node is an ore node, the server adds the corresponding ore to the player's inventory. If the ore_type function indicates the node is not an ore node, the server adds cobblestone to the player's inventory.

One last detail needs to be addressed. Suppose the player mines an ore node and then places a stone node in the same place. As stated thus far, if the player mines the newly-placed stone node, s/he'll receive another of the same ore as s/he did the first time the node was dug. (Basically, s/he'll be able to convert stone into diamonds 1-for-1 once s/he's found a single diamond node and smelted a bunch of stone.)

To address this, we might have to have two different kinds of stone nodes. One that the mapgen produces and one that is produced when the user smelts cobblestone.

So, pros and cons:

pro:
- No gameplay changes required
- Simplifies mapgen a bit
- Doesn't add much server load
- No round-trip required at time of displaying a potential ore node face
- No need for serve to maintain "two views" of the map
- Allows server admins some control over the performance/anti-oredetect tradeoff

con:
- Complex
- Difficult to imagine how this could be done in a backwards-compatible way
- Might require a lot of major refactor
- Does not defeat oredetect 100%

Anyway, I'm mostly just spitballing here. I'm not intimately familiar with the code, so there are some bits that I don't know how feasible they'd be to implement.

(One more thing... since the first I've heard of CSM "flavours" is here in this thread, I'll ask here... can you point me to any documentation about these "flavours" and how they work? I must say I'm a bit disappointed to hear hints that CSM may have been somewhat "nerfed" in any particular way, but I guess I can understand why it was done given the (misdirected IMO) backlash against CSM.)

(One more thing still... let me know if anything above is unclear or anything. I'll try to clarify as necessary.)

User avatar
Lone_Wolf
Member
Posts: 2575
Joined: Sun Apr 09, 2017 05:50
GitHub: LoneWolfHT
IRC: LandarVargan
In-game: LandarVargan

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Lone_Wolf » Post

Why not have stone pretend to be every valuable ore on the server?
My ContentDB -|- Working on CaptureTheFlag -|- Minetest Forums Dark Theme!! (You need it)

AntiMS
New member
Posts: 2
Joined: Sat Nov 28, 2015 17:50
GitHub: AntiMS
In-game: Sawag

Re: [discussion]Technical notes on how to defeat(*) oredetec

by AntiMS » Post

One more question.

Surely scriptable clients must be a problem for... that one other mining game. Any idea if/how the good folks at Microsoft have solved this problem? (And does their solution depend on the client and/or server being proprietary?)

Edit: Looks like the answer is "orebfuscator" (https://github.com/lishid/Orebfuscator). It's open-source and third-party to MineCr*ft. Just from glancing over the source code, it looks like it's basically what Wuzzy calls "idea 2".

Sokomine
Member
Posts: 4276
Joined: Sun Sep 09, 2012 17:31
GitHub: Sokomine
IRC: Sokomine
In-game: Sokomine

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Sokomine » Post

A server running that other game had occasional postings in its forum about players beeing banned for using "xray". Back then when the server had many players, the problem was solved by moderators taking a look and rules forbidding that "xray" function, counting it as cheating. It's possible that things have changed since then. MT in general seems to have a more relaxed attitude towards it. Even on servers where admins and moderators are not exactly enthusiastic about oredetect.
A list of my mods can be found here.

User avatar
LemonFox
Member
Posts: 25
Joined: Tue Jan 09, 2018 16:27
IRC: LemonFox
In-game: LemonFox

Re: [discussion]Technical notes on how to defeat(*) oredetec

by LemonFox » Post

Is oredetect avalible for download?
I cannot find it anywhere.
I am thinking of making a server and this sounds like a problem
Can i be supplied a link even if in a pm for saftey.
I may be able to help form a solution

Thanks

turducken
Member
Posts: 23
Joined: Fri Feb 02, 2018 01:43

Re: [discussion]Technical notes on how to defeat(*) oredetec

by turducken » Post

How about detection of using oredetect? Perhaps a server side check for number of ore nodes dug in a time-frame, or number of ore nodes versus stone nodes.

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Linuxdirk » Post

turducken wrote:How about detection of using oredetect?
Impossible. Nothing happens server-side.
turducken wrote:Perhaps a server side check for number of ore nodes dug in a time-frame
Banning people for farming cobblestone …
turducken wrote:Perhaps a server side check for number of ore nodes dug in a time-frame, or number of ore nodes versus stone nodes.
… or banning people for exploring caves instead of mining is not a solution. :)

turducken
Member
Posts: 23
Joined: Fri Feb 02, 2018 01:43

Re: [discussion]Technical notes on how to defeat(*) oredetec

by turducken » Post

Linuxdirk wrote:
turducken wrote:How about detection of using oredetect?
Impossible. Nothing happens server-side.
Sure, but the server is ultimate authority and responsible. It sends the map data to the client, which the client responds to with what nodes are to be removed. The client can do whatever it wants with the data, but ultimately the server has to accept any and all modifications the client wants to make to the world.
Linuxdirk wrote:
turducken wrote:Perhaps a server side check for number of ore nodes dug in a time-frame
Banning people for farming cobblestone …
No, detection of people with unusually high ore to cobblestone ratios. Perhaps over specific time-frames. Someone that is surrounded deep in a mine with stone but gets a high number of valuable ore nodes surrounded by rock (compared on average to other players) is suspicious. This could raise a flag, like the anti-cheat mod that randomly looks if players are within nodes using a no-clip mod.
Linuxdirk wrote:
turducken wrote:Perhaps a server side check for number of ore nodes dug in a time-frame, or number of ore nodes versus stone nodes.
… or banning people for exploring caves instead of mining is not a solution. :)
Or it's called brainstorming as the OP requested.

We can't block that the clients receive all the node data from the server. But, the server is told where the client is, and what nodes are mined. Somehow one should be able to detect that. Will there be false positives or cheating that is missed? Yes, but that comes down to admins and moderators to use their discretion.

There should be a metric that can be applied, as the entire basis of this cheat is that players get more valuable ore faster than "normal" players. Hence, work with that you DO know.

User avatar
xeranas
Member
Posts: 162
Joined: Fri Feb 05, 2016 11:06

Re: [discussion]Technical notes on how to defeat(*) oredetec

by xeranas » Post

I think in this case is better think of some engine-based solution even if it will not eliminates every possible way to cheat. Not every player is skillful enough or willing to invest time into learning more difficult approach to cheat. Lowering possible cheaters number is win nonetheless.

That's being said I do think that some sort of CSM Lua API wrapper could be defined on server by server admins and clients during login to server would download and load into memory, CSM mods should invoke server wrapper instead of local api definition. Probably MT client cpp code could be modified to mitigate it. However, I do think at least official MT builds should 'respect' server rules.

SergioAvelar
New member
Posts: 2
Joined: Mon Mar 05, 2018 21:30

Re: [discussion]Technical notes on how to defeat(*) oredetec

by SergioAvelar » Post

Hi there. Just installed the game and played some hours and it seems to me that the ore detection is not that big of a deal.

I played MC before and checked this wiki to see where to find more diamonds (Y -256 and below). I dug straight down, putting stairs and was planning to make 2x1 tunnels 2 cubes apart as I used to do in MC but by the 4th tunnel I decided to dig around the ores before removing them.

I am around Y - 260 and the current distribution of minerals is so dense that if you find one ore and dig a 3x3x3 cube around it (leave it floating), you will find another ore and so on and on.

My 2x1 tunnel is now this huge cave with ores exposed everywhere. It is exponential, each ore you uncover may lead to more than one vein.

User avatar
Beerholder
Member
Posts: 199
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: [discussion]Technical notes on how to defeat(*) oredetec

by Beerholder » Post

When players use xray or ore detection, tunnels towards the ores have certain shapes. I.e. I would dig in a certain direction and then make a turn to tunnel towards the ore. How feasible would it be to detect such movements e.g. by keeping track of the last x amount of nodes before the ore was dug? And then detect if there were any dubious movements in this pattern, such as sharp turns? This should obviously be disabled for ores that are exposed to air or better still, when the player is approaching the ore from within a cave.

If someone is making a turn towards the ore, either remove it when there is one node left between player and ore and make it appear as if nothing had happened, remove it the moment it becomes exposed to air, remove it on dig or give it randomly to any of the other players when dug by offender.

Players could defeat this by getting the ore position and making a straight tunnel, but for a sufficiently large x amount of last dug nodes, the player would be forced to back track all the way back to a starting position and work towards the next ore. At which stage the player is possibly better off digging mining tunnels. Especially if his ore disappears (leveling the playing field) or is given away to someone else (turning ore detection into a significant disadvantage)

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [discussion]Technical notes on how to defeat(*) oredetec

by bell07 » Post

Beerholder wrote:When players use xray or ore detection, tunnels towards the ores have certain shapes. I.e. I would dig in a certain direction and then make a turn to tunnel towards the ore. How feasible would it be to detect such movements e.g. by keeping track of the last x amount of nodes before the ore was dug?
Nice idea! Just track ~10 digged nodes for each player using on_dignode, check if the nodes are a straight chain to the same direction, look forward 2-3 nodes for ores in chain direction, and if found swap the ore with a stone nearly outsite the chain direction. And maybe an additional check for this player in next 10 digs if he adjust the direction to the new ore position ...

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests