I took my Server Down

[L-Dog]

Its not worth me hosting this game if its just gonna be attacked with 100 second lag spikes, repeatedly and making my players time-out.

Its Offline, Staying offline... id rather they focus their attack on another server so the damn Devs/Admins/Mods do something about it.

My server is Down.. the Game is the reason it gets lag anyway.... i disabled mods that caused 'lag' (hahaha but it was to no use as its AN EXTERNAL ATTACK... NOTHING TO DO WITH SERVER SETUP!!!!!!!!!!)

i might as well do a back up since its offline but doubt ill put it online again.

it was fun, a laugh... pure joy.. until '********' and '*****' started to DDoS the server tower / data center.

They can cry that it werent them all day, but explain why it started happening the EXACT time they put a server up.


anyway.... ima look into hosting an Xontic Server without risk of attack



Edit: i redacted their names

[L-Dog]

Example

minetest.zarnica.org.ua:30001 0/50 0/1 5.2.0, minetest Zarnica Survival Zarnica Survival Server Dmg Pwd 7d / 2.2y 36 / 3109

no one online.... 3109ms lag... come bros, open your eyes or be left with a suprise

toyshost2.ddns.net:30050 0/15 0/0 5.7.0-dev, minetest C&C City Survival Server [MT5.5.0+ Only] The original server is down. Dmg Pwd 56d / 64d 190 / 1450

same story... no one online but high lag/ping

[LMD]

[skill issue, shift blame to damn Devs/Admins/Mods]

[apparently has not understood how Minetest or their mods work, has no idea where lag comes from and how to fix it]

[considers two players a DDoS]

[starts a witch hunt by accusing some players, not respecting "innocent until proven guilty", not understanding causation vs. correlation]

Some people are not meant to be hosting servers :)

In particular people lacking the necessary technical skills to maintain their servers as well as the bare minimum of respect for their players and the developers of the software.

[L-Dog]

[skill issue, shift blame to damn Devs/Admins/Mods]

[apparently has not understood how Minetest or their mods work, has no idea where lag comes from and how to fix it]

[considers two players a DDoS]

[starts a witch hunt by accusing some players, not respecting "innocent until proven guilty", not understanding causation vs. correlation]

Some people are not meant to be hosting servers :)

In particular people lacking the necessary technical skills to maintain their servers as well as the bare minimum of respect for their players and the developers of the software.

Bro.. its been up more than a year.. i had heavier mods on it... i disabled all the 'laggy' mods ... result of profiler showing my what mods caused lag.. i disable them.

then get hit with 3 massive lag spikes..... its got nothing to do with server nor the Mods, its the players attacking it so my players can go join their trash server.

LMD... im surprised.. i thought you were well versed in MT and linux... telling me im wrong is an insult to your intelligence.... Tell me im wrong bro, i ran this server from day 1, been through everything , been through the lag lad all the lag... this time its different.. im getting solid lag... if it was due to mod makers (of which i edit the mods to clear the errors) then bro - open source mod making is LAGGY.

its got literally nothing to do with mods..nor my server setup... its being attacked lad

im a Perfect Server Admin... Server is hosted in Germany

[Lone_Wolf]

There is a way to prevent the attacks. Have you asked for help with that yet?

[L-Dog]

There is a way to prevent the attacks. Have you asked for help with that yet?

i asked for Help in the 'Connection to server' thread.

its been going on for a week or so... i never seen this lag even when world was first made the world and MapGen was going mental.

iv had the same mods for ages.. months.. never seen a problem (Unified invi caused lag though - i now use sfinvi) - but still get the lag spikes... i disabled many many mods to test it - lag came back even harder and that shouldnt be a possibility as the laggy stuff was disabled

i duno what to do bro... taken if offline was best bet and saves me being online and experiencing it... someone advised me to check my auth.sqlite but i dont have a reader to view it.

[L-Dog]

im going bring server back... i cant give up on my beautiful game, MT with Guns.... Without it, there be a handful of kids in amongst the 8 billion without a smile. its home to someone bro, i just need tackle the lag, shoot at it with my ban gun

Monitor its heart beat... i would like a few people to review what im seeing though, review my questionable areas of the DeBug Log

[c56]

im going bring server back... i cant give up on my beautiful game, MT with Guns.... Without it, there be a handful of kids in amongst the 8 billion without a smile. its home to someone bro, i just need tackle the lag, shoot at it with my ban gun

Monitor its heart beat... i would like a few people to review what im seeing though, review my questionable areas of the DeBug Log

without it they would probably just go shoot guns in the real world so in that regard maybe your server is saving lives

also you might want to look into hiding the game server behind a proxy server (should be a relatively fast computer (remember it only really needs much cpu ) i think a i3-i5 would do the job , any old (2006-2014) gaming rig from ebay or amazon should be plenty to scavenge for parts ) and configuring the proxy server to ignore packets from players that send to many packets (more than 100 per second ) and maybe even shadow ban them (just ignore them ) for 1 day after (and if they still keep spamming packets increase ) using some firewall (best one coded in c++ but luajit could work )

[LMD]

By Occam's Razor, I still consider it much more plausible that something in the mod soup, engine issues or bad hosting or the like is responsible (rather than an "attack"). How have you ruled all of these out?

You must provide hard evidence (which would be as simple as visualizing traffic) to back up your claims before accusing players of DDoSing your server. I hardly doubt that anyone would DDoS your server just to "steal your players". Two players/clients also don't make a DDoS (they might make DoS if you have inefficient mods however).

[L-Dog]

By Occam's Razor, I still consider it much more plausible that something in the mod soup, engine issues or bad hosting or the like is responsible (rather than an "attack"). How have you ruled all of these out?

You must provide hard evidence (which would be as simple as visualizing traffic) to back up your claims before accusing players of DDoSing your server. I hardly doubt that anyone would DDoS your server just to "steal your players". Two players/clients also don't make a DDoS (they might make DoS if you have inefficient mods however).

My mods are fine.

i get this before a lag spike...

Code: Select all

2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts

What that mean?.... i got thousands of lines of this... a 100 second lag spike contains like 15000 characters of this - this post can only handle 6000 characters so i cant show you a full one.

is that a single player causing it or is it external?

Mod Soup: ... i disabled mods that caused most CPU% usage... prior to disabling the Mods i had no lag problems, it was low lag and stable.

Even with 14 players online it was low lag.. below 3 seconds at the most... i sit alone at spawn no one online - 100 second lag spikes!!! over and over again... So how can the server handle 14 players without much lag but struggle to handle me alone at spawn doing nothing? .... dont make sense to me.

its like - they seen me with a few players and trying to sh*t on me and players think its a laggy bag of trash.

i think click-rate adjusted players could be causing it and i count their clicks... placing 20 nodes a second, digging 33 a second... i even nerfed the fasted tools to try catch them out.

They will get bored of it soon... even mine and others RTT is high, lastnight it was at 1500ms during lag spikes. others get 300ms... normally mine should be 28 - 33ms, not 1500ms.... RTT might mean nothing but its adding to my case.

Tell me how to get you this 'Traffic' data and ill show you everything... my Host is really good, never had lag with them, it has 4gb ram and i think 2 cores

[LMD]

My mods are fine.

i get this before a lag spike...

Code: Select all

2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts

What that mean?.... i got thousands of lines of this... a 100 second lag spike contains like 15000 characters of this - this post can only handle 6000 characters so i cant show you a full one.

How can you be sure that "your mods are fine"? This very much looks like poorly written mods spamming packets to me.

[L-Dog]

My mods are fine.

i get this before a lag spike...

Code: Select all

2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts

What that mean?.... i got thousands of lines of this... a 100 second lag spike contains like 15000 characters of this - this post can only handle 6000 characters so i cant show you a full one.

How can you be sure that "your mods are fine"? This very much looks like poorly written mods spamming packets to me.

im not a lua code expert nor do i know what 'clean code' looks like... with the Example of me standing alone, at spawn, doing nothing, i doubt theres a mod going to spam packets... i had the thought someone managed to build a 'lag machine' but they'd have to be in the active block to make it work.

i just cant see a mod doing it... i even changed the inventory from i3 to sfinv and still getting the same results even though i3 was causing up to 80% usage on average.... iv had i3 a long time and never seen this.

i use 204 mods... Mesecons is the heaviest one no one really uses it (they dont understand it fully) so doubt its that.

[Lone_Wolf]

i get this before a lag spike...

Code: Select all

2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts

That suggests a mod issue, player DDOS SDOS will normally be things like player x failed to join, wrong password repeated 10000x times in the logs

[L-Dog]

i get this before a lag spike...

Code: Select all

2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 170 pkts
2022-10-08 15:52:41: WARNING[ConnectionSend]: con(4/1) Packet quota used up for peer_id=43, was 149 pkts

That suggests a mod issue, player DDOS SDOS will normally be things like player x failed to join, wrong password repeated 10000x times in the logs

i get players failing to join (empty passwords) and few close together, some even wrong passwords... The example of me alone at spawn, the log didnt show anything, it was max lag and then i timedout a couple times in a row.

how can i prevent the players joining like this?.. a pre-join mod / code? .... whatever is being on has started on X day and continues... it wasnt that i installed a mod then seen lag, i was running as it was forever, only then did i disable stuff.

i was even mining with a grenade launcher and no lag due to it... just this persistent 6 seconds lag.

When i reboot server, the lag sits at 0.5 seconds or less.... a couple players join and it hits 6 seconds.. then after a while or a few mins it hits 12 seconds (today)... yesterday it was hitting 100 seconds... the Players online notice this lag, i have a staff member and i tell him to keep an eye on any lag/laggy players.

i dont wanna disable mesecons, but im considering it to test what the lag is... i still think its a couple bad players doing something External.

i thought it was Mod called 'inventory_bags' coz the 'connectionSend' thing comes up around time player uses it but i spammed it and no lag... it comes around during 'HidroPlane' mod, i disabled it and still same lag.

next i disable mesecons and if the lag improves ill be tracing that log to ban anyone in connection with mesecons... but again, i doubt its this... i just tidied up some code... Even in Profiler the Bags didnt use much CPU.

if its due to players failing to join i can fix it... its the Hax Click Rate players i want abolished from MT (when i find these players i wreck them into a rage quit).

[Lone_Wolf]

i get players failing to join (empty passwords) and few close together, some even wrong passwords... The example of me alone at spawn, the log didnt show anything, it was max lag and then i timedout a couple times in a row.

A 'few' is normal, it's only an attack if you're seeing hundreds. If you are then this will probably help: https://github.com/mt-mods/beowulf/blob ... elimit.lua

Otherwise this sounds a lot like an obscure mod causing lag or being abused to cause lag

[Desour]

Just FYI, those warnings were written by sending side of the server to a client (peer_id != 1).
You could investigate what packets were actually sent, and what caused them to be sent. I.e. you could start by logging how much of each packet type is sent per server-step.
Also, note that you can't measure the performance impact of mods by measuring the time it needs in lua. E.g. last time I tried to set_node very often one node (something like `local v = vector.new(0,0,0); local node = {name="default:stone"}; for i = 1, 10000 do minetest.set_node(pos, node) end`), I got it fast in lua, but afterwards I received much lag (even client-side).

Anyway, you probably don't want to know any of this. You came here to complain, and not to ask for help.
So, this is rather for the other persons reading this thread.

[L-Dog]

Just FYI, those warnings were written by sending side of the server to a client (peer_id != 1).
You could investigate what packets were actually sent, and what caused them to be sent. I.e. you could start by logging how much of each packet type is sent per server-step.
Also, note that you can't measure the performance impact of mods by measuring the time it needs in lua. E.g. last time I tried to set_node very often one node (something like `local v = vector.new(0,0,0); local node = {name="default:stone"}; for i = 1, 10000 do minetest.set_node(pos, node) end`), I got it fast in lua, but afterwards I received much lag (even client-side).

Anyway, you probably don't want to know any of this. You came here to complain, and not to ask for help.
So, this is rather for the other persons reading this thread.

i need help fighting this....

im disabling mods at the min... trying to see whats going on.... lag was sitting at 5 seconds today.. spiking to 15 seconds..

i asked a player to try spam build... he done so and no lag came.... i asked him to do so coz i noticed a spike... but the 8 second spike was due to player 'Brad' joining... massive page of connectionSend warnings.

with most of mesecons disabled and with i3 re-enabled im sitting with low lag.. but this can change later when the other players join.

ill check it in the morning ... study the debug, see whats happening.

[L-Dog]

i get players failing to join (empty passwords) and few close together, some even wrong passwords... The example of me alone at spawn, the log didnt show anything, it was max lag and then i timedout a couple times in a row.

A 'few' is normal, it's only an attack if you're seeing hundreds. If you are then this will probably help: https://github.com/mt-mods/beowulf/blob ... elimit.lua

Otherwise this sounds a lot like an obscure mod causing lag or being abused to cause lag

i have this already... but i just updated it... noclip hurt = 500hp (full health of a player) .. noclip hax will kill you.

i like its prejoin thing.. might save me some lag.

[L-Dog]

i respect you guys that are trying to help me.

its not nice spending £244 on a server to have it f**ked over by some trolls.

i disable mesecons (part of it) .... ill find out tomorrow if it has worked, if not then its an external thing... i got info from a player saying its prob an external thing.. i believe him so far as the lag persists ... if its external there needs to be better protection for servers... i mean bro, i dont have £244 to waste, i dont have it to waste on a server.

i was a player... now im an admin of an epic server... the brightest star bro... the hottest fire gets extinguished... you know.. they see it and want to destroy it

[L-Dog]

Its External...

with all BS disabled i get 9 second lag spikes.. im being attacked.

[Festus1965]

wondering that not even your provider is automatic stopping attack

normally OS Linux based offers tools to see connections onto your IP, but even my easy standard router can block them, and tell me some data

there should not pass any attack behind into my NAT / Intranet - if not on the open/forward port

home or provider server ?
if provider ask them about what kind of attack and IP source ... just block that range ...

[L-Dog]

empty

[Festus1965]

hmm

the server-list offers : dogofwar.freemyip.com:25000

via ping or iplocation.net I get the same IP and same provider,

and looking the provider webpage I find this text

Erstklassiger DDoS-Schutz

Wir betreiben einen eigenentwickelten DDoS-Schutz und verfügen über jahrelange Expertise. Wir bieten Schutz für Ihre Infrastruktur, von Layer 3 bis Layer 7.

or after translate.google.com

Best-in-class DDoS protection

We operate an in-house developed DDoS protection and have many years of expertise. We offer protection for your infrastructure, from layer 3 to layer 7.

hmmm

[L-Dog]

hmm

the server-list offers : dogofwar.freemyip.com:25000

via ping or iplocation.net I get the same IP and same provider,

and looking the provider webpage I find this text

Erstklassiger DDoS-Schutz

Wir betreiben einen eigenentwickelten DDoS-Schutz und verfügen über jahrelange Expertise. Wir bieten Schutz für Ihre Infrastruktur, von Layer 3 bis Layer 7.

or after translate.google.com

Best-in-class DDoS protection

We operate an in-house developed DDoS protection and have many years of expertise. We offer protection for your infrastructure, from layer 3 to layer 7.

hmmm

Hmmmmmm.... i thought i had this protection... its why i pick server in Germay.

iv kept an eye on lag and player numbers / players activity... i stand alone now, on my server with 0.09ms lag, repeated over 40 mins....

Code: Select all

2022-10-09 12:44:54: ACTION[Server]: Server: fofack supplied empty password from XXX.XXX.XX.XXX
2022-10-09 13:02:38: ACTION[Server]: Server: superwereld supplied empty password from XX.XX.XXX.XXX

this increases Max_Lag... iv been having this 'Empty Password' for a couple weeks... with 9 players on this morning the lag was 2 seconds to 6 seconds then spiked to 15 seconds... its a deliberate act of a player/players to cause this, im also thinking its Auto-clicker .. or some player knows how to lag it.

i have the mods re-enabled and i sit at 0.09ms, its not the mods causing it, its players abusing my server.... like an in-game DDoS.

But by far.. this 15 seconds DOES NOT EXPLAIN why i was getting 100 second Time-Out spikes, that is a different matter, thats an attack.... nothing in-game could generate this other than WorldEdit or /clearobjects full ....

[LMD]

The first D of DDoS stands for distributed. An "in-game DDoS" is not a DDoS but rather a simple DoS. The only proper way to fix it is to fix your mods (which obviously becomes nontrivial to impossible with a large enough mod soup).