mod LAG impact comparison [closed]

[Festus1965]

Could you test the impact of the supercub mod? And any other airplane mod I made.

sure can:
https://github.com/APercy/supercub : bbdfe4a / yesterday

depends : mobkit, default, biofuel
but get:

Code: Select all

ERROR[Main]: mod "supercub" has unsatisfied dependencies:  "airutils" "biofuel" "mobkit"

checking
* mobkit (still ddea141 / 01.02. 2021) as tested before, but
* biofuel (oh newer: dd8a199 / 21.02.2022) - testing ...
* airutils (d1e5ec2 / 03.05.2022) - ??? depends.txt did not show airutils ???
* supercub (bbdfe4a / 03.05.2022) ...

fun only : 'd1e5ec2' die after 5 sec 2 times ?
but may change ... programmer is still working on it ... new version I just saw ... I check that tomorrow when test should start ... sad of my preparing I did already

... testing biofuel ... started 7:20 about ... 12:10
/ profiler show no sign = no impact ... as of ABM, globalstep, etc.
player_api still ruling with 79.4 % as maybe setting 0.033

... testing ... airutils ... sarted 12:15 about ... back to standard 0.09 ... stopped 18:15
sure found:
* 2022-05-03 12:13:19: WARNING[Main]: Field "tile_images": Deprecated; new name is "tiles".
* 2022-05-03 18:14:43: WARNING[Server]: Deprecated call to get_attribute, use MetaDataRef methods instead. (at /home/thomas/.minetest/mods/airutils/pilot_skin_manager.lua:138)

/status 0.09xxx no problem
/profiler:
* player_api : 79.9 %
* fire : 20.0 %
* stinv : 0.2 %
* binoculars, airutils, ?, flowers 0.1 %
* other 0.0 %

so airutils has no own useless impact : good !


test finished:
/profiler show no any sign of supercup
* airutils: 0.1 % again

so sure, supercup is not producing lag with out existing nodes or items on empty server. good.

[Festus1965]

to test games,
I have to take another measure ... as to hope there might be a difference in /status I don't think so

I think to take a slow Acer AllInOne I have
1 CPU / 2 Cores / 2 Threads
E1 1200 / 1400 MHz
4 GB RAM - use 3.4

Ubuntu 18.04.6 LTS

and check then the CPU impact alone,
mean not only see the % in htop, even take the sum of time accumulated
near also profiler to see what part inside is most active, like player_api (~lag machine) in MTG

but need to prepare something there before

start then with:

Code: Select all

minetest --server --config ../games.conf

* minetest game (5.5.0)
* mineclone5 ?

* ... suggestions ?

[Festus1965]

minetest game (5.5.0)
(5.5.0 cc3e7be/31.01.2022)

compare to running htop : CPU 1.3 %, MEM 0.1 % - 1:19:xx yet - as every time reference also

start 7:36 stop 18:08 = 10:32 hh:mm

htop show:
** 1st minetest --server : 7:56:xx, also as 0.7-2.0 % CPU
** 2nd 6:29:xx
** 3rd 0:50:xx
** 4th 0:11:xx
** 5th 0:05:xx
alle show used MEM 3.9%

htop as reference: 09:05:xx

/profiler:
* player_api : 75.2 %
* fire : 24.8 %

so as htop same time is a stable reference used 9.03 min
and all mts threads 15:31 hh:min or 15.5 min
the relarionship is factor 1.71, or mts used 71 % more CPU then reference.

just shore to see another reference: client, small window, fps 5, paused at exit screen :
= CPU 15-17%, MEM 17.9 %, and in short time take place 1 with accumulated time ...
(on my gaming rig same paused client does CPU 1.3%, MEM 5.4 %, so CPU effect is 12.3 times higher on small server and easier to spot)

[LMD]

Obviously supercub requires no resources while idling as it uses no globalstep. This "profiling" is ridiculous and the comparison isn't in the slightest fair; the numbers are being grossly misinterpreted.

BTW, which version of player_api are you running? The MTG 5.5 one?

[Festus1965]

Obviously supercub requires no resources while idling as it uses no globalstep. This "profiling" is ridiculous and the comparison isn't in the slightest fair; the numbers are being grossly misinterpreted.

BTW, which version of player_api are you running? The MTG 5.5 one?

useless to argue with you, as you seam even not able to read ...
* MTG
* empty server engine against used server data
to let all mods run on same hardware, same OS/settings, same engine/MTG, same empty world is most fair possible

but instead of useless, fact-less complain - make it better ! in your own testing thread ?

yes, sure - if I look into the fE mod supercup and check the code for ABM, globalstep or lbm I might have imaged that there is no impact. But that is the proof of testing real : facts and proof of
no impact on empty server engine world ... and that I am testing at first.

That real data (world with generated Mapblocks, build nodes and gamer) looks a bit different you could see at my world thread AsiaThailand and the offered /profiler there, but also there I proof the difference Impact of pipeworks with technic against techage:
profiler result with all three mods:
* technic : 35.2 %
* pipeworks : 28.2 %
...
* techage : 0.5 %
Wow ... why I should continue using lag mods, when most of actual gamer already changed to techage and still this impact of pipeworks+technic ?
And how can I recommend others to use it by this FACTS ?

[Festus1965]

not know that mineclone5 is also just a fork of mineclone2 ... took this first

source : https://github.com/kay27/MineClone5
version : 361314e / Dec 10, 2021

test running since 7:35 ICT (server and htop together) ... until 18:23
* mts 8:56 with most 0.7 - 1.6 % - 3.9 % MEM
* mts 7:22
* mts 5:03
* mts not more found ?
to
* htop 9:43 -

during login interesting - all time no gamer, then I log in:
Warning : active block modifiers took 201 (processed 437 of 445 active blocks)
/profiler:
* player_api 80.9 %
* fire : 19.1 %
* default : 1.6 % ... is over 100%
so far no diff to mtg

releationship or impact:
htop 9:43 hh:mm or 9.716 min
against mts 21:21 hh:mm - 21.25 min
the relationships is 2.75 times more than htop or 175 % more CPU time then htop

[LMD]

useless to argue with you, as you seam even not able to read ...

I am not sifting through 4 pages of this crap to find one information I need.

but instead of useless, fact-less complain - make it better ! in your own testing thread ?

I am still very much convinced that there is almost nothing to be optimized in player_api. MCL mods are not the same as the MTG ones BTW.

[Festus1965]

I am not sifting through 4 pages of this crap to find one information I need.

All Information is just in the 2 first posts, and then I linked all results inside the Ranking of Post 2. So if follow this, there is no big need so scroll over all.

This is freedom of thinking and meaning:
You think ! there is no optimization needed, I proofed with facts there IS some optimization needed.

As I tell also the base again :
a mod, even inside MTG or even builtin, that is using near 75-90% of all CPU usage on an empty world,
in this case player_api WITHOUT any logged in gamer, I wasting time, CPU an money via electric !!!

[Festus1965]

as seen interesting mod offered and direct pointed to a older similar mod

testing:
https://github.com/Rotfuchs-von-Vulpes/ ... o-minetest
0aa3e1a / 05.05.2022

run just short: (and was usable)
/profiler:
* what_is_this_owo: 66.6 %
* poshud: 19.1 (show still 0.09-0.09-0.09)
* player_api: 10.2 %
* fire: 2.3 %
* default: 1.5

so, when what_is_this_owo uses 66% or 2/3 off all CPU work, as the other 33% are old empty 100% = adding 200%
in that short a impact of tripple the base lag / cpu usage.


new test with new version: 59459cc/07.05.2022

/profiler:
* player_api : 77.5 %
* fire : 22.5 %
* default : 2.1 %
* sfinv : 1.0 %
* flowers : 0.6 %
* what_is_this_owo : 0.0 %

that is a performance change.
If the mod does still the same it as build for - Congratulations.
As no gamer in server, a mod should have near NO tasks to work.

[Festus1965]

compare to

here mod witt
https://github.com/Pevernow/Witt-2
4c7e23b - 01.07.2021

short tested (but then use command = crash)
/profiler: new without mod poshud
* witt : 64.7 %
* player_api : 23.8 %
* fire : 11.4 %
* default : 1.3 %
* sfinv : 0.3 %

compare ... tomorrow

so as taken 64.7 %, the other 35.3 % was 100% usage before = so is 183 % rising the load or near tripple old CPU usage.

All results

[Festus1965]

testing

poshud
https://github.com/orwell96/poshud
3ad4783 / 13.11.2019

running longer
but a preview:
/profiler:
* poshud : 84.3 %
* player_api : 8.9 %
* fire : 6.8 %
* sfinv : 0.7 %
looks not good ... as I also already request the using on my server, as after pipeworls+technic are gone ...
rising to 8.8 % then

[Festus1965]

still working on some explanations:

Even on my server AsiaThailand I could proof last Sunday that without pipeworks+technic+depending mods ...
ARE lowering the used CPU time a lot:

from /profiler I should have saved near 50% of mt work before

the % of minetestserver on CPU fall a lot to older setting, from before 33 % now to under 20%

BUT it has by far not the same effect on the /status max_lag number.

Mean, even I use 50% less work, max_lag mean to show near same high lag.

Only at poshud I can imagine that my lag got less a lot.

But as long gamer, also from lag servers are wondering why I can do so fast ... that bothers me.

There is a key point I don't get yet ...
every gamer as about 15ms, ok ... but why : what does a gamer data flow mean to server and where can I optimize that also ?

[Festus1965]

tested short sprint

as often gamer ask for faster basic move ...
source : https://github.com/GunshipPenguin/sprint
d714ce3/12.05.2018

as usual /profiler:
* sprint : 51.1 % ... on empty server, and even I am in did NOT use it.
* player_api : 38 %
* fire : 9.9 %
* default : 1.8 %

Code: Select all

 sprint:                                                 |         0 |       111 |         7 |   0.0 | 100.0 |  51.5
  - globalstep[1] .....................................  |         1 |       111 |         7 |   0.0 | 100.0 |  51.7
  - on_joinplayer[1] ..................................  |        28 |        28 |        28 |   0.9 |   0.9 |   0.9

51.1 % on all, mean half of new usage is sprint, or it rise basic load by 104.5 % = double the basic lag, CPU usage

[Festus1965]

testing hpsprint

source: https://github.com/minetest-mods/hbsprint
f566d0f/19.02.2021

as usual: /profiler save
* player_api : 56.2 %
* hbsprint : 28.9 %
* fire : 14.9 %
* default : 1.6 %
* sfinv : 0.5 %

Code: Select all

 hbsprint:                                               |         0 |       195 |         1 |   0.0 | 100.0 |  28.9
  - globalstep[1] .....................................  |         1 |       195 |         1 |   0.0 | 100.0 |  34.9
  - on_joinplayer[1] ..................................  |        13 |        13 |        13 |   0.5 |   0.5 |   0.5

rising a bit, 28% part now, mean 38% more work, lag, CPU usage ... also just by set true - not even used

[LMD]

As I tell also the base again :
a mod, even inside MTG or even builtin, that is using near 75-90% of all CPU usage on an empty world,
in this case player_api WITHOUT any logged in gamer, I wasting time, CPU an money via electric !!!

This is factually wrong. You can't convert the relative Lua CPU consumption determined by the profiler - which only covers whatever the profiler chooses to cover, which pretty much boils down to globalsteps in an empty server - to absolute CPU consumption in percent, especially when these globalsteps are in the microseconds. What you're also completely missing are the C++ costs: ABMs are a lot more expensive than the player_api globalstep but won't show up at all if there's nothing to execute because the required checks (looping over blocks) are done by Minetest. Not to mention all the other serverstep C++ code executed won't show up either.

And now you're dragging in witt and poshud the same way: These mods have nearly-optimal code, but they have to rely on looping over the connected players each globalstep as well and thus will take a few µs too. This is entirely insignificant and only using relative numbers you can pretend that there is a problem.

TL;DR: Your benchmarking of empty servers and your conclusions are full of crap. Mods that use literally the only proper pattern to loop over all (zero) players every step are being incorrectly depicted as being unnecessarily inefficient / having a performance problem.

Neither player_api, nor witt or poshud have a performance problem.

[Festus1965]

Sorry,

but mods that use a LOT of CPU time
without been used as of gamer in or mod nodes/items existing
have a huge lag impact problem !

But don't you see I do it ... I compare it ... whatever you tell.
Make a better checking, with what you think is right ... I will not stop it.

Have a nice day.

I give server admins a hint, what they can expect with some of the mods ... and compare inside the list, look server, and what most mods they use and see their lag ... is quit the result I show up here.

Yes, plus I have the advantage of full control of my hardware, especial cache setting.
So on my used rig, the impact is even very low. If I simulate the typical VPS most use ... wow ... more bad even.

I know from two server that also use pipeworks+technic very well how it feels to game there, plus one using advtrains, the other petz ... and that teached me a lot.

There are admins, telling I am not helping, just complain ... but I can proof on my server all tuning options from hardware, OS setting, Minetest setting, database and mod tuning and it works. 25 gamer under 500ms ... a (unseen) dream for them.

Bye !

[LMD]

I give server admins a hint, what they can expect with some of the mods ... and compare inside the list, look server, and what most mods they use and see their lag ... is quit the result I show up here.

And I sure hope admins don't take your hints regarding your idle measurements of player_api, witt, poshud etc.; pipeworks, petz and adv_trains however are known to be resource hogs to some extent and I won't question your measurements in that regard.

Bye too.

[Festus1965]

beside this,
it is much more interesting me how data from client is 'worked out' inside engine,
that every gamer join into my server, is raising it by average of 15ms.

If then the engine is doing normal, and is not having a problem ... how can then every gamer raise it ?

Are there data, waiting for each other, that it takes longer to work a dig, build grow in?

Here I am a bit in a black hole now ...
as I last Sunday saw with 24h pipeworks and technic off, and normal loosing 50% of all impact, the max_lag, also the poshud numbers don't show this.

That is my open point now, hen 50% of CPU minetest work should be gone as mods off, why then no faster calculation ans also 50% of max_lag ?

The CPU usage was sure far don, less % on the Threads ... but Does this not how up in lag reducing ?

[wsor4035]

I give server admins a hint, what they can expect with some of the mods ... and compare inside the list, look server, and what most mods they use and see their lag ... is quit the result I show up here.

And I sure hope admins don't take your hints regarding your idle measurements of player_api, witt, poshud etc.; pipeworks, petz and adv_trains however are known to be resource hogs to some extent and I won't question your measurements in that regard.

Bye too.

agree with this, these measurements shouldnt be used at all

[Festus1965]

agree with this, these measurements shouldn't be used at all

that would indicate, that /status max_lag might be useless and the /profiler results also
but they ARE part of inner minetest, the official engine ... so why they NOW are not taken to compare mods now ?

Your are both funny !


I imagine minetest engine with mods like a Formula one circuit,
mt has to repeat the same loop every time - with different parts, but anyway.

There is like a box stop, like player logged out,
but basic routine goes on.

The circuit is still running, with less data racer/gamer on it, but here comes my results about some mods, builtin and free to use, that are doing much more on the empty circuit than obviously needed.

And that is a fact with only one open question for me actual:

As less 'impact' compare to profiler results, also clear the impact on CPU usage falls - but the same logic event is not seen on reduce overall max_lag yet.
And also I didn't understand yet, why every gamer rise the lag, when expecting mtg impact might be nothing, pointing on so tiny part of seconds a globalstep takes.

Maybe my way to go into it is not perfect, but I see no better option by your posts.

It would be nice, to have like a analyzer, like for a OS start, showing the cause of every time working 'part', and see learn more about internal mechanic, mean what task takes longer, or has to wait for something ready before.

[wsor4035]

your measurements shouldnt be used, since you misrepresenting them.

[Festus1965]

your measurements shouldn't be used, since you misrepresenting them.

then interpret them better as you think !

[LMD]

agree with this, these measurements shouldn't be used at all

Maybe my way to go into it is not perfect, but I see no better option by your posts.

Let me show you two better options for profiling alone:

• The LuaJIT profiler if you want to profile the Lua code alone (note that it is a sampling one, so it will have to run for a while);
• Profiling the full C++ code, e.g. using gprof. Disclaimer: I haven't done this as I'm mostly interested in the Lua side of things.

[Festus1965]

• The LuaJIT profiler if you want to profile the Lua code alone (note that it is a sampling one, so it will have to run for a while);
• Profiling the full C++ code, e.g. using gprof. Disclaimer: I haven't done this as I'm mostly interested in the Lua side of things.

luajit fail as of missing other ...ata... behind

Code: Select all

luajit: init.lua:2: attempt to index global 'minetest' (a nil value)
stack traceback:
	init.lua:2: in main chunk
	[C]: at 0x55bd481681d0
[No samples collected]

... no idea how get around there.

The other is for C so far I understood that.

I will see.

added later:
* https://dev.minetest.net/Development_Tools
* viewtopic.php?f=9&t=20699
?
" https://dev.minetest.net/minetest.get_us_time (nice to know)

[joe7575]

Measuring the CPU load caused by mods is not that easy. I've struggled with this for a long time. I put a lot of effort into this, especially for techage.

CPU load is caused by many things:
- Running code via `minetest.register_globalstep`
- Running code via `minetest.register_abm`
- Running code via `minetest.after`
- Running code via `node.on_timer`
...and others.

But computing time is also consumed in the C++ code, e.g. for communication to the clients

Techage hardly uses `minetest.register_globalstep`, but massively uses `node.on_timer`. That's why techage scores so well on your tests :)

I measure the impact of a mod on the CPU load with:

- a larger test setup (e.g. a typical techage machine)
- a server on a very weak CPU like a Raspi 2 (you can see the impact much better here)
- multiple dummy players that always spawn at the same position
- Nothing else that generates CPU load is allowed to run on the server

Then you can determine the CPU load with `top` and observe the data traffic with `nload`, for example.

If you do this for typical mod use cases, you will get real results...