Massive memory leak in MT 5.0.0?

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Massive memory leak in MT 5.0.0?

by Wuzzy » Tue Mar 12, 2019 07:40

(I wrote this in the News thread, but I repeat it here so it's more visible:)

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.
If I keep running Minetest for a while and explore the world, it eventually eats up all my ~8 gigs of memory at an alarming rate.
This never happened to me in previous versions.

When I quit the world, only some memory seems to be freed. Sooner or later my memory is full and I have to close Minetest. That problem killed my system multiple times in the last days.

The same seems to happen with MTG, the memory consumption also grows and grows, although not that fast.

Did anyone else notice extreme memory consumption? Might this be a regression?
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

User avatar
Linuxdirk
Member
 
Posts: 1873
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk

Re: Massive memory leak in MT 5.0.0?

by Linuxdirk » Tue Mar 12, 2019 09:41

Wuzzy wrote:Did anyone else notice extreme memory consumption? Might this be a regression?

Not personally, but currently I'm on this issue with one of my mods: https://gitlab.com/4w/mtimer/issues/7

During testing added print(collectgarbage("count")) to a globalstep construct that runs every second and the value rised for some seconds and then is reduced and rises again and is reduced again but to a higher value it was reduced before. This continuesd during testing but may trigger an OOM exception like H4mlet explires with Windows 7.


Edit: Found the issue and it is unrelated to potential memory leaks and maybe was actual memory usage gone crazy on Windows. So no, I did not notice any extreme memory consumption..
Last edited by Linuxdirk on Tue Mar 12, 2019 17:37, edited 1 time in total.
 

User avatar
jas
Member
 
Posts: 346
Joined: Mon Jul 24, 2017 18:15
GitHub: jastevenson303
IRC: jas_
In-game: jas
 

sofar
Developer
 
Posts: 1998
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Massive memory leak in MT 5.0.0?

by sofar » Tue Mar 12, 2019 21:59

Quit message received.
AL lib: (WW) ReleaseALBuffers: (0x6b08ac0) Deleted 135 Buffers
==15368==
==15368== HEAP SUMMARY:
==15368== in use at exit: 316,355 bytes in 3,350 blocks
==15368== total heap usage: 1,267,313 allocs, 1,263,963 frees, 3,223,400,122 bytes allocated
==15368==
==15368== LEAK SUMMARY:
==15368== definitely lost: 2,456 bytes in 20 blocks
==15368== indirectly lost: 6,068 bytes in 138 blocks
==15368== possibly lost: 0 bytes in 0 blocks
==15368== still reachable: 307,831 bytes in 3,192 blocks
==15368== suppressed: 0 bytes in 0 blocks
==15368== Rerun with --leak-check=full to see details of leaked memory
==15368==
==15368== For counts of detected and suppressed errors, rerun with: -v
==15368== Use --track-origins=yes to see where uninitialised values come from
==15368== ERROR SUMMARY: 69036 errors from 56 contexts (suppressed: 0 from 0)


This is the output from a valgrind run where I actually connected to a server, using the UI, then disconnected.

It appears that any memory leaks are reasonably small. I didn't play for a long time, but 2k bytes lost is tiny compared to the amount of memory allocated and not a major concern. More data would be better, of course, but running under valgrind is expensive and (unplayably) slow.
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Massive memory leak in MT 5.0.0?

by Wuzzy » Wed Mar 13, 2019 06:16

I believe I found the root of all evil!

The profiler. I totally forgot I had turned it on because I was testing mapgen. So it just kept running and running and running. Surely this allocated tons of data. Turning the profiler lowers the memory consumption greatly! :D
Still a bit strange that Minetest doesn't free the memory after I left the game.

And yeah, the memory leak you found is tiny, it can't explain why my memory was eaten away so fast (it was a matter of, like 15-20min to have my 8 gigs eaten away).
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Massive memory leak in MT 5.0.0?

by Wuzzy » Thu Mar 14, 2019 11:41

Oh, man. Turns out it was not that simple. :-(
After a longer session I saw my memory getting eaten up fast again. But my profiler was off the whole time. WTF?

Maybe I should see and check if this happens with default settings as well. But something smells fishy …
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

ShadMOrdre
Member
 
Posts: 252
Joined: Mon Dec 29, 2014 08:07
Location: USA
GitHub: ShadMOrdre
In-game: shadmordre

Re: Massive memory leak in MT 5.0.0?

by ShadMOrdre » Thu Mar 14, 2019 16:27

Wuzzy,

Maybe a mod, that now acts differently under 5.0?
MY MODS: lib_ecology lib_materials lib_clouds lib_node_shapes ---- Inspired By: Open Source Virtual World Simulator Opensimulator.
 

sofar
Developer
 
Posts: 1998
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Massive memory leak in MT 5.0.0?

by sofar » Fri Mar 15, 2019 05:31

It's entirely possible that my short session wasn't really representative, and it can't really be because valgrind is really really costly to use. Knowing if Wuzzy used e.g. MCL2 or MTG or some other game would help a lot.
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Massive memory leak in MT 5.0.0?

by Wuzzy » Fri Mar 15, 2019 06:27

Sofar fails at reading comprehension.
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

User avatar
Festus1965
Member
 
Posts: 788
Joined: Sun Jan 03, 2016 11:58
Location: Thailand - Chiang Mai (+5-6h to MEZ)
In-game: Thomas Explorer

Re: Massive memory leak in MT 5.0.0?

by Festus1965 » Sat Mar 16, 2019 01:16

guys,
I have less time but yesterday happened something remind me this thread.
I have an own web server here beside the mts running, what I use also to test clients and jail minetest clones, whatever ...

Fact:
I still have running there the mtc 5.0.0-bba4563 and didn't update them as they run since I use them,
but yesterday I changed 5 of them to "enable : saving map received from server" and logged them in first time ever server freeze and no any ram out of 16 GB was free anymore.

So for sure that was the only change I did in last 2 weeks for sure made this big difference.

It this is fixing to that thread, take it, if not tell me. As then also I might be able to change all of them to 5.0.0-stable and do the same again, disable and enable and let run again. But first I wait for your response keep this config, but no saving and testing ... just now running
I own minetest.one and offer a Minetestserver called Thailand Minetest.one (5.1.0-dev) on server.minetest.one.
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Massive memory leak in MT 5.0.0?

by Wuzzy » Sat Mar 16, 2019 03:03

No, this setting was off for me.
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

sofar
Developer
 
Posts: 1998
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Massive memory leak in MT 5.0.0?

by sofar » Sat Mar 16, 2019 04:04

Wuzzy wrote:Sofar fails at reading comprehension.


I don't tend to re-read the entire conversation again, sorry.
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

For those who didn't even read the first post …

by Wuzzy » Sat Mar 16, 2019 06:09

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

sofar
Developer
 
Posts: 1998
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: For those who didn't even read the first post …

by sofar » Mon Mar 18, 2019 19:41

Wuzzy wrote:
I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.

I believe I noticed a massive memory leak in Minetest 5.0.0 while developing MineClone 2.


I should just report this.

I clicked on "new posts" as usual, and nothing on my page that my browser showed me included the information. Because the thread was several replies long, I didn't scroll back up. I have other things to do, and re-reading long threads is something I just don't do.

But hey, copy and paste it a few more time if that makes you happy. Go run valgrind yourself while you're at it.
 

User avatar
Wuzzy
Member
 
Posts: 3399
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Re: Massive memory leak in MT 5.0.0?

by Wuzzy » Tue Mar 19, 2019 02:02

:-(
My creations. I gladly accept bitcoins: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 


Return to Problems



Who is online

Users browsing this forum: No registered users and 1 guest