Minetest Forums

Moderator edit: You need to update LuaJIT, this is fixed in latest versions. (Note that LuaJIT no longer tags releases, it's rolling release from git)


Some people keep reporting OOM errors for my subgame “MineClone 2”.

Here's one of the posted errors:

2017-11-14 21:07:37: ERROR[Main]: ServerError: AsyncErr: Lua: finishGenOOM error from mod 'mcl_dungeons' in callback environment_OnGenerated(): not enough memory
2017-11-14 21:07:37: ERROR[Main]: Current Lua memory usage: 14 MB

AFAIK “OOM” means “Out Of Memory” and this might not be my fault. My question is, is there anything that I, as subgame maker, can use to prevent this anyway? Or does the user have to set a certain setting?
Or did I just screw up? What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D

But 14 MB Lua memory limit doesn't seem much, I'm surprised.

Wuzzy wrote:What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D

While I have no idea what your problem might really be, I suspect any sort of intensive code loop could bog down memory and cause such an error.

I often got this in 0.4.14 and below with no mods enabled at all by just exploring the world. I explored too much, I got this error, which repeated commonly until I close down Minetest and restarted it.

I also get it from enabling way too many mods, nearly all of which added only static nodes.

Most likely caused by LuaJIT in combination with map generating mods.
These errors are usually caused by VManip data or noise tables and may reach a few gigabytes of RAM in a short time. I guess the unexpected low reported memory use in the logs were caused by the garbage collector, which cleaned up the unused data right before the original RAM use was measured.
Please see this comment and the following ones to get possible solutions for this problem if reducing the memory use in the mods isn't possible.

You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.

Another thing you could do is put all of your on_generated calls into one function, assuming you have more than one. Not duplicating the map data saves a lot of memory thrashing. If you're worried about using each mod separately, just check for the presence of one and use its on_generated loop, breaking out the independent parts of each mod's callback as a separate function the main loop can call. I did that with booty and squaresville (the lua version). It saves a lot of time too, as moving map data sucks up a lot of cpu time.

Best fix:
Run a server and connect local. This solves most oom errors for me.

duane wrote:You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.

LuaJIT is out of memory, not their system

What Krock wrote, don't use LuaJIT and/or apply the memory use optimisations.

have also found this one Minetest with LuaJit GC64 mode (>2gb mem)
and this one Lua OOM crashes
but this here seams the with the lastest post.

I go today ... so 1626 MB is already much more than the other posts with 12 MB or 52 MB
but as I have 16 GB RAM all, and Linux is using less than 1 GB
its a shame cant use more yet ... without extra compiling movements ...

The max of used memory minus the 820 from Linux itself was yet near 6 GB, guess most map-data

I noticed this code in duane's Integral Tree mod, invoked at the start of its mapgen on_generated() function:
Can this sort of thing help? Should I be putting it in my mapgen mods before using voxelmanip?

Doing a full GC cycle might help yes, could clear the memory before the OOM ever occurs. But there is a bit of overhead in calling garbage collectors.

I am seriously wondering how effective it is in duane's case. The data array in the latest versions is a local outside the function (local data = {}), in previous versions it was local to the function (local data = vm:get_data()).

So in the latest version there will be a reference to the data array at all times and as such it will never be garbage collected. And I don't think vm:set_data(data) clears the data array in the end, correct?

Then again, maybe the VM data array was not what was causing memory issues for duane in the first place ... (or, it was not the only thing)

Anyways, let's hope we will have a stable LuaJIT 2.1 release soon so we can start to play around with GC64 mode :-) I have regularly been checking luajit.org for news, but alas the last beta is from already quite a while ago. It will solve the 2GB limit, but IMO that does not mean games and mods should not be memory efficient anymore ...... ;-)

DrFrankenstone as far as i know Duane's garbage collection code is a personal hack that was used before i updated my advice in viewtopic.php?f=18&t=16043 and before i posted the latest LVM example in viewtopic.php?f=18&t=19836

If you code according to my latest optimisation advice i doubt garbage collection is needed, as the noise and voxelmanip tables (which are what use so much memory) are all reused.

Note that, as far as i know, the message 'Current Lua memory usage' does not show the memory usage that caused OOM, you can see this value is often very low and not near 2GB. Or maybe, OOM is something to do with memory addresses and not actual memory used.