Lua OOM errors

Post Reply
User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Lua OOM errors

by Wuzzy » Post

Moderator edit: You need to update LuaJIT, this is fixed in latest versions. (Note that LuaJIT no longer tags releases, it's rolling release from git)


Some people keep reporting OOM errors for my subgame “MineClone 2”.

Here's one of the posted errors:
2017-11-14 21:07:37: ERROR[Main]: ServerError: AsyncErr: Lua: finishGenOOM error from mod 'mcl_dungeons' in callback environment_OnGenerated(): not enough memory
2017-11-14 21:07:37: ERROR[Main]: Current Lua memory usage: 14 MB
AFAIK “OOM” means “Out Of Memory” and this might not be my fault. My question is, is there anything that I, as subgame maker, can use to prevent this anyway? Or does the user have to set a certain setting?
Or did I just screw up? What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D

But 14 MB Lua memory limit doesn't seem much, I'm surprised.

User avatar
GreenXenith
Member
Posts: 1356
Joined: Wed Oct 28, 2015 01:26
GitHub: GreenXenith
Location: UTC-8:00
Contact:

Re: Lua OOM errors

by GreenXenith » Post

Wuzzy wrote:What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D
While I have no idea what your problem might really be, I suspect any sort of intensive code loop could bog down memory and cause such an error.
YouTube | Mods | Patreon | Minetest Discord @greenxenith

You should not be able to read this message.

User avatar
v-rob
Developer
Posts: 969
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Lua OOM errors

by v-rob » Post

I often got this in 0.4.14 and below with no mods enabled at all by just exploring the world. I explored too much, I got this error, which repeated commonly until I close down Minetest and restarted it.

I also get it from enabling way too many mods, nearly all of which added only static nodes.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
Krock
Developer
Posts: 4648
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker
Location: Switzerland
Contact:

Re: Lua OOM errors

by Krock » Post

Most likely caused by LuaJIT in combination with map generating mods.
These errors are usually caused by VManip data or noise tables and may reach a few gigabytes of RAM in a short time. I guess the unexpected low reported memory use in the logs were caused by the garbage collector, which cleaned up the unused data right before the original RAM use was measured.
Please see this comment and the following ones to get possible solutions for this problem if reducing the memory use in the mods isn't possible.
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>

User avatar
duane
Member
Posts: 1715
Joined: Wed Aug 19, 2015 19:11
GitHub: duane-r
Location: Oklahoma City
Contact:

Re: Lua OOM errors

by duane » Post

You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.

Another thing you could do is put all of your on_generated calls into one function, assuming you have more than one. Not duplicating the map data saves a lot of memory thrashing. If you're worried about using each mod separately, just check for the presence of one and use its on_generated loop, breaking out the independent parts of each mod's callback as a separate function the main loop can call. I did that with booty and squaresville (the lua version). It saves a lot of time too, as moving map data sucks up a lot of cpu time.
Believe in people and you don't need to believe anything else.

User avatar
maikerumine
Member
Posts: 1420
Joined: Mon Aug 04, 2014 14:27
GitHub: maikerumine
In-game: maikerumine

Re: Lua OOM errors

by maikerumine » Post

Best fix:
Run a server and connect local. This solves most oom errors for me.
Talamh Survival Minetest-->viewtopic.php?f=10&t=12959

User avatar
rubenwardy
Moderator
Posts: 6969
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: Lua OOM errors

by rubenwardy » Post

duane wrote:You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.
LuaJIT is out of memory, not their system
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
paramat
Developer
Posts: 3700
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat
IRC: paramat
Location: UK

Re: Lua OOM errors

by paramat » Post

What Krock wrote, don't use LuaJIT and/or apply the memory use optimisations.

User avatar
Festus1965
Member
Posts: 4181
Joined: Sun Jan 03, 2016 11:58
GitHub: Festus1965
In-game: Festus1965 Thomas Thailand Explorer
Location: Thailand ChiangMai
Contact:

Re: Lua OOM errors

by Festus1965 » Post

have also found this one Minetest with LuaJit GC64 mode (>2gb mem)
and this one Lua OOM crashes
but this here seams the with the lastest post.

I go today ...

Code: Select all

2018-08-02 06:37:18: ACTION[Server]: Player digs pipeworks:mese_sand_tube_2 at (21735,14,23884)
compassgps writing settings
2018-08-02 06:37:39: ERROR[Main]: ServerError: AsyncErr: ServerThread::run Lua: OOM error from mod 'technic' in callback LuaABM::trigger(): not enough memory
2018-08-02 06:37:39: ERROR[Main]: Current Lua memory usage: 1626 MB
so 1626 MB is already much more than the other posts with 12 MB or 52 MB
but as I have 16 GB RAM all, and Linux is using less than 1 GB
its a shame cant use more yet ... without extra compiling movements ...

The max of used memory minus the 820 from Linux itself was yet near 6 GB, guess most map-data
Last edited by Festus1965 on Mon Aug 06, 2018 08:00, edited 1 time in total.
Human has no future (climate change)
If urgend, you find me in Roblox (as CNXThomas)

User avatar
DrFrankenstone
Member
Posts: 231
Joined: Tue May 24, 2016 05:36
GitHub: treer
Location: Australia
Contact:

Re: Lua OOM errors

by DrFrankenstone » Post

I noticed this code in duane's Integral Tree mod, invoked at the start of its mapgen on_generated() function:

Code: Select all

    -- Deal with memory issues. This, of course, is supposed to be automatic.
    local mem = math.floor(collectgarbage("count")/1024)
    if mem > 500 then
        print("Integral is manually collecting garbage as memory use has exceeded 500K.")
        collectgarbage("collect")
    end
Can this sort of thing help? Should I be putting it in my mapgen mods before using voxelmanip?

User avatar
Beerholder
Member
Posts: 199
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Lua OOM errors

by Beerholder » Post

Doing a full GC cycle might help yes, could clear the memory before the OOM ever occurs. But there is a bit of overhead in calling garbage collectors.

I am seriously wondering how effective it is in duane's case. The data array in the latest versions is a local outside the function (local data = {}), in previous versions it was local to the function (local data = vm:get_data()).

So in the latest version there will be a reference to the data array at all times and as such it will never be garbage collected. And I don't think vm:set_data(data) clears the data array in the end, correct?

Then again, maybe the VM data array was not what was causing memory issues for duane in the first place ... (or, it was not the only thing)

Anyways, let's hope we will have a stable LuaJIT 2.1 release soon so we can start to play around with GC64 mode :-) I have regularly been checking luajit.org for news, but alas the last beta is from already quite a while ago. It will solve the 2GB limit, but IMO that does not mean games and mods should not be memory efficient anymore ...... ;-)

User avatar
paramat
Developer
Posts: 3700
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat
IRC: paramat
Location: UK

Re: Lua OOM errors

by paramat » Post

DrFrankenstone as far as i know Duane's garbage collection code is a personal hack that was used before i updated my advice in viewtopic.php?f=18&t=16043 and before i posted the latest LVM example in viewtopic.php?f=18&t=19836

If you code according to my latest optimisation advice i doubt garbage collection is needed, as the noise and voxelmanip tables (which are what use so much memory) are all reused.

Note that, as far as i know, the message 'Current Lua memory usage' does not show the memory usage that caused OOM, you can see this value is often very low and not near 2GB. Or maybe, OOM is something to do with memory addresses and not actual memory used.

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 2 guests