Lua OOM errors

User avatar
Wuzzy
Member
 
Posts: 3233
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy

Lua OOM errors

by Wuzzy » Wed Nov 15, 2017 04:43

Some people keep reporting OOM errors for my subgame “MineClone 2”.

Here's one of the posted errors:
2017-11-14 21:07:37: ERROR[Main]: ServerError: AsyncErr: Lua: finishGenOOM error from mod 'mcl_dungeons' in callback environment_OnGenerated(): not enough memory
2017-11-14 21:07:37: ERROR[Main]: Current Lua memory usage: 14 MB


AFAIK “OOM” means “Out Of Memory” and this might not be my fault. My question is, is there anything that I, as subgame maker, can use to prevent this anyway? Or does the user have to set a certain setting?
Or did I just screw up? What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D

But 14 MB Lua memory limit doesn't seem much, I'm surprised.
My creations. I gladly take any bitcoins you have lying around: 17fsUywHxeMHKG41UFfu34F1rAxZcrVoqH
 

User avatar
GreenDimond
Member
 
Posts: 1189
Joined: Wed Oct 28, 2015 01:26
Location: A place
GitHub: GreenXenith
IRC: GreenDimond
In-game: GreenDimond

Re: Lua OOM errors

by GreenDimond » Wed Nov 15, 2017 05:51

Wuzzy wrote:What are possible stupid ways to code a mod so horribly bad that it can trigger an OOM error for everyone? :D

While I have no idea what your problem might really be, I suspect any sort of intensive code loop could bog down memory and cause such an error.
My YuTube channel | I moderate the HOMETOWN Server. | Click here to see my (5) mods!
 

User avatar
v-rob
Member
 
Posts: 559
Joined: Thu Mar 24, 2016 03:19
Location: Right behind you.
GitHub: v-rob

Re: Lua OOM errors

by v-rob » Wed Nov 15, 2017 15:41

I often got this in 0.4.14 and below with no mods enabled at all by just exploring the world. I explored too much, I got this error, which repeated commonly until I close down Minetest and restarted it.

I also get it from enabling way too many mods, nearly all of which added only static nodes.
 

User avatar
Krock
Developer
 
Posts: 4117
Joined: Thu Oct 03, 2013 07:48
Location: Switzerland
GitHub: SmallJoker

Re: Lua OOM errors

by Krock » Wed Nov 15, 2017 18:55

Most likely caused by LuaJIT in combination with map generating mods.
These errors are usually caused by VManip data or noise tables and may reach a few gigabytes of RAM in a short time. I guess the unexpected low reported memory use in the logs were caused by the garbage collector, which cleaned up the unused data right before the original RAM use was measured.
Please see this comment and the following ones to get possible solutions for this problem if reducing the memory use in the mods isn't possible.
Mod Search Engine - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
 

User avatar
duane
Member
 
Posts: 1273
Joined: Wed Aug 19, 2015 19:11
Location: Oklahoma City
GitHub: duane-r

Re: Lua OOM errors

by duane » Thu Nov 16, 2017 02:10

You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.

Another thing you could do is put all of your on_generated calls into one function, assuming you have more than one. Not duplicating the map data saves a lot of memory thrashing. If you're worried about using each mod separately, just check for the presence of one and use its on_generated loop, breaking out the independent parts of each mod's callback as a separate function the main loop can call. I did that with booty and squaresville (the lua version). It saves a lot of time too, as moving map data sucks up a lot of cpu time.
Believe in people and you don't need to believe anything else.
 

User avatar
maikerumine
Member
 
Posts: 1343
Joined: Mon Aug 04, 2014 14:27
GitHub: maikerumine
In-game: maikerumine

Re: Lua OOM errors

by maikerumine » Fri Nov 17, 2017 02:49

Best fix:
Run a server and connect local. This solves most oom errors for me.
 

User avatar
rubenwardy
Moderator
 
Posts: 5501
Joined: Tue Jun 12, 2012 18:11
Location: United Kingdom
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy

Re: Lua OOM errors

by rubenwardy » Fri Nov 17, 2017 13:28

duane wrote:You might want to ask your posters what system and software they're using. That's a suspiciously low number for memory. If they're running a server on an old cellphone, you're probably not going to get anywhere.


LuaJIT is out of memory, not their system
 

User avatar
paramat
Developer
 
Posts: 3209
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat

Re: Lua OOM errors

by paramat » Sat Nov 18, 2017 02:05

What Krock wrote, don't use LuaJIT and/or apply the memory use optimisations.
 

User avatar
Festus1965
Member
 
Posts: 309
Joined: Sun Jan 03, 2016 11:58
Location: Thailand - Chiang Mai (+5-6h to MEZ)
In-game: Thomas Explorer

Re: Lua OOM errors

by Festus1965 » Thu Aug 02, 2018 04:19

have also found this one Minetest with LuaJit GC64 mode (>2gb mem)
and this one Lua OOM crashes
but this here seams the with the lastest post.

I go today ...
Code: Select all
2018-08-02 06:37:18: ACTION[Server]: Player digs pipeworks:mese_sand_tube_2 at (21735,14,23884)
compassgps writing settings
2018-08-02 06:37:39: ERROR[Main]: ServerError: AsyncErr: ServerThread::run Lua: OOM error from mod 'technic' in callback LuaABM::trigger(): not enough memory
2018-08-02 06:37:39: ERROR[Main]: Current Lua memory usage: 1626 MB


so 1626 MB is already much more than the other posts with 12 MB or 52 MB
but as I have 16 GB RAM all, and Linux is using less than 1 GB
its a shame cant use more yet ... without extra compiling movements ...

The max of used memory minus the 820 from Linux itself was yet near 6 GB, guess most map-data
Last edited by Festus1965 on Mon Aug 06, 2018 08:00, edited 1 time in total.
 

User avatar
DrFrankenstone
Member
 
Posts: 71
Joined: Tue May 24, 2016 05:36
Location: Melbourne, Australia
GitHub: treer

Re: Lua OOM errors

by DrFrankenstone » Fri Aug 03, 2018 03:08

I noticed this code in duane's Integral Tree mod, invoked at the start of its mapgen on_generated() function:

Code: Select all
    -- Deal with memory issues. This, of course, is supposed to be automatic.
    local mem = math.floor(collectgarbage("count")/1024)
    if mem > 500 then
        print("Integral is manually collecting garbage as memory use has exceeded 500K.")
        collectgarbage("collect")
    end

Can this sort of thing help? Should I be putting it in my mapgen mods before using voxelmanip?
 

User avatar
Beerholder
Member
 
Posts: 198
Joined: Wed Aug 03, 2016 20:23
GitHub: evrooije
In-game: Beerholder

Re: Lua OOM errors

by Beerholder » Fri Aug 03, 2018 09:33

Doing a full GC cycle might help yes, could clear the memory before the OOM ever occurs. But there is a bit of overhead in calling garbage collectors.

I am seriously wondering how effective it is in duane's case. The data array in the latest versions is a local outside the function (local data = {}), in previous versions it was local to the function (local data = vm:get_data()).

So in the latest version there will be a reference to the data array at all times and as such it will never be garbage collected. And I don't think vm:set_data(data) clears the data array in the end, correct?

Then again, maybe the VM data array was not what was causing memory issues for duane in the first place ... (or, it was not the only thing)

Anyways, let's hope we will have a stable LuaJIT 2.1 release soon so we can start to play around with GC64 mode :-) I have regularly been checking luajit.org for news, but alas the last beta is from already quite a while ago. It will solve the 2GB limit, but IMO that does not mean games and mods should not be memory efficient anymore ...... ;-)
 

User avatar
paramat
Developer
 
Posts: 3209
Joined: Sun Oct 28, 2012 00:05
Location: UK
GitHub: paramat
IRC: paramat

Re: Lua OOM errors

by paramat » Sun Aug 05, 2018 00:34

DrFrankenstone as far as i know Duane's garbage collection code is a personal hack that was used before i updated my advice in https://forum.minetest.net/viewtopic.php?f=18&t=16043 and before i posted the latest LVM example in https://forum.minetest.net/viewtopic.php?f=18&t=19836

If you code according to my latest optimisation advice i doubt garbage collection is needed, as the noise and voxelmanip tables (which are what use so much memory) are all reused.

Note that, as far as i know, the message 'Current Lua memory usage' does not show the memory usage that caused OOM, you can see this value is often very low and not near 2GB. Or maybe, OOM is something to do with memory addresses and not actual memory used.
 


Return to Modding Discussion



Who is online

Users browsing this forum: No registered users and 2 guests