Lua Threads for Computer Mod

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Lua Threads for Computer Mod

by electrodude512 » Post

Hi,

My friend just told me about Minetest a few days ago after I started writing my own Minecraft clone in C++ and Lua (after M****s*** bought Mojang). I immediately abandoned my project and switched to Minetest, and am very happy with it.

I'm interested in writing a ComputerCraft-type mod for Minetest. I plan to use digilines instead of peripheral cables, and computers will have external terminals and storage devices, similarly to RedPower's computers and unlike ComputerCraft's. Obviously, they will run (sandboxed) lua. Mesecons' luacontrollers are hard to use because you can't have libraries and such and you can't have loops or functions. This is why I want to make my own mod - in addition to the fact that I want more peripherals like hard and floppy drives and terminals with keyboard entry. I can easily handle the libraries and peripherals part, but just removing the loop and function disabling code from luacontrollers won't work - someone could run 'while true do end' and hold up the whole server indefinitely.

Is there any way for me to spawn my own Lua thread that will run indefinitely but independently from the main server thread? I suspect stuff from here would have to be implemented into the core for this to work, if it isn't already. I found minetest/builtin/async/init.lua and minetest/builtin/common/async_event.lua, but I couldn't figure out how to use them. It seems they can only be used by the main menu. Will they do what I want?

Maybe I could just give my sandboxed code a special loadstring that puts coroutine.yield()'s in appropriate places (inside non-tiny loops, in potentially recursive functions, etc)? This would probably involve converting the whole program into a tree first, which would be expensive and basically like compiling it twice.

Is it possible for me to spawn my own threads? I could write a C++ library and load that with Lua if necessary, but would that be necessary?

I am currently using version 0.4.10 from the Gentoo package manager. I plan to do a manual install so I can fetch updates with git soon, however.

Thanks,
electrodude

EDIT: edited stuff

User avatar
Neuromancer
Member
Posts: 958
Joined: Tue Jun 12, 2012 22:28
GitHub: Neuromancer56

Re: Lua Threads for Computer Mod

by Neuromancer » Post

I was going to work on a similar Mod a while back. There may be some things in the thread that you will find useful.

viewtopic.php?id=8399

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Re: Lua Threads for Computer Mod

by electrodude512 » Post

Thanks, but that still doesn't answer the question of how to run completely sandboxed lua code that can't take over the server. Lua Lanes looks useful, though - I'll have to look into it. I've never used LuaJIT and my install of MT has decided to use normal Lua, so I have no idea about compatibility but will have to look into that.

Also, I don't see turtles as being as important as computers if you have Technic's frames, so I don't plan to immediately add turtles. I will add them eventually - I'm not giving up my plan to basically implement Dwarf Fortress using turtles. Also, CC-style turtles conflict with my plan of computers having no terminals or harddrives - my turtles would basically be dumb non-programmable peripherals that normal computers can communicate with somehow. I plan to add an I/O expander and bundled cables (unless some other mod already has bundled cables that work similarly to Mesecons/Digilines cables). Normal computers might not have mesecons support, forcing you to use digilines, to simplify the API. However, I'll probably do something I've always wanted ComputerCraft to do - make it so modders could embed standard computers in their own nodes, entities, items, etc. That way, turtles could be a separate mod.

A mod can include a C library (with require() or something), right? I really don't want to have to do that, but I might have to. Maybe I could convince the MT devs to put something smallish into the official MT codebase that will allow something like what I want (asynchronous but interacting threads), with a github PR or something.

Thanks,
electrodude

P.S. I'm completely astonished there is no good computer mod for MT yet. MT's been around for years and is written in Lua. One of the first mods I planned to write for the above-mentioned minecraft clone I started but abandoned was a computer mod.

Zeno
Member
Posts: 140
Joined: Sun Jun 29, 2014 03:36
GitHub: Zeno-
Location: Australia

Re: Lua Threads for Computer Mod

by Zeno » Post

electrodude512 wrote:A mod can include a C library (with require() or something), right? I really don't want to have to do that, but I might have to.
Yes (although as you said it might be best not to). A very simple example without error checking or much else. After compiling you'll have to make sure mttestc.so is in your path.

Code: Select all

/* gcc -Wall -shared -fPIC -o mttestc.so test.c */

#include <stdio.h>  /* Just for basic testing (e.g. printf()) */
#include <lua.h>
#include <lauxlib.h>

#define LUA_NAMESPACE "test"
#define MODNAME "[testc] "

#if LUA_VERSION_NUM > 501
#   define luaL_reg luaL_Reg
#endif

struct Params {
    int a, b;
};

static int l_testfn(lua_State *L);

/* Private functions */
static int parseargs(lua_State *L, struct Params *args);


const struct luaL_reg lua_funcs[] = {
    { "testfn",      l_testfn },
    { NULL, NULL}
};

/* Entry point to the module */
int luaopen_mttestc(lua_State *L)
{
    const luaL_reg *f;

    luaL_newmetatable(L, LUA_NAMESPACE);
    lua_pushstring(L, "__index");
    lua_newtable(L);

    /* "install" functions into metatable */
    for (f = lua_funcs; f->func != NULL && f->name != NULL; f++) {
        lua_pushcfunction(L, f->func);
        lua_setfield(L, -2, f->name);
    }

    return 1;
}

/* "Lua Prototype": int testfn(a, b); --> a*b */
static int l_testfn(lua_State *L)
{
    struct Params args;

    if (!parseargs(L, &args)) {
        lua_pushstring(L, MODNAME"Error parsing settings");
        lua_error(L);
    }

    printf("************** FROM C args: %d, %d\n", args.a, args.b);

    lua_pushnumber(L, args.a * args.b);	// Push a*b onto stack (return value)

    return 1;
}

static int parseargs(lua_State *L, struct Params *args)
{
    args->a = luaL_checknumber(L, 1);
    args->b = luaL_checknumber(L, 2);

    return 1;
}
Compile the above with and make a Lua mod with init.lua as...

Code: Select all

local MYCMOD = require("mttestc")
print("Something:" .. MYCMOD.testfn(5, 6))

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Re: Lua Threads for Computer Mod

by electrodude512 » Post

Thanks! That'll be really helpful if I end up making a C library.

What are the chances of a github PR I might submit getting merged into the official minetest core? It would basically add a few functions to spawn new lua threads and control them in basic ways. I can see this as having a very low chance of happening, because anything part of a mod isn't going to take over the server and can use coroutines if it wants to take a long time.

Does anyone know of a good way to make a custom fixed-width character display and keyboard event listener, for the screen, in a formspec? Is this feasible? Would just using lots of images be really slow? Would that be updatable? Would lots of labels work? How could I get individual keystrokes with that? Could I abuse a HUD? Or is the only way to make a core mod PR or a C library (I doubt a C library could do this) that adds keystroke detectors to formspecs or, for that matter, anywhere?

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Re: Lua Threads for Computer Mod

by electrodude512 » Post

I can't get it to find your example C library. It compiles fine, but it gives this error when I try to play the game. It gives the same error if I change the name of the library to look for to something that doesn't exist, so it must not be finding it.

Here's the lua code:

Code: Select all

local path = minetest.get_modpath("computertest").."/threads.so"
print("Loading threads library at "..path)
local threads = require(path)  -- line 9 in error message
And here's the error:

Code: Select all

Loading threads library at /home/<username>/src/minetest/bin/../mods/computertest/threads.so
12:13:55: ERROR[main]: ========== ERROR FROM LUA ===========
12:13:55: ERROR[main]: Failed to load and run script from 
12:13:55: ERROR[main]: /home/<username>/src/minetest/bin/../mods/computertest/init.lua:
12:13:55: ERROR[main]: caught (...)
12:13:55: ERROR[main]: stack traceback:
12:13:55: ERROR[main]:  [C]: in function 'require'
12:13:55: ERROR[main]:  ...<username>/src/minetest/bin/../mods/computertest/init.lua:9: in main chunk
12:13:55: ERROR[main]: ======= END OF ERROR FROM LUA ========
12:13:55: ERROR[main]: Server: Failed to load and run /home/<username>/src/minetest/bin/../mods/computertest/init.lua
12:13:55: ERROR[main]: ModError: ModError: Failed to load and run /home/<username>/src/minetest/bin/../mods/computertest/init.lua
Do you have any idea what's wrong?

Thanks,
electrodude

User avatar
Neuromancer
Member
Posts: 958
Joined: Tue Jun 12, 2012 22:28
GitHub: Neuromancer56

Re: Lua Threads for Computer Mod

by Neuromancer » Post

Would you consider making your mod compatible with the blockly add on for computer craft?
http://blockly-lua.appspot.com/static/a ... /index.htm

I see the computer craft mod as a gateway to teaching beginners lua. They start with blockly lua using the computer mod. Move on to lua in computer craft, and then writing Mods for Minetest. What I like about minetest is that with the API already in lua, a Minetest computer craft mod is potentially much more powerful because it can access the entire API. For example it is difficult to detect water in MC Computercraft, but that would be trivial in the Minetest version.

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

You dont load libraries like that anymore.

Minetest is using LuaJIT. LuaJIT includes a thing called LuaFFI.

LuaFFI allows writting mods in C language and calling external libraries.

See: http://luajit.org/ext_ffi.html

I was going to write a computer emulator for minetest, but other more pressing matters hindered my advancement.

http://piumarta.com/software/lib6502/

Unfortunately this library is unix-like only.

http://6502.org/source/monitors/ehbasic/ehbasic.html

This is a basic (very advanced for a 8bit) for 6502.

My idea was to load the 6502 emulator library using LuaFFI and then run enhbasic inside the emulated computer.

Feel free to advance the idea if you like.

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Re: Lua Threads for Computer Mod

by electrodude512 » Post

Neuromancer wrote:Would you consider making your mod compatible with the blockly add on for computer craft?
http://blockly-lua.appspot.com/static/a ... /index.htm

I see the computer craft mod as a gateway to teaching beginners lua. They start with blockly lua using the computer mod. Move on to lua in computer craft, and then writing Mods for Minetest.
That link is dead. Also, it would have to be done under a compatibilty layer, as my computer mod is going to be very, very different from ComputerCraft at many different levels, especially the lower levels. If someone else wrote the compatibilty layer, I might include it in my mod. I doubt I'll have the time or interest to do it myself, but I might - I have a lot of ComputerCraft code I don't really feel like porting.
Neuromancer wrote:What I like about minetest is that with the API already in lua, a Minetest computer craft mod is potentially much more powerful because it can access the entire API. For example it is difficult to detect water in MC Computercraft, but that would be trivial in the Minetest version.
It's hard to detect water in MC ComputerCraft because Dan200 made it hard. It would be pretty easy for him to add this capability, he just doesn't want to. Also, a Minetest computer won't be able to directly access any of the Minetest API (except in special cases for server admins only), or people would be able to trivially destroy a whole server with just a few for loops, not to mention be able to take over the server computer and delete or steal all of the server owner's files.
aldobr wrote:You dont load libraries like that anymore.

Minetest is using LuaJIT. LuaJIT includes a thing called LuaFFI.

LuaFFI allows writting mods in C language and calling external libraries.

See: http://luajit.org/ext_ffi.html
Is this merely the recommended way or actually the only way?

The Minetest that I built manually from git uses LuaJIT, but the version that the Gentoo package manager gave me uses normal Lua. Is there some way that will work for both Luas? Also, FFI probably won't work, because I need a reference to the lua state so I can lua_xmove stuff between multiple server and computer threads. FFI will only work if Lua is set up to be threadsafe, otherwise I'll have to use lua_xmove and normal libraries. Lua isn't set up to be threadsafe by default but Minetest might set it up right, and I have no idea about LuaJIT's threadsafeness.

User avatar
Neuromancer
Member
Posts: 958
Joined: Tue Jun 12, 2012 22:28
GitHub: Neuromancer56

Re: Lua Threads for Computer Mod

by Neuromancer » Post


Zeno
Member
Posts: 140
Joined: Sun Jun 29, 2014 03:36
GitHub: Zeno-
Location: Australia

Re: Lua Threads for Computer Mod

by Zeno » Post

electrodude512 wrote:I can't get it to find your example C library. It compiles fine, but it gives this error when I try to play the game. It gives the same error if I change the name of the library to look for to something that doesn't exist, so it must not be finding it.

Do you have any idea what's wrong?

Thanks,
electrodude
If I put the compiled .so in my home directory it works fine. (I'm using LuaJIT as well)

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

Is this merely the recommended way or actually the only way?
There is no recommended way. But doing it other way makes no sense.
The Minetest that I built manually from git uses LuaJIT, but the version that the Gentoo package manager gave me uses normal Lua.
If i remember correctly, theres no such thing as using normal lua or luajit. You simply replace the library. Anyway, modern versions of minetest (0.4.10) are distributed with luajit.
Is there some way that will work for both Luas? Also, FFI probably won't work, because I need a reference to the lua state so I can lua_xmove stuff between multiple server and computer threads. FFI will only work if Lua is set up to be threadsafe, otherwise I'll have to use lua_xmove and normal libraries. Lua isn't set up to be threadsafe by default but Minetest might set it up right, and I have no idea about LuaJIT's threadsafeness.
If you add a symlink or simply ship your mod with lua lib (possibly renaming it) the OS will think that they are different libraries and link them twice to the runtime image. This can avoid any threadsafeness problem. Besides that, lua interpreter and the code being FFIied would run on the same thread. So your question makes no sense at all.

Zeno
Member
Posts: 140
Joined: Sun Jun 29, 2014 03:36
GitHub: Zeno-
Location: Australia

Re: Lua Threads for Computer Mod

by Zeno » Post

aldobr wrote:
Is this merely the recommended way or actually the only way?
There is no recommended way. But doing it other way makes no sense.
Which method does the minetest source code use? I'll give you a hint... the traditional way.

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

WTF are you talking about ?

Do you really understand what a FFI is ?

I will give you a clue, its not used to embbed lua inside an application. It has nothing to do with that.

FFI means FOREIGN FUNCTION INTERFACE. What it is used for ? To make calling outside code (libraries) from inside a language (in that case, Lua).

So, what that has to do with minetest calling lua api ? A-B-S-O-L-U-T-E-L-Y NOTHING.

C++ calls lua api by the commong C++ way of calling libraries. Its the Lua environment that needs a FFI because it would be pretty hard to translate from a generic c/c++ library to the lua interpreter context and vice versa.

What you said, "how minetest does it ?" can be interpreted in two ways :

1 - That you think that minetest, when it need to call an external library, calls lua code, and that lua code calls an external library (by external, i mean dynamically loaded). That makes absolutely no sense.
2 - That you dont know what an FFI is and think that its a way to call Lua from C/C++ applications. Again makes no sense at all.

LuaFFI makes the bridge between the lua interpreted/jitted code and external, dynamically loaded libraries. Its not used - nor makes sense - to interface between minetest host code in C++ and lua code (besides some corner cases, like when you write a lua callback).

But what that has to do with everything discussed in this thread ?

What you are intending to do is calling lua from lua. As the minetest host C++ code wont provide you an API to create arbitrary sandboxed environments where to rn lua code, you will need to directly call lua52.dll/liblua.so. This means that you will need a way to interact with foreign code. Well. FFI was created exactly to do such kind of things (calling an external dynamic library). Its a special case becouse the DLL being called is lua dll itself. So you might be afraid about thread safeness in the lua library, because dlls/so are linked at runtime to the same memory space as the executable. Code is shared across multiple instances of a library on most OSes, but data is not. Data is created for each instance of a library and mapped into the executable runtime. But only once per process, if the library does not explicit use threadvars. So i answered that this is not usually a problem, becouse the whole lua interpreter in minetest runs on a single thread. Theres no lua multithreading in minetest. Thats why there are no mod race conditions, becouse all code is executed sequentially. What that does mean to the special case of loading lua dll from inside lua ? Besides the possible reentrancy problem (i dont know if luajit allows this), the threadsafess is guaranteed, becouse even the FFI code (including the called dll functions) run on the same thread as the LuaJIT itself.

Got it ?
Last edited by aldobr on Fri Sep 26, 2014 03:41, edited 2 times in total.

Zeno
Member
Posts: 140
Joined: Sun Jun 29, 2014 03:36
GitHub: Zeno-
Location: Australia

Re: Lua Threads for Computer Mod

by Zeno » Post

aldobr wrote: I will give you a clue, its not used to embbed lua inside an application. It has nothing to do with that.
You realise that embedding and extending are essentially the same thing, right?

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

Dude, i did a favour to you by enlightening you about LuaFFI.

Instead of learning you are now resorting to nitpicking on dictionary definitions. But, anyway, i lose nothing by ignoring you.

There are some people that cannot be wrong.

Zeno
Member
Posts: 140
Joined: Sun Jun 29, 2014 03:36
GitHub: Zeno-
Location: Australia

Re: Lua Threads for Computer Mod

by Zeno » Post

aldobr wrote:Dude, i did a favour to you by enlightening you about LuaFFI.

Instead of learning you are now resorting to nitpicking on dictionary definitions. But, anyway, i lose nothing by ignoring you.

There are some people that cannot be wrong.
No, I'm able to be wrong. I already knew about LuaFFI but thanks for bring it up in the discussion, it's worthwhile.

The main problem with LuaJIT, from my point of view, is that is doens't play nicely with valgrind (without recompiling LuaJIT with some extra flags that should only be used for debugging). Therefore for debugging minetest I (and I expect others, but I don't know) occassionally build minetest without LuaJIT and fallback onto "plain" Lua. So I'm saying that expecting or relying on LuaFFI being available is probably not a great plan. Of course if it's for a mod that only you will be using then the point is moot.

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

So now you understood how luajit/luaffi works ?

Good that i did not write all those paragraphs for nothing.

Well, i wont discuss your nitpicking. Its up to you to embrace (in your projects) the tech or not. Be sure that others might very well be less restrictive and use LuaFFI, and have a much faster interpreter than yours.

Edit:

https://github.com/jgrahamc/lulip/

Lulip, lua profiler.

electrodude512
Member
Posts: 39
Joined: Wed Sep 17, 2014 21:34
GitHub: electrodude
IRC: electrodude512
In-game: electrodude512
Location: USA

Re: Lua Threads for Computer Mod

by electrodude512 » Post

I'm not really sure who any of that was to, me or Zeno.

I got my library to load. I was passing "threads.so" to require, when I should have been passing just "threads". I made a function that passes the lua state to lua_newstate, lua_xmove's the function on top of the stack of the old state onto the top of the stack of the new state, and passes the new state to a new pthread, which lua_pcall's the function. Under plain LuaJIT outside of Minetest, it works for some time until it segfaults, but my actual computer mod segfaults immediately inside Minetest with LuaJIT. Obviously, any segfaults are unacceptable. I'll either try to PR minetest on github to fix concurrency not working, or I'll just have a separate lua interpreter to run the computer code.

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

LuaJIT is more strict. It enforces the semantic rules of the language while the non-jit variety is more permissive with erroneous code.

prestidigitator
Member
Posts: 647
Joined: Thu Feb 21, 2013 23:54

Re: Lua Threads for Computer Mod

by prestidigitator » Post

There might be a middle ground between using an external library to gain threads and including entire parse trees. Have you considered using debug.sethook() to force a coroutine.yield() every so many instructions/lines? For a real sandbox you'd probably also need to replace many of the coroutine and debug functions (not to mention many other standard API functions) during the "sub-interpreter's" execution to do something intelligent, of course. Not sure how paranoid you really want to be.

User avatar
ninnghazad
Member
Posts: 36
Joined: Fri Sep 26, 2014 12:33
GitHub: ninnghazad
IRC: ninnghazad
In-game: ninnghazad

Re: Lua Threads for Computer Mod

by ninnghazad » Post

thought about the matter, for the same reasons it seems. i just started coding something without thinking too much about it,
and after a weekend have ended up with kind of a demo or proof-of-concept-mod.
there is a computer-node, a drone-entity (turtle) and a drone-spawner-node.
the drone and the computer have ugly guis (that don't automatically update/redraw themselfes yet...) and there isn't much functionality yet.
however using a in-lua-sandbox-approach with coroutines the systems when activated spawn a sandboxed coroutine,
run a bootstrap in it, include some rudimentary APIs and drop to a "shell". there are a few commands and the beginnings of a few CC-like APIs.
the drone can move and interact with inventories. the systems each get a directory and io/fs/os function have to be wrapped to chroot users to it.
theres a ROM directory which holds APIs and premade programs.
commands a user typed run in a very restricted environment, APIs and programs have a wider environment.

issues i have come across/things to do:
  • - much work to wrap standard lua5.1 functions like getfenv/setfenv to be available in userspace and be safe to use, ie not allow environment escalation
    - lots of functions just cannot easily be wrapped and will have to be unavailable (debug?!)
    - nodes can't move, so drones have to be entities. entities don't seem to have inventories. using detached inventories works, but they don't seem to get saved.
    - guis/formspecs are ... cumbersome. formspecs don't provide for fast redraws on changing data and don't provide character-based input - as far as i've seen.
    - ComputerCraft isn't just "nodes that run lua scripts", its a lot of APIs and functions - to have similar functionality those would have to be recreated, or at least most of them - especially io,fs,parallel and so on
    - a good command-parser for the GUI-shell is needed, fancy features like tab-completion may fail because of lack of support on the side of formspecs.
    - how peripherals should work and how to implement them proper - are they own entities to be attached() to the turtle, or are they inventory items that get wielded?!
    - allow direct access to other nodes' functions to allow drones/computers to control other blocks - is this feasible or does every foreign block have to provide special hooks for us to use?
    - tons more i can't think of right now
it should be possible to get a similar mod like computercraft with reasonable effort, but some differences.
i don't think a 1-1 clone is easily achievable and isn't what we should go for,
rather something that can run most of the old scripts with little or no change.
i tried writing it in pure lua, but as i could not find any good fs functions in minetest's lua i had to include LFS,
which works when provided as a .so/.dll in the mods directory, and is needed to provide chroot-functionality.

about the yielding - firstoff computercraft has a bunch of problems with it too, and long calls often end up in trouble on heavily loaded servers.
i find it works well to just spread around yields in the apis and the functions a user will have to call - and doing something like
debug.sethook(co, coroutine.yield, "", 100)
to prepare against endless loops users may produce.

btw: of course i wanted to call it computertest... hmpf, well its called dronetest now ;)



edit:
debug.sethook(co, coroutine.yield, "", 100) won't work with pcall/xpcall in version of lua minetest uses, so
pcall(function() while true do end end) would still mean whole server locks. meh...

User avatar
aldobr
Member
Posts: 316
Joined: Sun Nov 25, 2012 05:46

Re: Lua Threads for Computer Mod

by aldobr » Post

1 - Detach computer from the turtle. The computer doesnot need to move.
2 - Its possible to "move" (sort of) a node, but is it trully needed ?
3 - Store the program on the node metadata.
4 - Create a per user periferial bus. Each periferial gets an address that can be set on the periferial formspec. CPU addresses those periferial via the per-user bus and adress (Ex.: "aldo:123123") Each periferic is a node.
5 - You dont need threads. You simply start as many lua51.dlls as need (one per node). This wont overwhelm memory. Dll will only be loaded once by the os. Data will be copied-on-write on most systems. This means that each node placed in the world will have his own lua context.
6 - Computer should run as ABM, else it will sit idle on a unloaded chunk. Every time a ABM hits, some code is executed.
7 - Give it a basic set of functions (ROM) that enables it to access minetest resources. Dont give it OS access, its too risky.
8 - Use a big text box for text output and a small (below the big one) text box for command input. Dont try to do it using character based logic. It should be done line by line, not char by char.

Etc.

Thats my two cents.

prestidigitator
Member
Posts: 647
Joined: Thu Feb 21, 2013 23:54

Re: Lua Threads for Computer Mod

by prestidigitator » Post

ninnghazad wrote:edit:
debug.sethook(co, coroutine.yield, "", 100) won't work with pcall/xpcall in version of lua minetest uses, so
pcall(function() while true do end end) would still mean whole server locks. meh...
Hmm. Yeah. I did some testing and searching myself, and it looks like the issue is that you can't yield from a debug hook at all in 5.1 (but you apparently can in 5.2). If you try you get a, "attempt to yield across metamethod/C-call boundary," error thrown from coroutine.yield() (and the error isn't reported unless you explicitly do so where you call coroutine.resume()):

Code: Select all

Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> function foo() while true do print("Hello"); end; end
> t = coroutine.create(foo)
> debug.sethook(t, coroutine.yield, "", 10)
> function r() s, e = coroutine.resume(t); if not s then print("ERROR: "..e); end; end
> r()
Hello
Hello
ERROR: attempt to yield across metamethod/C-call boundary
Bummer.

EDIT: Never mind about 5.2. Despite people posting in various places that you can do it in 5.2, I get the exact same error (except without "metamethod/" in the message) shown above when running a 5.2 interpreter.

EDIT: This might help, but it's actually a patch to Lua, so it goes far beyond simply wiring a mod or a loadable Lua extension: http://coco.luajit.org/

User avatar
ninnghazad
Member
Posts: 36
Joined: Fri Sep 26, 2014 12:33
GitHub: ninnghazad
IRC: ninnghazad
In-game: ninnghazad

Re: Lua Threads for Computer Mod

by ninnghazad » Post

@prestidigitator
coco is integrated in luajit since 1.x, which is what minetest uses unless you force it not to at compiletime. that doesn't help much as it seems.
however, i have found a way to catch infinite loops in userprovided code without patching minetest, or using anything external. it goes roughly like this:

local save_env = {} -- put functions that are safe for users here
local function error_handler(msg) print("ERROR: "..msg) end
local function yieldkill() coroutine.yield() end -- hackish: will not actually yield but kill the coroutine
local function hang() while true do end end
jit.off(hang,true) -- this disables JIT for the coroutine's function, enabling sethook to work on hang()
setfenv(hang,save_env)
local co = coroutine.create(function() xpcall(hang,error_handler) end)
debug.sethook(co,yieldkill,"",1000000)
coroutine.resume(co)

so the trick is that jit.off will enable the debug.sethook to work on tight infinite loops - it would never fire with jit.on - in combination with the fact that yieldkill will produce an error (like cannot cross c-call-bounds or so on). the effect is that when the user calls something he wrote that contains an infinite loop, sethook will hook after a million instructions, try to yield, fail and return to xpcall, firing its error handler.
so even if it is ugly as hell, it does exactly what i need, and seems to do so consistently.
however, even if jit.off's second argument specifies recursive functionality, i still have to test in how far that goes through more pcalls - it prolly won't go through load* do* functions - but the userspace-versions of these would have to be specially wrapped and secured anyways.

@aldobr
computers are nodes and drones are entities, that works quite well so far. i know i *could* move nodes, but entities are meant for moving, and have nice moveto(). that part is good as it is i think - drones can already do the dance like CC turtles, with the rotating and so on. getting nodes to spin on the other hand...

right now i have all systems resumed in a globalstep hook, and plan on adding force loading to them - while i did not know that ABMs worked that way, i want the systems that are active to always be in an active chunk. otherwise they may be trying to run a program that makes them interact with neighbouring blocks, and they aren't loaded. or so i imagine.

the ROM way is how i am doing it, however there is misunderstanding, by os i did not mean lua's os api, but rather something like CC has - it is a clone of that api (minus a few methods) that works the same way, but acts upon the virtual system and not the real OS. so os.execute would ie just execute something lua on a drone, not the server.

thats the way i have input gui organized right now - however i really want the charbased input, and make the virtual systems handle it (with a input.readline or smth for the user). without it, a lot of stuff isnt possible - the first thing that comes to mind is the texteditor. CC obviously has charbased input, and without it a lot of the cool community-scripts for it wouldn't work.
also one has to press a redraw button right now to update gui and see the results of whatever it is one did. while i have enough ideas to try and make this work, im not sure any of them will work. i pondered bout using the HUD functions instead of the formspecs, but that is just... ah, meh... u know.

oh and about the threads, i don't even want em. i want it small and simple and without changes to minetest (that wouldn't have to happen anyways). spawning coroutines within the minetest's luajit-thread is just what i want.
the problem just was, maybe is, that i super don't want userspace code to hang the whole server.

thanks for the input guys!

oh, btw: https://github.com/ninnghazad/dronetest


EDIT:
for the above method to work, make sure that debug.sethook is called before every resume - otherwise all long running programs will yieldkill, even if they yielded correctly. with multiple sethooks each coroutine.yield will reset the counter.

Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests