[Mod] Debug [dbg]

Post Reply
User avatar
LMD
Member
Posts: 1386
Joined: Sat Apr 08, 2017 08:16
GitHub: appgurueu
IRC: appguru[eu]
In-game: LMD
Location: Germany
Contact:

[Mod] Debug [dbg]

by LMD » Post

Debug (dbg)

Debugging on steroids for Minetest mods

Motivation

Lua offers very powerful introspective facilities through its debug library, which unfortunately almost always go unused due to their clunky APIs.

Current state-of-the-art in Minetest mod debugging appears to be print/log/chat "debugging" using serialization or dump, all of which should be rendered obsolete by dbg.

API

Optionally depend on dbg in mod.conf to ensure that it is available for load-time debugging.

dbg()

Shorter alias for dbg.dd().

debug.debug() on steroids: Starts a REPL (read-eval-print-loop); equivalent to a breakpoint. Features include:
  • Full access to locals & upvalues of the calling function
  • Own local table _L where local debug vars get stored
  • Ability to enter expressions (rather than statements)
  • Continuation, which works the same as in the Lua REPL (+ empty lines working)
  • Pretty-printing using dbg.pp, including syntax highlighting
Enter cont to exit without an error. Use err to throw after error debugging sessions (dbg.error, dbg.assert).

Use EOF (<kbd>Ctrl + D</kbd> on Unix) to exit & shut down Minetest.

dbg.error(message, [​level])

Starts a debugging session at the given (optional) level, printing the error message.

dbg.assert(value, [​message])

Returns value. Starts an error debugging session if value is falsey. Error message is optional.

dbg.pp(...)

Pretty-prints the given vararg using the default parameters.

If the argument list of functions is unreliable (see dbg.getargs_reliable below), a question mark (?) will be appended to the argument list to indicate this.

dbg.ppp(params, ...)

Parameterized pretty-print. Requires a custom pretty-printer parameter table params:
  • write = function(str, token_type), where token_type is optional and may be one of nil, boolean, number, string, reference, function or type
  • upvalues = true, whether upvalues should be written
dbg.vars(level)

Returns a virtual variable table of locals & upvalues vars for the given stacklevel that supports the following operations:
  • Getting: vars.varname
  • Setting: vars.varname = value
  • Looping: for varname, value in vars() do ... end
dbg.locals(level)

Returns a virtual variable table of local values at the given stack level.

Locals include upvalues.

dbg.upvals(func)

func may be either a function or a stack level (including nil, which defaults to the stack level of the calling function).

Returns a virtual variable table of upvalues at the given stack level.

dbg.traceback(level)

Formats a stack trace starting at level. Similar to Lua's builtin debug.stacktrace, but shortens paths and accepts no message to prepend.

dbg.stackinfo(level)

Returns a list of info by repeatedly calling debug.getinfo starting with level and working down the stack.

dbg.getvararg(level)

Only available on LuaJIT; on PUC Lua 5.1, dbg.getvararg will be nil.

Returns the vararg at the given stack level.

dbg.getargs(func)

Function parameter list detection doesn't work properly on PUC Lua 5.1; unused params are lost and varargs are turned into arg. Use dbg.getargs_reliable (boolean) to check for reliability.

Returns a table containing the argument names of func in string form (example: {"x", "y", "z", "..."} for function(x, y, z, ...) end).

dbg.shorten_path(path)

Shortens path: If path is a subpath of a mod, it will be shortened to "<modname>:<subpath>".

Security

Debug deliberately exposes the unrestricted debug API globally, as well as the dbg wrapper API, both of which can be abused to exit the mod security sandbox.

Only use dbg in environments where you trust all enabled mods. Adding dbg to secure.trusted_mods (recommended) or disabling mod security (not recommended) is required.

The /lua chatcommand must explicitly be enabled on servers by setting secure.dbg.lua to true; if enabled, server owners risk unprivileged users gaining access through MITM attacks.

Usage

Prerequisites: LuaJIT and a terminal with decent ANSI support are highly recommended.

/dbg

Calls dbg() to start debugging in the console.

/lua <code>

Executes the code and pretty-prints the result(s) to chat. Only available in singleplayer for security reasons (risk of MITM attacks).

Links: GitHub, ContentDB, Minetest Forums

License: Written by Lars Müller and licensed under the MIT license (see License.txt).
My stuff: Projects - Mods - Website

User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Re: [Mod] Debug [dbg]

by Mr. Rar » Post

I like the idea of this but it did not work for me. I was investigating a crash so I put dbg() on the line just before the crashing line and the server just shutdown when it hit the line with dbg(). Nothing special was shown in the logs. I didn't test anything else.

My setup: Windows 10, Minetest 5.5, dbg in secure mods.
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
LMD
Member
Posts: 1386
Joined: Sat Apr 08, 2017 08:16
GitHub: appgurueu
IRC: appguru[eu]
In-game: LMD
Location: Germany
Contact:

Re: [Mod] Debug [dbg]

by LMD » Post

Mr. Rar wrote:
Tue Jul 12, 2022 14:29
I like the idea of this but it did not work for me. I was investigating a crash so I put dbg() on the line just before the crashing line and the server just shutdown when it hit the line with dbg(). Nothing special was shown in the logs. I didn't test anything else.

My setup: Windows 10, Minetest 5.5, dbg in secure mods.
Which version were you running? Did Minetest segfault ("just shutdown")? Did you have a console open? (You must have a console open to use dbg).
My stuff: Projects - Mods - Website

User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Re: [Mod] Debug [dbg]

by Mr. Rar » Post

>Which version were you running?
I'm not sure when I downloaded it so I re downloaded it and retested so this is the latest github as of 7/13/2022

>Did Minetest segfault ("just shutdown")?
It just shutdown. It said on the screen shutting down and also in the log that it was shutting down.

>Did you have a console open? (You must have a console open to use dbg).
I did not.

I tried:
minetest --terminal --server --world test

And got:
ERROR[Main]: Cmd arg --terminal passed, but compiled without ncurses. Ignoring.

That would be pretty inconvenient to have to run it as a server every time.
I am using Minetest downloaded from minetest.net
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
LMD
Member
Posts: 1386
Joined: Sat Apr 08, 2017 08:16
GitHub: appgurueu
IRC: appguru[eu]
In-game: LMD
Location: Germany
Contact:

Re: [Mod] Debug [dbg]

by LMD » Post

Mr. Rar wrote:
Wed Jul 13, 2022 17:36
>Which version were you running?
I'm not sure when I downloaded it so I re downloaded it and retested so this is the latest github as of 7/13/2022

>Did Minetest segfault ("just shutdown")?
It just shutdown. It said on the screen shutting down and also in the log that it was shutting down.

>Did you have a console open? (You must have a console open to use dbg).
I did not.
That explains it. Debug requires a console - otherwise it's useless. Doing this in chat is not possible since dbg has to block the main loop
Mr. Rar wrote:
Wed Jul 13, 2022 17:36
I tried:
minetest --terminal --server --world test

And got:
ERROR[Main]: Cmd arg --terminal passed, but compiled without ncurses. Ignoring.

That would be pretty inconvenient to have to run it as a server every time.
I am using Minetest downloaded from minetest.net
That's the way dbg is intended to be used: In a console. I actually prefer running a (test) server if possible, but if you want just singleplayer testing, simply starting minetest from the console should suffice. Why are you passing the --terminal flag though? I never had to use that. If you want to use it, get a better build that is compiled with ncurses support.
Last edited by LMD on Wed Jul 13, 2022 20:24, edited 1 time in total.
My stuff: Projects - Mods - Website

User avatar
Mr. Rar
Member
Posts: 109
Joined: Tue Oct 04, 2016 20:13
GitHub: MrRar
In-game: MrRar

Re: [Mod] Debug [dbg]

by Mr. Rar » Post

> Why are you passing the --terminal flag though?
I guessed that was the way of having Minetest in a console window. I don't know what I am doing.

I tried starting Minetest in Command Prompt and in PowerShell in both cases Minetest seamed to "pop out" in that no log messages from minetest were printed in the console and a new prompt was displayed before Minetest closed. The server shutdown when it got to gdb().
I tried running Minetest from MSYS2 terminal it did not pop out. However when it got to gdb() it did not crash or shutdown it printed one line from a stack traceback:
...Downloads\minetest-5.5.0-win64\bin\..\mods\edit\init.lua:42: in function defined at line 41

Line 42 had dbg()

So is this basically a Linux only thing?
"Only in Christ do we find real love, and the fullness of life. And so I invite you today to look to Christ." - St. John Paul II

User avatar
LMD
Member
Posts: 1386
Joined: Sat Apr 08, 2017 08:16
GitHub: appgurueu
IRC: appguru[eu]
In-game: LMD
Location: Germany
Contact:

Re: [Mod] Debug [dbg]

by LMD » Post

Mr. Rar wrote:
Wed Jul 13, 2022 19:06
So is this basically a Linux only thing?
No. It is a console-only thing that has only been tested on Linux. It only uses basic commands to read from stdin & write to stdout. I don't know how to properly run Minetest with a console on Windows, but in principle this mod should work on Windows too.
My stuff: Projects - Mods - Website

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: [Mod] Debug [dbg]

by Blockhead » Post

Mr. Rar, you were on the right track, --terminal is indeed the way you have to invoke Minetest when on Windows in order to get output to Command Prompt/PowerShell. Unless invoked with --terminal Minetest is what they call in Windows parlance a 'Windows application' as opposed to a 'Console application', and won't spawn a terminal when double-clicked in Explorer and (I think) won't show output to any command prompt that calls minetest.exe either. Input to stdin from your Command Prompt/PowerShell should work fine just fine by typing in the console that is running Minetest, and let you work with this mod and others that use the terminal.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
Mantar
Member
Posts: 584
Joined: Thu Oct 05, 2017 18:46
Contact:

Re: [Mod] Debug [dbg]

by Mantar » Post

Blockhead wrote:
Thu Jul 14, 2022 15:17
Unless invoked with --terminal Minetest is what they call in Windows parlance a 'Windows application' as opposed to a 'Console application', and won't spawn a terminal when double-clicked in Explorer and (I think) won't show output to any command prompt that calls minetest.exe either.
Smells like a marketing-imposed decision from back when they wanted to push the idea that Windows was not "just a shell over DOS" anymore. It clearly doesn't provide any technical benefit.
Lead dev of Exile, git repo: https://codeberg.org/Mantar/Exile

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: [Mod] Debug [dbg]

by Blockhead » Post

Mantar wrote:
Thu Jul 14, 2022 20:19
Smells like a marketing-imposed decision from back when they wanted to push the idea that Windows was not "just a shell over DOS" anymore. It clearly doesn't provide any technical benefit.
I think you're just being cynical. Most people just don't need a terminal attached to their process, and will just ignore it if it does exist or even think the program is not as nice for showing them one. So that's what most Windows programs do, they don't spawn a terminal window, and have to instead tell Windows they want to show a terminal window explicitly. That's what --terminal does, is tell win32 API to not hide the terminal. Seriously think about it as someone launching Minetest to play a game on Windows: Double-click it in explorer or a desktop icon - where's the terminal? Nowhere, you launched it from a graphical shell, why would it have a terminal? you launch Minetest on Linux there are plenty of ways to not end up with a terminal: Launch it from your desktop environment via a desktop or taskbar icon; via dmenu on i3. If you browse to where your Minetest binary lives and open it via a file manager, it depends on the manager: pcmanfm will ask you if you want a terminal or not; dolphin will not open a terminal.

Clearly Dolphin follows the same school of thought as Windows: Most people running a program don't need a terminal attached to it. If you get told you need terminal output, most users, whatever their operating system, can still be instructed to run it. Furthermore that's often not necessary for Minetest either when you have the log file.

None of this denigrates the terminal's usefulness or should reflect badly on terminal users. I'm an i3-wm user who launches heaps of terminals myself! Clearly the terminal is great for this debug mod. But you have your head screwed on backwards if you think every GUI application needs to spawn a terminal. Refusing to allow GUI applications to hide their terminal, now that would be backward thinking from a programmer who says "why don't people love the console like mee?". That's what I hear when I hear "no technical benefit". What technical benefit? The software works fine without a console!

Bonus points: Poor criticisms of Microsoft checklist
  • Marketing decides everything for some reason.
  • Still thinks in terms of DOS.
  • Linux somehow always implicitly technically superior for things of neutral technical merit.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
Mantar
Member
Posts: 584
Joined: Thu Oct 05, 2017 18:46
Contact:

Re: [Mod] Debug [dbg]

by Mantar » Post

Not opening a terminal is one thing, and is both understandable and fine, not sending any error messages to a terminal when the application was opened from one is another thing entirely, and there's no technical benefit for that to be the case, which is the thing I was actually talking about. It seems like an arbitrary dividing line was established between "windows application" and "console application."

Also I've spoken with several people who've worked for Microsoft over the years and I know damn well that marketing is a major factor overriding technical decisions from time to time, and many of the programming folks over there resent this sort of meddling from management.
Lead dev of Exile, git repo: https://codeberg.org/Mantar/Exile

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: [Mod] Debug [dbg]

by Blockhead » Post

Mantar wrote:
Fri Jul 15, 2022 18:38
not sending any error messages to a terminal when the application was opened from one is another thing entirely, and there's no technical benefit for that to be the case, which is the thing I was actually talking about.
Ah, well more judicious use of quoting or using italics in your quote would have established your narrow focus better. Yes, not sending output to the terminal that opened it seems pretty dumb to me too. One another side though, most Command Prompt users aren't used to using job control like Unix shells' & suffix. I know that's their problem, but Windows is full of allowances for dumb users, just not as many as MacOS. When it comes to windows, it's usual practice to log stuff and offer to send logs to the developers if you need help. Obviously that goes hand-in-hand with most of it all being proprietary, but if there's one silver lining it means people don't trust terminals to hold their logs, which they usually shouldn't.
Mantar wrote:
Fri Jul 15, 2022 18:38
It seems like an arbitrary dividing line was established between "windows application" and "console application."
It's now embedded into the win32 API, not that that's a good thing necessarily. It's quite literally not just a marketing thing though, even if it was totally their decision.
Mantar wrote:
Fri Jul 15, 2022 18:38
Also I've spoken with several people who've worked for Microsoft over the years and I know damn well that marketing is a major factor overriding technical decisions from time to time, and many of the programming folks over there resent this sort of meddling from management.
As is the way with many businesses sadly :( Marketing exec goes "what if I could wave a wand and all these terminals would go away? That'd entice more people into thinking our Windows NT isn't DOS-based" and some poor code grinder has to make it so.

Anyway counterpoint: would --no-terminal make sense? It'd be an inversion to the --terminal switch, which would override the default on Linux. Probably not sensible, it's better to just redirect stdout and stderr with the normal mechanisms.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
Mantar
Member
Posts: 584
Joined: Thu Oct 05, 2017 18:46
Contact:

Re: [Mod] Debug [dbg]

by Mantar » Post

Yeah I don't think --no-terminal would be very useful.

Oh, I forgot to say, this mod is fantastic. I can see this being very useful!
Lead dev of Exile, git repo: https://codeberg.org/Mantar/Exile

Post Reply

Who is online

Users browsing this forum: No registered users and 21 guests