[Mod] QA-Block [qa_block]

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

[Mod] QA-Block [qa_block]

by bell07 » Post

This is a developer helper mod, allow run any lua code for testing reason. The mod can list and run lua-scripts placed in checks subfolder. some check scrips provided.
A second future is a browser trough lua variables/global tables.

Image
Image

Features
  • redirection of print() output to minetest chat. (can be disabled by changing code line "print_to_chat")
  • robust call of the scripts trough "pcall" does not crash the game in case of syntax- or runtime errors
  • all functionality available trough chat commands and the QA-Block node
  • refresh and list the checks script list at runtime
  • edit the code before calling them
  • type code and run them (with smartfs only)
  • browser trough lua variables/global tables.
  • edit the lua variables
Dependencies
  • smartfs - https://github.com/minetest-mods/smartfs
    GUI for check selection and manipulation. Optional, but without smartfs there is limited functionality available.
    smartfs is provided again (my version) because of missed futures in upstream
Available check modules
  • empty - Empty file for your own checks
  • global_variables - List suspicious global variables
  • is_ground_content - This checker lists all nodes for which is_ground_content == true
  • list_entities - Lists all the registered entities (except builtin)
  • list_spawning_mobs - List entities that are mobs from mobs_redo or compatible framework
  • no_item_description - Lists all items without description
  • same_recipe - Find duplicate crafting recipes
  • unobtainable_items - Lists items which seem to be unobtainable
  • useless_items - Lists all items which are probably useless
  • get_items_csv - Write a csv-file from minetest.registered_items to the world folder that can be pivot-analyzed in a spreadsheet like localc or excel

How to use:
  • add the mod to the game you like to test
Using chat command
  • /qa help - print available chat commands
  • /qa ls - list all available check modules
  • /qa ui - display and run check using the selection dialog (smartfs only)
  • /qa checkname - run check
  • /qa set checkname - set default check
  • /qa - run default check
Using the block
  • get the QA-Block from creative inventory
  • place the block somewhere
  • - without smartfs - wait till the default check is finished and the block disappears
  • - with smartfs - start the check using selection dialog
In all cases - check the debug.txt for test results

License GPL-V3: https://www.gnu.org/licenses/gpl-3.0.html
Github: https://github.com/bell07/minetest-qa_block

Credits: Wuzzy, thanks for ideas, code optimizations and the most check scripts
Attachments
screenshot_20170121_011613.png
screenshot_20170121_011613.png (156.96 KiB) Viewed 2387 times
screenshot_20170121_012152.png
screenshot_20170121_012152.png (377.13 KiB) Viewed 2387 times
Last edited by bell07 on Wed Aug 02, 2017 11:46, edited 10 times in total.

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

Interesting idea, although I think if the mod would include more checking scripts, it will become more useful.
Just puking the entire script output into chat does not really seem useful to me. Even your screenshot shows that.
Although it is useful later to extract it from the log files. But not when using Minetest.
I have an idea: Post any script output in a separate formspec window. Honestly I am not sure if this is a good idea.

The list_spawning_mobs module has an error: If the mobs variable is not present, the module will show a runtime error.

I have ideas for some other scripts:
- List all global variables (important to hunt down “suspicious” variables which should never be global)
- List all items/nodes/tools/craftitems
- List all recipes

I would also like to have an option to disable redirection of print() at runtime. I suppose there are situations even for developers where this redirection may cause trouble. I have not tested it but maybe this could interfere with the luacmd mod.

The default dependency should be removed no matter what. This is a development tool after all, it should not hard depend on any mod. Or the dependency should at least be optional. Ideally, just draw a simple texture (the default texture is confusing anyways). Sounds are not that important.

Oh, and depends.txt does not support comments. If a dependency is not present, just don't put it there. Everything else will just confuse Minetest.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Thank you for your remarks. The main goal of this mod was previously to get a way to run lua code snippets without restart the MT on each change and that does not crash on syntax errors. It was my way to learn lua and minetest-API. But now it "might" be a collection of helpers code snippets that can anyone need for quality checks. So all code snippets welcome! Even if one line check!

The check output should be read from debug.txt or from console. Usually I start the MT trough the command line in windowed mode so I can work on scripts and read the output without leaving MT. I updated the screenshot to reflect this.
I did not found any way to redirect the check output only, the redirection print ALL messages to the chat. So I see no way to redirect the print messages (from a check run) to something like a formspec. The current redirection to the chat is just for fun, I like the background noise in chat that where I can see if something happens. If you don't like the noise in chat it or if it interferes with any other mod, just disable them in the source code.
The list_spawning_mobs module has an error: If the mobs variable is not present, the module will show a runtime error.
fixed
List all global variables (important to hunt down “suspicious” variables which should never be global)
developed. The other ideas you need more specify or just paste the code snippets for them ;)
I would also like to have an option to disable redirection of print() at runtime
Please disable it in source code if you don't need it.
The default dependency should be removed no matter what.
Dependency removed. An own png added for the tool.

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

Thank you very much! With the globals check, this becomes very useful. If I find the time, I will contribute additional checkes for this block. I can recall that some people around these forums posted some checking snippets, it might be useful to gather these.
The more checker scripts are added, the more useful the node will become.

The globals checker still shows many globals even if I use this node in the Empty subgame (subgame without any mods) with only qa_block and origin mod installed.
I figured that some globals are part of the Lua API and can be safely blacklisted:

Code: Select all

	dump2 = true,
	PerlinNoise = true,
	PerlinNoiseMap = true,
	VoxelManip = true,
	VoxelArea = true,
	AreaStore = true,
	SecureRandom = true,
	PcgRandom = true,
	PseudoRandom = true,
	ItemStack = true,
	Settings = true,
However, even after clearing this up, the globals check still shows many globals, namely:

Code: Select all

	cleanup_path
	gcinfo
	on_placenode
	hack_nodes
	file_exists
	nodeupdate
	check_attached_node
	drop_attached_node
	newproxy
	get_last_folder
	spawn_falling_node
	on_dignode
	basic_dump
	nodeupdate_single
	INIT
	DIR_DELIM
	PLATFORM
Those are mostly functions. The all-caps variables are strings. I have checked, those variables seem to be leaked into the global namespace by builtin and they are not documented in lua_api.txt. I am not sure if those should be blacklisted (also, I have not checked every single name) since this is clearly a violation of the Minetest Lua styleguide (“Avoid globals like the plague.”). I think I have to post a Minetest issue about this soon to make sure that builtin does not leak so many globals.

I think this just shows that your mod is pretty cool already, although it has its rough edges. Just shortly after you added the globals check, I already found a style problem in builtin. :-)

I have tweaked the output code of the globals check a bit and applied some pretty-printing:

Code: Select all

local first = true
for name, val in pairs(_G) do
	if minetest.get_modpath(name) then
--		print(name, "mod exists")
	elseif blacklist[name] then     -- skip blacklisted
--		print(name, "builtin")
	else
		if first then
			print("Suspicious global variables:")
			first = false
		end
		local t = type(val)
		if t == "number" or t == "string" or t == "boolean" then
			local sval = dump(val)
			if string.len(sval) < 64 then
				print(name .. ": type \""..t.."\", value: " .. sval)
			else
				print(name .. ": type \""..t.."\", long value")
			end
		else
			print(name .. ": type \""..t.."\"")
		end
	end
end
if first then
	print("No suspicious global variables found.")
end
Feel free to use it in your mod. :-)

Oh, your qa_block mod leaks one global variable itself: It is old_print in init.lua. Please make it local.


Unrelated sidenode: I suggest to add the group “dig_immediate=2” for the node, to make sure it can always be dug, even without pickaxe.

Another unrelated sidenode: Why are you using multiple arguments in print()? The strings will appear concatenated in Minetest's chat output, which makes it hard to read. :-/

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

I added a second blacklist for the "maybe bug in MT builtin" globals and set them enabled by default

Code: Select all

local enable_second_blacklist = true
The second blacklist can be disabled to check the MT-core/builtins, but for usual modder this globals should not spam the output. So I can see next time if the qa_block leaks any globals oO.

I added your code to the check and updated the block as proposed.
Why are you using multiple arguments in print()?
The main reason is I do not need to check if the variables are "nil" that I like to print. Concatenating works with strings only. Nice effect, the output is tabbed in case of multiple variables. And as I said, the chat output is just noise for me so I did not seen the issue. Do you know a way to get tabbed output in the chat? So I can adjust the redirection.

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

I have written 7 new checkers for this mod. See here for more information:
https://github.com/bell07/minetest-qa_block/pull/2

Hints: If you don't know the type of a variable, but still want to print it, you can always shove it through tostring first. Also look up Minetest's dump function. In strings you can add a horizontal tab with “\t”.

PS: Don't forget to update the dependencies in the first post.

PPS: What exactly is this smartfs thing and why may this be needed?

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Pull request merged, and your "filesys" change too.
If you don't know the type of a variable, but still want to print it, you can always shove it through tostring first.
Mostly I use the "print" for debugging output, and "print(a,b)" is smarter for me as "print(tostring(a)..tostring(b))"
In strings you can add a horizontal tab with “\t”.
I know and use it in the print redirection. But "\t" does nothing in chat :/
What exactly is this smartfs thing and why may this be needed?
Smartfs is a framework for easy creation of formspecs: viewtopic.php?f=11&t=7553&p=116028
I did some enhancements on them primary for my "townchest" mod, tested the enhancement with qa_block and it seems to be usable. Agree the framework is oversized for qa_block needs. The in qa_block used version is not in upstream currently: https://github.com/minetest-mods/smartfs/pull/12. rubenwardy does the hard job to get my "it should work"-changes upstream ready.
If you interested you can have a look to my last enhanced version in the "view" branch: https://github.com/bell07/minetest-smartfs/tree/views. Just uncomment the "example-views.lua" in init.lua and try the /sfs_v and /sfs_t chat commands ;)
The smartfs dependency is optional in qa_block, but without them you have the chat-command only to select and run scripts. The block just calls the "defaultmodule" if placed and then removes the block if done. It's really a "light" version. I do not want to miss the selection screen any more (that can be called with /qa_block sel" too).

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Today I updated the mod description to the current git version. You could say it is a new release ;) Thanks to Wuzzy for many contributions in this version!

By the way I have a question: which way should I prefer to handle the used smartfs lib. All needed changes are merged to upstream last days and for now I can go two ways with qa_block and smartfs
1: As is: qa_block provide a copy of smartfs library.
Pro: There is no dependency defined to other mod because all needed is provided. Provided version is in a known working version.
Contra: If any other mod uses smartfs the library is loaded twice.

2: remove the copy and set a dependency to smartfs library mod.
Pro: Proper way, no redundancy. This way was intended originally. A fallback mode (without gui) is implemented already in chat commands
Contra: Additional dependency, maybe not tested or working combination to smartfs version in feature possible (in theory each smartfs change needs a qa_block retest)

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

I'd go for option 2. This is the proper way to do it, IMO this is the whole point of providing a library.
But please make rubenwardy promise not to break the smartfs API too often. ;-)

Formspecs in Minetest themvelves are still very much subject to change anyway, so either way you have maintenance work. :-/

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Ok, I did it. For now there is no smartfs copy provided. If you like the gui, you need to install the https://github.com/minetest-mods/smartfs mod in the current git version (Nov, 13, 2016 or later)
To get more functionality without smartfs I added /qa help and /qa set <modname> chat commands. /qa sel is renamed to /qa ui.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

The smartfs is provided again :-| I enhanced the QA-Block and used new futures that are not in upstream.

For now the QA-Block does have a Lua-Tables Explorer. See screenshot in updated first post.

Byakuren
Member
Posts: 818
Joined: Tue Apr 14, 2015 01:59
GitHub: raymoo
IRC: Hijiri
In-game: Raymoo + Clownpiece

Re: [Mod] QA-Block [qa_block]

by Byakuren » Post

newproxy is an undocumented part of Lua by the way, not from Minetest.
Every time a mod API is left undocumented, a koala dies.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

I do not see a relation qa-block to newproxy oO
by the way if I trust google, the newproxy is deprecated. Still present in lua-5.1 that is used in minetest, but is removed from lua-5.2.

Byakuren
Member
Posts: 818
Joined: Tue Apr 14, 2015 01:59
GitHub: raymoo
IRC: Hijiri
In-game: Raymoo + Clownpiece

Re: [Mod] QA-Block [qa_block]

by Byakuren » Post

Wuzzy mentioned it as an unaccounted global.
Every time a mod API is left undocumented, a koala dies.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Thanks for the hint. I moved the newproxy from "to clarify"-Blacklist to the "clarified"-Blacklist inside the check script.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Mod updated:
- The globals/lua tables browser does sort the variables now
- Search/Filter field added to the globals/lua tables browser
- Check light_source_15 removed, since MT-0.4.15 correct the values by self and therefore the list is always empty
- New check: get_items_csv: Write a csv-file from minetest.registered_items to the world folder that can be pivot-analyzed in a spreadsheet like localc or excel. Item groups expanded in the list.

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

The new features sound awesome, I would have suggested them at one point or another.

Sadly, just by putting a QA block from the creative inventory into my player inventory, Minetest fails:
2017-02-19 19:25:53: WARNING[Server]: Undeclared global variable "arg" accessed at /home/wuzzy/.minetest/mods/qa_block/init.lua:57
2017-02-19 19:25:53: ERROR[Main]: ServerError: Lua: Runtime error from mod '??' in callback detached_inventory_OnTake(): /home/wuzzy/.minetest/mods/qa_block/init.lua:57: bad argument #1 to 'ipairs' (table expected, got nil)
Tested with Minetest 0.4.14 on MineClone 2 0.10.0 and qa_block commit ID 02a5e4b96c741326619990c18d67a6c4777d86de.

The fact that there's an undeclared variable warning makes me believe it's not caused by MineClone 2.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Hi Wuzzy,

I know about the bug report, but I do not see the root reason for the issue and therefore no solution. :/ "arg" is a lua-builtin table and should be allways there, defined by "..." in function interface. Maybe any one other in forum know the issue and the solution? I cannot reproduce the issue.

As a workaround you can disable the nice-to-have only functionality "print-to-chat" till the solution is found. Just change the line 2 in "init.lua" to

Code: Select all

local print_to_chat = false

Byakuren
Member
Posts: 818
Joined: Tue Apr 14, 2015 01:59
GitHub: raymoo
IRC: Hijiri
In-game: Raymoo + Clownpiece

Re: [Mod] QA-Block [qa_block]

by Byakuren » Post

bell07 wrote:Hi Wuzzy,

I know about the bug report, but I do not see the root reason for the issue and therefore no solution. :/ "arg" is a lua-builtin table and should be allways there, defined by "..." in function interface. Maybe any one other in forum know the issue and the solution? I cannot reproduce the issue.

As a workaround you can disable the nice-to-have only functionality "print-to-chat" till the solution is found. Just change the line 2 in "init.lua" to

Code: Select all

local print_to_chat = false

Code: Select all

ipairs({...})
-- instead of
ipairs(arg)
Every time a mod API is left undocumented, a koala dies.

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Thanks!

In addition I found the http://stackoverflow.com/a/7630202. It seems the "arg" syntax is a relict of Lua-5.0 and does not work right on lua-5.1. In Lua 5.1 the new syntax should be used as proposed.

I changed it in git. @Wuzzy, can you please test if the change does fix the issue for you?

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Small update, both on globals explorer page:
- form will not be closed on enter in search field. Take the filter instead
- If the qa_block node is used, the path to the selected variable is saved to the nodemeta. So it is possible to come back to specific variable even after restart the game. Just come back to the placed node.

EDIT: 19.03.
- store more data to nodemeta (selected tab, function-hide in explorer)
- Globals-Explorer: search string per node - more intuitive. Search strings are stored now too

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

Added script graphviz_recipes_item.lua that generate a recipes paths in dot format that can be visualized using graphviz.

smart_inventory required to run because its items cache and recipes database is used.

Code: Select all

dot -Tjpeg recipesdefault_cobble.dot -o recipes_cobble.jpeg
Image
Attachments
recipes_cobble.jpeg
recipes_cobble.jpeg (107.44 KiB) Viewed 2387 times

User avatar
Wuzzy
Member
Posts: 4778
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Mod] QA-Block [qa_block]

by Wuzzy » Post

This mod is becoming better and better. :-)

User avatar
azekill_DIABLO
Member
Posts: 7507
Joined: Wed Oct 29, 2014 20:05
GitHub: azekillDIABLO
In-game: azekill_DIABLO
Location: OMICRON
Contact:

Re: [Mod] QA-Block [qa_block]

by azekill_DIABLO » Post

bell07 wrote: Image
wow it's seems very complicated on a graphical representation. I love this feature!
Gone, but not dead. Contact me on discord: azekill_DIABLO#6565
DMs are always open if you want to get in touch!

bell07
Member
Posts: 604
Joined: Sun Sep 04, 2016 15:15
GitHub: bell07

Re: [Mod] QA-Block [qa_block]

by bell07 » Post

It is the default output from graphviz. The check generates the input "dot" file like

Code: Select all

digraph G {
rankdir=LR;
default_cobble -> default_pick_stone;
default_cobble -> default_shovel_stone;
default_cobble -> default_furnace;
....

Post Reply

Who is online

Users browsing this forum: No registered users and 11 guests