Pain points when using Minetest

[rubenwardy]

When developing mods and games for Minetest, I've noticed a few things which make the development a little painful. I'd like Minetest to be easy as possible to make good, quality games. I asked on Twitter and Discord[1] to see what problems other people had, too

Documentation

• lua_api.txt is hard to read, badly formatted, too long, too hard to search. Often contains imprecise documentation
• the wiki is very easy to search for, but is out of dated and inaccurate. A suggestion was more examples should be added
  

  for the development of mods, I find that the wiki dev lacks examples. it might need more content like the modding book. When you start, it was important to have some examples and not only a documentation so that you could study it, see even fork this.

Missing Features

Obviously a big one, but missing features results in workarounds being needed

• Per-node and per-item texture
• Inability to customise anything related to the client or rendering

Bad APIs

APIs which are feel badly designed

• Anything callback related tends to suck
• Bone handling
• GUI/Formspecs
• Texture caching makes use of texture modifiers impractical
• Detached inventories always feel painful to implement.
• Registering items and such feels quite painful and verbose

Storage

Efficient storage is still painful, especially when targeting multiplayer. Mod storage isn't good to MBs of data.
It would be good to have a sandbox safe API for storage of non-relational records, ie: arrays of structs.


-----------------

[1] - Unofficial Minetest Discord - #dev - 2019-01-12 19:23

[sorcerykid]

These are all very good points. Thanks for taking the time to compile them. For me, the four biggest impairments are

• the inability to change formspec elements dynamically
• the inability to push new media to the client on the fly
• the lack of any sort of custom hotkey bindings for mods
• no tools to maintain, repair, or analyze the map database

Sadly there are no workarounds for any of these that I've found. They either can't be done at all, or they require a significant kludge to implement. The fourth one, I'm currently working on a Lua-based solution, so that might be resolved soon. But the others would require engine and client-server protocol changes.

Please don't take these as complaints. Just wanted to share some ideas since the subject was brought up :)

[salahzar]

From my (small) experience in creating small mods for education, I found that the formspec would need a bit of enhancement. Some syntactical sugar would be good for newcomers and children willing to make simple interfaces, and some ways to help in connecting forms with their replies would be also welcome (maybe putting replies as callback declaration inside the formspec itself?).

Also the chat window has some areas it can be enhanced: clickable links as in minecraft would be interesting to allow for sending information on external videos or material on internet and copy/paste is not working. Or is it?

[sorcerykid]

You're in luck. Session-based Formspec event callbacks are available with ActiveFormspecs mod. And copy and pasting links from chat can be accompllshed with Chat History mod (you can even specifically search for all posted links using command "/c http")

[rubenwardy]

You're in luck. Session-based Formspec event callbacks are available with ActiveFormspecs mod. And copy and pasting links from chat can be accompllshed with Chat History mod (you can even specifically search for all posted links using command "/c http")

ActiveFormspecs is a dirty mod which adds functions where it shouldn't. It also doesn't add callbacks per-element, but just one per-formspec - ie: it combines showing and callbacks for you

[Skulls]

There needs to be an easy way to use the global step / on step for time based things such as mobs. Right now every mob mod that I've seen has a massive amount of code in their on-step function for handling movement or entity detection so a lot of redundant work being done each step. I suggest a more entity component based system so that some optimization tricks can be done behind the curtains.

Groups are a fantastic tool but have very bad documentation.
Punch, use, secondary use, dig, and special need to be explained.

[sorcerykid]

I'm not sure why you think there is a requirement to have callbacks for every element. How that is a major detractor from using my mod? The person requested form-specific granularity, which I provided. Talk about nit-picking.

The official formspec API is inherently insecure, completely stateless, and permits mods to examine and manipulate the form fields of other mods. Is that somehow preferable? ActiveFormspecs was designed to override and extend the official API (without excessive code restructuring), to overcome these known security problems, most of which have been well documented by sofar.

I also disagree with your claim that my mod is "dirty". It's not uncommon to extend global namespaces in Lua (just look at the Lua docs for examples). Even Minetest engages in this exact same practice, as builtin adds countless functions like "string.split", "table.indexof", "table.copy", "string.trim", "math.hypot", and "math.sign". But I guess it's acceptable to "pollute" namespaces, so long as it's the core developers doing it.

You're being hyper-critical about a WIP mod that at least attempts to solve an issue. It's reminiscent of the knee-jerk reaction of core devs as soon as I released Auth Redux, ripping apart months of my hard work in the public IRC channel because they didn't "approve" of an alternative solution to a long-standing bug in the engine -- even making baseless claims that my implementation was inherently flawed and would even "become useless". Who says that kind of thing?

http://irc.minetest.net/minetest-hub/20 ... #i_5350190

Of course the vile anti-LGBT attack by a core dev on #dev-minetest proclaiming that I look "emo" and "creepy" and my real name is apparently not Leslie was extremely unprofessional. What kind of open source developer team gangs up to publicly single-out, degrade, and humiliate community volunteers behind their back that want to help improve a project? It's no wonder more people don't contribute given these kinds of callous, mean spirited attitudes.



In conclusion....

I think I'll add bullying, antagonism, and negativity to my "pain points when using Minetest".

[sofar]

Your avatar at the time looked to me like the Joker in the batman series. As in, I perceived it to display a disturbed horror persona that blows up hospitals. Nothing anti LGBTQ about it. You fantasized it to be anti LGBTQ. You have no evidence or knowledge of what I think of, or how I treat people who are LGBTQ, black, immigrants, republican or democrat, smart, breathing or dead.

If you keep up this perception of injustice, you will find it everywhere.

[rubenwardy]

I'm not sure why you think there is a requirement to have callbacks for every element. How that is a major detractor from using my mod? The person requested form-specific granularity, which I provided. Talk about nit-picking.

I read salahzar's request as wanting to be able to have callbacks for specific actions, but now I see that that is not the case, so I'm sorry for that

I also disagree with your claim that my mod is "dirty". It's not uncommon to extend global namespaces in Lua (just look at the Lua docs for examples). Even Minetest engages in this exact same practice, as builtin adds countless functions like "string.split", "table.indexof", "table.copy", "string.trim", "math.hypot", and "math.sign". But I guess it's acceptable to "pollute" namespaces, so long as it's the core developers doing it.

The issue with writing an API which adds functions into minetest. is that mods may break if minetest adds such a function later on. It's also less clear where the code is defined, ie: a mod writer may be confused and not depend on the mod.

The above is also sort of the case with Minetest adding things to table and math, I admit. Lua comes with an intentionally small standard lib, so Minetest extends that to be more useful to developers. The difference is that Lua is a fixed version, and also fairly stable in changes. Minetest's additions also don't form part of the dependency system, ie: builtin always loads first, so there's no risk of missing out a dependency.

You're being hyper-critical about a WIP mod that at least attempts to solve an issue.

This is true, it's a fairly minor problem I mentioned, really. The mod is certainly better than the current one, and would make a good addition to the API

It's reminiscent of the knee-jerk reaction of core devs as soon as I released Auth Redux, ripping apart months of my hard work in the public IRC channel because they didn't "approve" of an alternative solution to a long-standing bug in the engine -- even making baseless claims that my implementation was inherently flawed and would even "become useless". Who says that kind of thing?

Auth Redux is a very impressive mod which I do have a lot of respect for the implementation attempts, but it's using a nonstandard storage form which has several disadvantages: including less available tools, less documentation, a lot more code which can result in bugs being introduced, and also a heavy dependence on a single mod author, ie: you. This doesn't mean that the mod isn't good nor useful. Surely core developers should still be allowed to express their opinions about this solution?

Of course the vile anti-LGBT attack by a core dev on #dev-minetest proclaiming that I look "emo" and "creepy" and my real name is apparently not Leslie was extremely unprofessional. What kind of open source developer team gangs up to publicly single-out, degrade, and humiliate community volunteers behind their back that want to help improve a project? It's no wonder more people don't contribute given these kinds of callous, mean spirited attitudes.

I was not around at this time because it was during my exam season. Attacks like this are abhorrent, especially in a public developer channel, and really should not be tolerated

Nothing anti LGBTQ about it

I think it's more the context that makes it anti-LGBT, rather than the intention. There's still the fact of such an action in a public channel

[sofar]

Your avatar at the time looked to me ..

@sorcerykid

At the time I made those statements, they were absolutely an insult. I was extremely frustrated with our interactions in the linked thread. As a person, I lost it, and I realized it too, hence the last line in the linked image where I realized I should have ... just walked away a lot earlier.

That is obviously not an excuse, and I should have not made the comments, period.

My goal is for you to be a supportive and more importantly, a supported member of the community, and my statements at that time completely derailed that, and I regret that. I regard you as an extremely competent programmer, and I envy your ability to produce high quality additions to the MT world that are ingenious and well polished.

I hope you're willing to work with me, and even Rubenwardy, who could have also made some of the criticism at the `fsc` implementation I made as well. My first instinct is to fix or address the concerns, and I'm still interested in working on solutions for that with you, if you share that sentiment.

In the past I've written on "respect" vs "merit", and with the insult I made, I violated goals I set myself years back on how to treat others in OSS communities, badly. I understand it'll be difficult to regain your respect, and I need to work hard on that. (if you're interested: https://plus.google.com/+AukeKok/posts/PJpYYnQtQWV )

[Skulls]

Well that escalated quickly.

[Andrey01]

Missing features and API possibilities (i have whole list of this below):

• Making wielded item volumeous is impossible (wished)
• Impossible to make reflections of node e.g. real mirrors.
• Impossible to apply to inventory item animation.
• Impossible to attach object to node (modders can write themselves, however it`s to be wished this possibility to exist in Minetest API itself.
• Entities can not have more one collision box (i also think selection box should be added separately as in nodebox drawtype of node.
• Bad rendering of models from blender that have mesh drawtype in some cases.
• Minetest should have own (good) Mob API and as i think mobs in minetest_game should be, also add possibility to enable/disable their separately.
• More callbacks as minetest.register_on_point_thing, minetest.register_on_player_step, minetest.register_on_wield_item and minetest.register_on_press_key.
  
  
  **Clarification:
  register_on_point_thing will be called always when player is pointing at something (gets node at which he pointed) (necessary);
  register_on_player_step will be called when player is walking, it will get a node that he is going on, so you can know the player is going actually or falling down;
  register_on_wield_item will be called when a player is changing wielded item (necessary)
  register_on_press_key will return key name in that moment when it is pressed (necessary).

[Andrey01]

Also other missing API possibilities:

• Impossible to set up a point from which light will emit in node/entity (and impossible to set few light points);
• Impossible to set separate scale of an image of an inventory item (visual_scale parameter). Currently it will change size of node itself that is put in-world.

[Wuzzy]

As for me, here is my list of serious bugs (some of them very old):
viewtopic.php?f=6&t=15084

Other points (non-bugs):

The usability in general is pretty horrible. If you do not know Minetest very well already, it's easy to get lost.
The main menu is very ugly and feels very technical, not like part of a game. IMO the main menu needs a complete facelift and redesign.
The GUI in general is also very technical and sterile. It doesn't look like a proper game. The most pressing issue for me, however, is that Minetest uses the wrong font.

Some controls cannot be re-bound to new keys. For example, mining is hardcoded to the left mouse button.

No tutorial.

the wiki is very easy to search for, but is out of dated and inaccurate. A suggestion was more examples should be added

Are you referring to the dev wiki? Then that's true. I think the dev wiki should be deleted because it's redundant and not really official documentation. Also, the many outdated pages or outright incorrect pages do more harm than good.

Instead, the dev docs should be concentrated in ONE canonical place, which is always kept up to date. It should go without saying that no feature should be accepted if it doesn't update the docs in the same breath, but I believe you already enforce this. Instead, the official documentation should be improved. IMO wikis are terrible for writing technical documentation, there's a reason we have stuff like Doxygen.

[1] - Unofficial Minetest Discord - #dev - 2019-01-12 19:23

So you're secretly moving dev talk to a proprietary service? Nice. ;-)

Anything callback related tends to suck
Bone handling
Registering items and such feels quite painful and verbose

I don't understand the problems but whatever. Those aren't problems for me, at least. It would be nice if you elaborated on these things.

GUI/Formspecs
Detached inventories always feel painful to implement.

Yes.
To me, the biggest problem with formspecs are inconsistent and poorly documented formspec coordinates. There is not a single, consistent coordinate system which is used by all formspec elements. It seems there are at least two different coordinate systems, which are not even documented, and this sucks badly.



EDIT:
Oh, almost forgot this:

It seems like Minetest 5.0.0 will almost certainly ship the mod translation system without the essential toolchain to maintain and update the localization files. This will make it hard to keep translations maintained for large projects like MineClone 2, unless someone rolls out their own toolchain.

Oh, and the fact that mod translations will roll out their own obscure file format instead of PO. :-( https://github.com/minetest/minetest/issues/7290

Also, this hilarious bug: https://github.com/minetest/minetest/issues/6724

There are no inventories for entities: https://github.com/minetest/minetest/issues/6081

Player physics override is !!!FUN!!!. You can only overwrite the physics, not modify them. This makes implementing something like potions extra-!!!FUN!!!. The PR to fix this is still not being accepted. https://github.com/minetest/minetest/pull/7269

[rubenwardy]

So you're secretly moving dev talk to a proprietary service? Nice. ;-)

No official development discussion has ever taken place on there, it's a channel for user discussion about the development of the engine. Not dev talk at all

[Wuzzy]

No official development discussion has ever taken place on there, it's a channel for user discussion about the development of the engine. Not dev talk at all

Heh. My snarky comment was just a tongue-in-cheek-kind-of-thing anyway. :P

[Astrobe]

the wiki is very easy to search for, but is out of dated and inaccurate. A suggestion was more examples should be added

Are you referring to the dev wiki? Then that's true. I think the dev wiki should be deleted because it's redundant and not really official documentation. Also, the many outdated pages or outright incorrect pages do more harm than good.

Instead, the dev docs should be concentrated in ONE canonical place, which is always kept up to date. It should go without saying that no feature should be accepted if it doesn't update the docs in the same breath, but I believe you already enforce this. Instead, the official documentation should be improved. IMO wikis are terrible for writing technical documentation, there's a reason we have stuff like Doxygen.

I have trouble understanding this point of view considering that lua_api.txt is itself inaccurate in many places. Just today I had to use Rubenwardy's book to figure out the details of register_chatcommand(). And look, it is in the same sorry state in 5.0.0.

Devs don't have the time to update it, granted. But also understand that there are far less chances to receive contributions that fix documentation issues if you have to clone Minetest (and the chances are really zero if the person doesn't have a GitHub/GitLab account already) to submit a pull request than if all it takes is to register to a wiki and just edit it. Wikis are the path of least resistance.

Delete lua_api.txt and support fully the dev wiki. Or delete both and put the documentation in the main wiki. You'll increase your chances to get more modders.

[Sokomine]

I'm quite comftable with lua_api.txt, as long as it's kept current. Better have one place with correct information than multiple, user-friendly ones with outdated or incorrect information.

My personal issues would probably be:

• Formspecs. Something may be happening there already. What I miss is a way to simulate old point&click adventures where a mob might offer you some options how to talk to him. Making something like that visually appealing and readable with sufficient flexibility on the chat options currently doesn't really work.
• Entities in not loaded areas. This isn't a bug or a missing feature as such. Just something that arises from the way MT works. I.e. carts entering areas where there's nobody to see (and thus keep the map loaded), mobs walking around on the map - in general: Entities that have a purpose that's not very short-term.
• Libraries. Curerntly there's not much point in writing libraries as nobody besides the author is going to use them. Libraries make it more difficult to get the mods installed by potential users.
• A way to handle global texture changes on the fly. This was mentionned here as well. Could be very useful for handling seasons. Or playerskins added without server restart.
• Larger view range. Standing in front of a mountain and having to walk towards it in order to see it is a bit annoying and destroys immersion. The world feels so tiny sometimes. There might not be a good solution for that.

Some of these issues may already be under development. I didn't have much time recently and did miss most recent developments.

[rubenwardy]

It's very important to have a version-controlled API reference, so that documentation can be updated in PRs as part of adding a feature

[Linuxdirk]

In no particular order.

• Formspecs are a pain to use. Weird syntax, completely unflexible, inconsistent, not properly stylable, limited possibility to dynamically react on user input, basic UI features (audible/visual feedback for click actions, scrollable areas, tabs, etc.) are hard to implement (if at all) and are very unintuitive to use.
• We have settingtypes.txt but there is no standard way that mods have access to the values set there as default values if the value is not set in minetest.conf.
• The uber bloated default mod most mods have to depend on because default not only provides some relevant APIs but also some settings, nodes, and functions that otherwise won’t be available.
• It is impossible for mod authors to register functions that then can be mapped to keys by the user.
• There is no sane way to register and parse chat command parameters.
• There is no way to dynamically change node features like textures, breakability, sounds, groups, etc. because most node features are hardcoded to the node on registration of the node instead of being node meta data.
• No proper error handling for mods. And no: simply crashing the server and throwing a “Lua exception” is not proper error handling.

This list is to be seen as incomplete.

[rubenwardy]

• There is no sane way to register and parse chat command parameters.

I'm planning on adapting this into an engine API once I add support for generating help automatically: https://github.com/rubenwardy/chatcmdbuilder https://rubenwardy.com/minetest_modding ... mplex.html
I also don't like the anonymous function method of registration

I agree with all of your points

[Linuxdirk]

I'm planning on adapting this into an engine API once I add support for generating help automatically: […]

Absolutely looking forward to this since it is a must-have for creating chat commands that are slightly more complex than 1-2 different parameters.

[Astrobe]

It's very important to have a version-controlled API reference, so that documentation can be updated in PRs as part of adding a feature

To be accurate, what matters to the end user is that the documentation they read is valid for the minimal version they are targeting. The rest is nerd programmer stuff ;-)

Wiki pages have a history log that you can even tag (it's apparently not enabled on Minetest wikis, but I've seen the feature in other MediaWiki-based wikis). It can be almost the same thing as a git log.

Maybe the devs should just care about Doxygen documentation and let others fill the wiki based on this? Then merge the player/end-user and the scripting API wiki so that people only need to create one account to contribute to the project.

If you can make it so that mod creators put their player and dev documentation on your all-in-one wiki, then you increase your chances that they keep the engine's API up to date "en passant" by an order of magnitude.

Here are two examples of how one can put Wikis to good use:

Oolite Wiki
Dwarf fortress Wiki for version management and article quality.

[Wuzzy]

I 100% agree that it is critically important to have a version-controlled API reference. Wikis are completely unsuited for critical dev documentation, also because it is not kept in sync with the source code at all. Their history log is completely independent.
Also, you can't have documentation for multiple versions of Minetest in a wiki, only “latest” at best, unless you deal with a billion of complicated and painful-to-maintain MediaWiki plugins or extreme redundancy.
Also: It's a wiki. Which means that everyone with an account can destroy the work of others with a mouse click. It's very easy to introduce errors, whether intentional or not. Wiki edits also completely bypass the usual Minetest QA process. Wikis also tend to degrade fast in up-to-datedness. This is a direct result of the fact they're not in sync with the source code.

Also, I think it is good practice to force devs to update the documentation for every new feature they add, so that undocumented features don't make it into Minetest in the first place.

The lua_api.txt does it's job and it would be foolish to remove it from the repository. It just needs more readability, but wikis are NOT the solution.

[runs]

Yes, Minetest documentation is poor. The software developers, in general, forget to make suitable documentation, cos they prefer to code and code.

An example of an impressive documentation system: https://wiki.archlinux.org It's not only for that distro but for all Linux. You can find there all the necessary to maintain Linux and its world.

The Minecraft Wiki is cool too from a player perspective.

It's not only about a doc system but "obligate" the developers to write it.