Minetest GUIs, The Past, Present, and Future: An Overview

For people working on the C++ code.
Post Reply
User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Minetest GUIs, The Past, Present, and Future: An Overview

by v-rob » Post

If you want to know what's going on with GUIs, look at the later posts in this thread written by me. I'd rather not repeat everything in this first post. And, I suppose these posts in this thread can be considered my official-ish evolving roadmap.

Most important things to look at:
Lua-based GUI/HUD overview: viewtopic.php?p=382529#p382529
New UI API prototype: viewtopic.php?p=382974#p382974
Formspecs have gone through a lot of changes lately. What with proper coordinates, advanced element styling, fixing ancient bugs and limitations, and many other new features, using formspecs isn't even close to the same as it was just a few versions ago. As for me, I've been right in the middle of all of it (All I meant to do was to make real_coordinates[], but that's morphed into a lot more). This has been the work of multiple people, especially Rubenwardy, SmallJoker (Krock), Desour (DS-Minetest), Df458 (I don't know a forum username, if any), and myself, plus other contributors.

The purpose of this thread is to list of what has been done with formspecs, what is currently being worked on and discussed, and what the future looks like. It's important for users to know what is going on and why.

The Past

Organized in topical sections, here is a list of what has been done in recent history following my entry into formspec work as far as I can remember. This list is does not seek to give large descriptions; see each associated pull request for more information. Also, it is not comprehensive, but tell me if I forgot anything major.

Styling
#8383: The original PR adding style[] and style_type[]
#9380: Multiple element styling with comma separation
#9718: Universal formspec style selector *
#8916: Original hovered button styling
#9090: Internal button refactoring with consistent styling of different button types
#9378: Styling state selection with :hovered and :pressed.
#9187: Proper StaticText styling
#9290: 9-slice button backgrounds
#9661: Content offset and padding for buttons
#9763 and #10203: Font styling
#8678: Box styling

New Elements and Features
#8680: hypertext[]
#9258 and #9411: animated_image[]
#8660: 9-slice backgrounds with background9[]
#8996: Extra bgcolor[] features
#9101: scroll_container[]
#8530: More options for scrollbar use with scrollbaroptions[]
#9353: Set initial element focus with set_focus[]
#9496: Allow indexed dropdown events

General Improvements
#8524: Improvement of the coordinate system with real_coordinates[]
#8740, #9121, and #9262: Real formspec elements with proper draw order and clipping
#9534: Non-interactive element clickthrough
#8652: Item image clipping
#8507: Scrollbar variable bar size
#8665: Text cursor for fields and textareas
#10144: Optimize formspec size to stay within screen

The Present

In this section, current PRs for improving Minetest will be listed as far as I know. Descriptions are given because these things are currently being worked on now, and discussions (preferably in the GitHub pull request) are encouraged. If you want some of these features, help out by testing and bugging the core devs :)

Priority numbers for each feature are also given next to the PR number based on my opinion, mostly for the benefit of any core devs reading this topic (0 is highest priority, 4 is lowest). The idea is to get the more important PRs merged to pave the way for important newer ones as well as to get requested features merged.

Styling
#10259: Prio. 1 to allow selector combination
This PR allows for grouping elements together with a group[] element so that they can be styled together as a group like CSS classes. In addition, it allows naming elements that have no name in the element definition for named styling. Finally, it removes the cumbersome difference between style[] and style_type[] with a new selector.

#10291: Prio. 1 to unify buttons
This adds two new style state selectors: :focused to style the last interacted with element and :disabled to style an uninteractable element, made as such with set_disabled[]

#10265: Prio. 3, requested feature
This allows image[], animated_image[] and image_button[] fgimg properties to have a 9-slice image like background9[]

#10083: Prio. 3, requested feature
This gives the ability to style inventory list slot size and spacing with styling.

#9818: Prio. 1 to unify buttons One approval
This PR colorizes button bgimgs automatically based on the bgcolor property to better facilitate prepends.

New Elements and Features
#8679: Prio. 2, highly popular feature
This PR adds a new feature that allows highly advanced key and mouse events to allow advanced custom behaviour in formspecs.

#9755: Prio. 3, requested feature
Gives many textual elements horizontal and vertical text alignment as well as giving labels the ability to be sized for clipping and the aforementioned alignment

#10106: Prio. 1 to unify buttons
This gives buttons the ability to be toggleable similar to a checkbox and exposes the :checked state to style it.

#10261: Prio. 2, highly popular feature One approval
This relatively new PR makes a new element, mesh[], which allows displaying a 3D object in a formspec with optional rotation and mouse control.

#10262: Prio. 3, requested feature
This PR gives no_prepend[] the ability to remove specific elements from the prepend string so some will be kept but not others.

General Improvements
#10260: Prio. 1 for style scoping
This PR defines formspec syntax more strongly by giving a unified end[] element for all containers. In addition, it gives sized containers for better clipping and other future features.

#10095: Prio. 4, performance
This performance PR doesn't draw elements that are totally clipped; good for scroll containers.

#10222: Prio. 0, refactoring!
This PR has no user-facing features, but it cleans up part of the internal formspec code, mainly inventory lists.

The Future

Well, this is a lot already, but we aren't even close to done. This last section is my personal roadmap for formspecs in a somewhat haphazard format. This list is by no means in chronological order. What is done first is directly based on what current PRs get merged first (hopefully the priority 1 PRs as they are necessary prerequisites) and what I consider to be best to do before the others. So here it is:

Styling
  1. Upon merging all the Prio. 1 to unify buttons PRs, then two things will happen:
    1. The button code will be refactored to remove useless code, to fix up the other code, and to bring the style up to date with Minetest's code guidelines. This isn't a problem for Irrlicht updates because a) Irrlicht isn't updating and b) the button code is already drastically changed from the original.
    2. Styling will be added to other elements (woo!), probably scrollbars first since we already have the code and then fields since they are important. This will necessitate a new selector to select individual parts of an element, like the CSS pseudo-element selectors. Refactoring of those elements' code will follow. Every other element will have styling added in turn.
  2. After #10260 (unified container end[] element) is merged, two things will happen:
    1. Styling and other customization elements will be scoped to the current container. In addition, element names/groups will have namespaces in the events if the elements are in a named container to prevent name bleeding.
    2. A child selector, i.e. CSS >, will be added
  3. Finally, after #10259 (groups and new style selectors) is merged, selector combination will be added for even more advanced styling.
  4. I am planning to make styling animations (like CSS transition). This is a later style issue to deal with.
  5. This last point is a bit frivolous, but I am planning (at some point) to make fancy effects like blurring and noise filters for boxes, but this is not nearly as important as other points.
New Elements and Features
  1. After points 1, 1a, and 2 under Styling have been completed, button containers (issue #9754) will be added.
  2. table[] and the closely-related textlist[] will be completely refactored. Currently, it has terrible code, documented features do not work, and other things work badly in general. A new element will likely be created to allow better usage, although the existing ones will be preserved for compatibility. This may use button containers for better styling and usage.
  3. Formspec coordinates are rather limiting, so I plan to add units like CSS em and a combination between % and vx, vy, vmin, and vmax. Secondly, I hope to add layouting containers like grids, sizers, and whatnot.
  4. At some point far in the future after much else is completed, I will probably add some more useful elements like spinboxes and editable dropdowns that many GUI libraries have.
Main Code Refactoring Prio. 0, refactoring is really important!
After #10222 (inventory list refactoring) has been completed, more refactoring will be done. This is probably the most important point of all and is the highest priority of anything. The formspec code is garbage, and it needs fixing. This is the necessary prerequisite to getting many features that people have been wanting for so long. To name two specific things, dynamic formspec editing and a new syntax, but really for all complex features. This is, unfortunately, a lot of work with no immediate benefit for users, but it will pay off.

I won't go into detail in this forum post as it's all very technical internal stuff, but it mainly necessitates rewriting much of the existing code in GUIFormSpecMenu, especially the parsing code at first to allow migration to a new format, but later to add a proper layouting system and dynamic element creation/editing to allow dynamic formspecs. My specific roadmap for this can be read in this forum reply.

Final Notes

Most of this stuff is my personal opinion and roadmap and not necessarily the opinion of the other people working on formspecs. I would especially like their input on anything they see as wrong, misguided, or left out. I would also like to see what features not listed here users consider important because this is, after all, for the benefit of the Minetest community. I will, of course, keep these lists updated as time and motivation permit.
Last edited by v-rob on Mon Oct 26, 2020 15:39, edited 7 times in total.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
GreenXenith
Member
Posts: 1356
Joined: Wed Oct 28, 2015 01:26
GitHub: GreenXenith
Location: UTC-8:00
Contact:

Re: Formspecs, The Past, Present, and Future: An Overview

by GreenXenith » Post

I am rather excited for all of the formspec changes, I just wish they could happen sooner :D

Having styling features for every element is very important, because having the default irrlicht grey gradient on any element seriously breaks the look of themed formspecs. I am glad you have thought of this.

I saw the model element PR and was somewhat disappointed, to say the least. For one, the options for movement seem to be lacking. You can either click and drag to rotate, or control it via the element, but the element control is severely limited, and there is no option for just mouse control without clicking (Minecraft-like player preview .. not that I actually want to use that, but some people might). On top of this, I cant set the animation of the mesh if it has any, which is sort of important for certain previews. The ideal model element would somehow include object visual methods (set_animation, set_bone_position, set_attach, etc), that way you can make it look however you want. Since the element is so limited, I will probably end up making my previews using a form with a transparent viewing window showing an in-world entity, even though I would rather not as it brings up other complications (other players can see the entity and whatnot). I know it isn't your PR, but I'm still giving my thoughts about it.

One thing I regularly miss is dynamic formspec updating. You can change the form in a node live without interrupting the user interaction, so why cant we do so in regular formspecs? For example, I have made use of RGB sliders in various formspecs, but they only really work well in nodes. In a detached formspec, whenever you resend the form after moving the slider (say, to update a colorized preview), the slider gets dropped and the user has to click to start moving it again. This is not an issue in node formspecs, but I don't have the liberty of using node formspecs everywhere.

Your formspec refactor is probably one of the best items I see in this list. I severely dislike the string-based form creation. Yes, it's compact. But it is not always clean or pretty, and it also limits the features that can be added. If formspecs were object-based, elements could have methods (for movement, styling, etc .. could be nice for that model element). Even if they aren't object-based, any real GUI system would open up more opportunities than they current system grants.

And hey, being a core dev doesn't prevent you from specializing in a specific area. I think you'd be great as a formspec core dev :]
YouTube | Mods | Patreon | Minetest Discord @greenxenith

You should not be able to read this message.

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: Formspecs, The Past, Present, and Future: An Overview

by rubenwardy » Post

The formspec refactor won't change the string-based API of formspecs - just the internal code. Changing the API would require a formspec language replacement, there's an issue for that
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

Glad to hear some positive feedback. Yes, I wish things could happen faster too. ATM, I'm basically at a standstill because I need some PRs merged before I can work on other things. That's the main reason for the priority numbers, because these are the most important things to be merged for important things to progress. It's not saying which things will be merged first, just their relative importance to future PRs.

I could make one or two more PRs currently, e.g. style animations don't necessarily depend on anything else, but I'd rather not because I can't really start refactoring until I get most of my PRs merged. No one wants to rebase 11 PRs.

Of course, core dev time is limited, so I'm patient :) But if any core devs who want to review a formspec PR or two, this list should help out a bit.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
jp
Banned
Posts: 947
Joined: Wed Dec 18, 2013 09:03
GitHub: kilbith
Location: France

Re: Formspecs, The Past, Present, and Future: An Overview

by jp » Post

GreenXenith wrote:
Mon Aug 17, 2020 20:54
I saw the model element PR and was somewhat disappointed, to say the least. [...]
Since it's my PR, I will reply. First of all, you should be satisfied that this PR exists in the first place. It's not really encouraging to read that it's "disappointing, to say the least" whereas I spent late nights on this.

The options for movements will be made possible if the core-devs move their arses and review #8679 and merge it after a year of waiting and a lot of enthusiasm from the best modders. Then it will be possible to create a 3D game nested in another 3D game.

Setting animations to mesh[] is a low-hanging fruit BUT more improvements can come incrementally in later PRs, for now I just laid the foundations. Like my colleague did for hypertext[].

Priority number one should be making v-rob a core-dev for the continued attention to the formspec PRs over the time and to handle all of this workforce on this area that desperately need(ed) it.
Last edited by jp on Tue Aug 18, 2020 02:40, edited 1 time in total.

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

I do think that getting a base down is important, and that's what the mesh[] PR does. Honestly, complicated controlled interactions like spinning, rolling, panning, etc. would be best handled by the mod itself with the mesh[] element only handling the necessary rendering details. #8679 combined with dynamic formspecs would achieve this goal beautifully.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
Pyrollo
Developer
Posts: 385
Joined: Mon Jan 08, 2018 15:14
GitHub: pyrollo
In-game: Naj
Location: Paris

Re: Formspecs, The Past, Present, and Future: An Overview

by Pyrollo » Post

Hi V-Rob,

Thanks a lot for all your work on formspecs. It really moves things forward.

I’m not active on code these times but still thinking a lot. Code refactoring is one of my concerns. Specially about formspecs and horribly huge guiFormSpecMenu.cpp file. The first two though I had about this topic are :
  • Put Formspecs element class files (guiButton.*, guiScrollBar.*, ...) and windows class files (guiKeyChangeMenu.*, guiConfirmRegistration.*, …) in two separate dirs. Having them together is confusing, specially for newcomers.
  • Remove all parse* functions (parseButton, parseScrollBar, ...) from guiFormSpecMenu.cpp and put them in element class. And why not, have a sort of registry of element names and corresponding class (1).
I also think we should get rid of separate lists of elements (m_backgrounds, m_tooltip_rects, m_clickthrough_elements, …) that are drawn in specific ways. Instead a proper z-indexed list should do the job.


(1) Thinking a bit more, this idea is not optimal. Having formspec string parsing stuff in element class would bind rendering stuf to formspec string parsing. Bind parsing to rendering is never good. Maybe we will develop something else than formspec string in a future version.

Trying to find something smart, came in mind the idea that HUD and Formspecs should be managed the same way. Lot of elements are common : text, itemimages, minimap (when #9079 merged), …
It’s just a matter of sizing, positionning and when to display. A same text element type can be displayed :
  • In formspec, with position reletive to its parent;
  • In HUD as static element, with position relative to screen;
  • As Waypoint HUD, with position relative to a map coordinate;
At the end, once positionning and sizing is done, drawing is the same.

Sorry if this post is a bit messy, I’m not that good at putting ideas in words.
[ Display Modpack ] - [ Digiterms ] - [ Crater MG ] - [ LATE ]

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

I only have a general plan of the parsing parts of refactoring planned out. I don't have implementation details worked out yet fully (Rubenwardy has made a chart of his proposal [issue #9358 comment] which I will probably base mine off of in part), but I'll share my idea as far as I have it.

1) Separate inventory lists. This entails finishing #10222 and any subsequent PRs. This wouldn't have been my first choice to start working on, but it's already there, so I'll finish it. Unfortunately, this will take a while because I wasn't the one who created the PR, so I have to familiarize myself with it and suchlike.

2) Move everything into parserData as possible that only has to do with parsing. Things that don't have to do with parsing will be moved out. This will isolate all parsing things in a specific place so that all parsing can be moved out of GUIFormSpecMenu in the next refactor.

3) Move just the parsing code out into new classes. Currently, the parseSomeElement functions both parse and create the element. I will move only the parsing out, not the element creation. The reason why is that it isn't the parsing code's job to make the element itself, only to parse into something that can be used more easily.

Rubenwardy attempted a refactor earlier which tried to move the entire parsing functions out including the element creation code. I'm convinced that this isn't a good idea because we would have the same problems, just in a different class. Eventually, we'd have to move the element creation code again, so it's extra work for no benefit. (I'm not trying to insult or diminish Rubenwardy's effort -- it was a valiant one, but I think it was partly misguided in this one respect.)

Anyway, the parser will parse the entire formspec string, and then it will hand this parsed stuff back to GUIFormSpecMenu, which will proceed to create the elements.

The parsing code will be designed with the possibility of JSON serialization to replace the current formspec string syntax in the future. The JSON would be generated from whatever format is decided upon in #6527. I'm convinced that the formspec string needs replacement if dynamic formspecs are ever to be realized. There are too many problems with the current format to allow it to work -- too many forward references, element cross-dependencies, helper elements, and whatnot.

I don't know exactly what the parsing code will spit out, but I guess it'll look somewhat like what StyleSpec does right now. The parsing code will automatically handle forward references and cross-dependencies itself, e.g. applying scrollbaroptions[] to all following scrollbars and styling to all applicable elements so that GUIFormSpecMenu can create elements without having to worry about formspec syntax whatsoever.

These three refactors will solve many problems right away, but there are things to yet be considered, e.g. the prolific FieldSpec, where element creation should be moved, etc. I don't know about these yet, but I expect to figure them out after these refactors come to pass.

So, when will this happen? I don't think step 1 has any dependencies come to think of it, so I might be able to start that very soon, but steps 2 and 3 need all PRs that modify GUIFormSpecMenu's parsing code in any regard to be completed and merged. I'll add which PRs are dependencies to these second and third refactors next to the priority numbers.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
GreenXenith
Member
Posts: 1356
Joined: Wed Oct 28, 2015 01:26
GitHub: GreenXenith
Location: UTC-8:00
Contact:

Re: Formspecs, The Past, Present, and Future: An Overview

by GreenXenith » Post

jp wrote:
Tue Aug 18, 2020 00:47
you should be satisfied that this PR exists in the first place
My intent was not to offend or to belittle your work, but you do not get to tell me whether or not I am satisfied with a feature. As a modder, I do not find it to have enough features to be useful (yet).
Then it will be possible to create a 3D game nested in another 3D game
While I am the kind of person to do something silly and absurd like this, I seriously don't think this is something very many people care about, nor should it be. But on this point, a 3D game in a 3D game requires much more mesh control than the PR currently provides.

I am aware that it is just groundwork, but that does not change the fact that in the current state it is not terribly useful for much other than extremely simple player previews. I simply wanted to state the features I believe are needed to make it more generally useful, and I have done so.
YouTube | Mods | Patreon | Minetest Discord @greenxenith

You should not be able to read this message.

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

Well, things have just changed drastically for the better. I expect to make some more changes around here, because I have just been elevated to core developer! It's all happened rather fast; I didn't really expect to become one, not this soon at least. (I still don't quite believe it myself yet, but here it is: https://github.com/orgs/minetest/people)

Anyway, this means that PRs can be merged if I and one other developer approve. For my own PRs, I only need one other dev since I am the first. As the bulk of formspec PRs are my own, this should speed things up quite a bit, I expect. In fact, two PRs have been merged already (#10144 and #8678)

This also means that I can update the much-neglected official formspec priority list (https://github.com/minetest/minetest/projects/6), which is my first order of business. Then, it's on to reviewing PRs and getting things done!
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Formspecs, Hopefully to Just be the Past Soon

by v-rob » Post

I'm afraid this list has become all too obsolete. Not that there are many new formspec PRs, but plans have completely changed. The former main idea of refactoring is probably not going to be done. So here's what's been going on: for the past few weeks, I've been experimenting with CSM drawing to the screen, i.e. giving Lua the ability to draw rectangles, images, text, item images, and whatnot to the screen. After extensive testing, I've discovered that this is almost no slower than the speed of formspecs.

So, instead of refactoring, my master plan is to put all formspec work on hold except for highly anticipated features (like model[]) and critical bugfixes, and then to develop a new GUI system in Lua. Finally, after completion, a new SSM API will be developed to interface with the CSM GUI system, and a compatibility layer will be created for formspecs. Then, all that nasty existing formspec code will be thrown out the window, never to be seen again (yay!) and most of the formspec issues will be closed (that's quite a few).

There are many advantages to this Lua-side GUI idea:
  • Developing in Lua is much easier and faster than in C++, so this will make the initial development and later feature development much faster.
  • Everything can be developed from the ground up with features in mind from the start instead of jury-rigging every new feature onto the old system. Things like proper layouting and dynamic editing will be present from the beginning.
  • CSM can take advantage of the new system, defining its own elements with all the benefits of every other element.
  • Anything in CSM, not just the GUI, can take advantage of full mouse and key events (and hopefully touchscreen and joystick events soon), offering just as much as Irrlicht itself (and, well, a little more since Irrlicht's going to need a bit of patching to get proper touchscreen events everywhere).
  • The HUD and GUI can be unified into a single underlying system with similar APIs. In addition to the obvious benefits, making custom Android controls will also be a possibility in the future.
This all sounds far-fetched, but I think it's possible, and I'm devoting most of my development time to getting it to be a reality.

So, how far out is this? I don't know. I personally love the idea, and the other core devs seem to have no objections. ATM, I'm working on the C++ drawing and event API, fleshing it out before working on anything else. At the same time, I'm creating a TODO list for a lot of the internal things than need remedying before that can be completed. Once I get all that done, I'll try to get it merged into the engine before working on the actual Lua GUI code.

If you want to see the PR and API details in its current state, have a look at #10524.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: Formspecs, The Past, Present, and Future: An Overview

by Linuxdirk » Post

This looks quite wonderful! It's about formspecs, but understand I correctly that minetest.register_on_event() basically allows mods to get arbitrary input events and handle them as they like? Or does that only affect CSM (which was a mistake to release in the current state)?

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

Linuxdirk wrote:
Mon Oct 19, 2020 23:30
This looks quite wonderful! It's about formspecs, but understand I correctly that minetest.register_on_event() basically allows mods to get arbitrary input events and handle them as they like? Or does that only affect CSM (which was a mistake to release in the current state)?
It does, at present, only affect CSM because the client is where events come in and where everything is drawn, so the client is where the GUI code will be. But adding something to send events to the server sounds like a good idea to me (although I have no expertise with packets and whatnot, so it probably won't be me working on it, at least not any time soon). Drawing directly from the server will never work for obvious reasons, though :)
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: Formspecs, The Past, Present, and Future: An Overview

by Linuxdirk » Post

v-rob wrote:
Tue Oct 20, 2020 00:58
It does, at present, only affect CSM
So in the end we will have one awesome, modern system allowing random key combinations and one input system that chokes on non-QWERTY input :D

But what’s the state of CSM in general? The information available from the dozens of issues and discussions regarding CSM is that CSM in it’s current form was never intended nor planned. It’s a by-product from the time when some devs worked on SSCSM.

I hope your work does not force CSM to be enabled in the client in order to get what you described.

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

Linuxdirk wrote:
Tue Oct 20, 2020 08:25
I hope your work does not force CSM to be enabled in the client in order to get what you described.
In the present state, it is a CSM API so it does require CSM to be enabled. When it's used to make GUIs, then that specific part of the client builtin will have to be loaded separately, regardless of whether the rest of CSM is enabled. We can't have the HUD, GUI, chat, etc not work because someone doesn't turn on the entire CSM API :) (User: Hey, why isn't the main menu showing up?! Dev: Yeah, um, you need to enable CSM for it to work. Ha ha.)

As for sending events to the server, that would bypass the client builtin completely by sending Irrlicht's internal event over the network with minimal serializations, and then the server would run the same C++ -> Lua code on it that the client does, just for SSM instead of CSM. Not hard at all as long as I can figure out what part of the networking code I need to change.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
Andrey01
Member
Posts: 2574
Joined: Wed Oct 19, 2016 15:18
GitHub: Andrey2470T
In-game: Andrey01
Location: Russia, Moscow

Re: Formspecs, Hopefully to Just be the Past Soon

by Andrey01 » Post

v-rob wrote:
Mon Oct 19, 2020 22:46
I'm afraid this list has become all too obsolete. Not that there are many new formspec PRs, but plans have completely changed. The former main idea of refactoring is probably not going to be done. So here's what's been going on: for the past few weeks, I've been experimenting with CSM drawing to the screen, i.e. giving Lua the ability to draw rectangles, images, text, item images, and whatnot to the screen. After extensive testing, I've discovered that this is almost no slower than the speed of formspecs.

So, instead of refactoring, my master plan is to put all formspec work on hold except for highly anticipated features (like model[]) and critical bugfixes, and then to develop a new GUI system in Lua. Finally, after completion, a new SSM API will be developed to interface with the CSM GUI system, and a compatibility layer will be created for formspecs. Then, all that nasty existing formspec code will be thrown out the window, never to be seen again (yay!) and most of the formspec issues will be closed (that's quite a few).

There are many advantages to this Lua-side GUI idea:
  • Developing in Lua is much easier and faster than in C++, so this will make the initial development and later feature development much faster.
  • Everything can be developed from the ground up with features in mind from the start instead of jury-rigging every new feature onto the old system. Things like proper layouting and dynamic editing will be present from the beginning.
  • CSM can take advantage of the new system, defining its own elements with all the benefits of every other element.
  • Anything in CSM, not just the GUI, can take advantage of full mouse and key events (and hopefully touchscreen and joystick events soon), offering just as much as Irrlicht itself (and, well, a little more since Irrlicht's going to need a bit of patching to get proper touchscreen events everywhere).
  • The HUD and GUI can be unified into a single underlying system with similar APIs. In addition to the obvious benefits, making custom Android controls will also be a possibility in the future.
This all sounds far-fetched, but I think it's possible, and I'm devoting most of my development time to getting it to be a reality.

So, how far out is this? I don't know. I personally love the idea, and the other core devs seem to have no objections. ATM, I'm working on the C++ drawing and event API, fleshing it out before working on anything else. At the same time, I'm creating a TODO list for a lot of the internal things than need remedying before that can be completed. Once I get all that done, I'll try to get it merged into the engine before working on the actual Lua GUI code.

If you want to see the PR and API details in its current state, have a look at #10524.
Wow! It`s so nice to hear all that! You really have amazing ambitions for MT! Perhaps, are you going to jump on an other 3D engine that adapted for Lua? Also, currently I`m preparing a new feature for the old formspec system, but does it make sense to embed any new ones to that now if you have already got involved into making a new GUI? Just I`d not like to waste a time and effort...

User avatar
v-rob
Developer
Posts: 970
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: Formspecs, The Past, Present, and Future: An Overview

by v-rob » Post

Andrey01 wrote:
Sat Oct 24, 2020 20:00
Also, currently I`m preparing a new feature for the old formspec system, but does it make sense to embed any new ones to that now if you have already got involved into making a new GUI? Just I`d not like to waste a time and effort...
It's your call, really. There's no telling how long this all will take. If you're willing to wait any amount of time from a few months to a year or two, then by all means wait. But it might be better to just work with formspecs for now and transition later after the new stuff is done. Compatibility with formspecs won't be dropped regardless.
_______________________________________________________________________

I don't feel like coding at the moment, so here is my concept for the new GUI/HUD API so far. First off, this is the client API, but I intend to make the server API reflect the client API as much as possible, but some features just won't be possible, e.g. custom elements, without SSCSM.

Everything will be in the ui namespace to keep it together outside of the minetest namespace. UI is a more general term, and it encompasses both GUI and HUD.

Every element will inherit from the base class ui.Element. This will have a number of properties (some of these may not actually be implemented, and others will probably be added):
  • id: The unique internal ID of the element, different for each individual element
  • rect: The rectangle with layouting units (more on that later)
  • draw_rect: The rectangle in pixels for drawing the element
  • parent: A reference to the parent element
  • children: A table of children of this element
  • sub_elements: A table of elements that are sub-elements but not actually children, e.g. the text on a ui.Button might be a ui.Label.
  • visible: If the element should be drawn and receive events
  • disabled: Whether the element is disabled
  • noclip: Should the element be clipped to the parent element?
  • name: The user-provided unique name for the element, like HTML id
  • class: A table of the classes for the element, like HTML class
  • type: The type of the element, e.g. "ui.Element" or "ui.Button"
The base class will also have plenty of methods. Some of these are:
  • draw(dtime, drawer): Called every frame to draw the element
  • on_event(event): Called if an event occurs and the element has focus
  • clone(): Returns a copy of the element
  • move_after/before(other): Moves this element after or before another
  • remove(): Removes this element
  • get_element_by_id/name/class/type/etc(thing): Gets an element that is the child of this one by some attribute.
  • get_element_at_position(pos)/(x, y): Gets the topmost element at the given screen position
  • set_rect(rect)/(x, y, w, h): Sets the rectangle with layouting units. This internally calls resize.
  • resize(): Resizes the element and all children/sub elements because the rect changed or the screen/parent element resized.
  • add/remove_class(class): Adds or removes a class
There will also be event handler callbacks a lot like HTML+JavaScript event handlers, e.g. on_click, on_mouseover, on_mouseout, etc, with some differences.

Lastly, elements will have a fancy constructor and the bare-minimum constructor. Bare-minimum is probably a lot like ui.Element(parent, rect, label, name) where all parameters are optional. The fancy constructor, rather XML/HTMLish, would look more like this:

Code: Select all

local elem = ui.Element {
    rect = {x=0, y=0, w="1w", h="1h"},
    name = "cool_element",
    
    ui.Button {
        rect = {w = "100p", h = "20p"},
        name = "btn_1",
        class = {"btn", "red"},
        label = "Some Button"
    }
    -- More child elements go here
}
Actual elements will have extra properties, methods, and callbacks, of course. ui.Element is just the bare minimum.

As mentioned above, rect is a rectangle with layouting units. This is essentially CSS units that look like "<number><unit>". These would be p (pixel, or px), f (proportional to font height, or em), w h > < (proportional to parent width, height, greater dimension, or smaller dimension, like a combination of % with vx, vy, vmax, and vmin). I have not decided whether there should be something akin to formspec coordinates. I also haven't decided the default unit if a unitless number/string is provided. Lastly, units can be combined with a + symbol, e.g.0.5w+20p.

What elements will we have? Here's a non-comprehensive on-the-moment list:
  • ui.Element doubles as a simple container.
  • ui.Scroller is a container that scrolls when elements overflow outside of the boundaries, or alternatively, the width/height can be explicitly set.
  • ui.Sizer arranges child elements itself, like a vertical or horizontal sizer or a grid
  • ui.Button is the all-purpose button. No need for button, image_button, and item_image_button
  • ui.Label is just that, a static label. Optional word wrapping
  • ui.Input is an input text box, single or multiline, word wrapping and not.
  • ui.ScrollBar is a scrollbar, but can also be used like a slider
  • ui.DropDown is a dropdown with optional textual input
  • ui.CheckBox is a checkbox and can also be a radiobutton
  • ui.Tabs and ui.ProgressBar are what they sound like
  • ui.List is a simple table or textlist. If one wants more options, use a ui.Element with ui.Button children
  • ui.HyperText is like the formspec hypertext. This is one of the last elements that will be made because of the complexity
  • ui.Inventory is an inventory list.
  • ui.Minimap, ui.HotBar, and ui.ChatBox are the probable equivalents of their HUD counterparts. Statbars will probably be ui.ProgressBar's.
Really, I can add all sorts of goodies to this list once the base API gets in place. Many of these won't be very hard at all. The hardest are likely to be things that deal with lots of text, like multiline ui.Input and ui.Hypertext.

Styling. Well, I really don't know how this should work yet, and I'm quite open to suggestions.

The base that does a lot of the management is ui.Environment. This manages a lot of the stuff like where to make events go, which element holds the focus, etc. Not very interesting.

So, how will things like handling both the HUD and GUI and even things like multiple GUIs work? Well, the ui.Environment, which is also the root element, will have a number of built-in child elements which will be ui.Elements as large as the entire screen, which are, in essence, the replacement for many of the z_index positions. These elements are so important that they will have global variables associated with them for easy adding to them. These might be something like ui.effects (for graphical effect elements), ui.hud, and ui.gui, although more specific. These are not for inserting single things like ui.Button (although they can be used that way), but a ui.Element to hold everything (a lot like formspec size[] make a box for everything).

And how will the HUD work in a GUI system? Actually, it's going to literally be a GUI, but when in game, you can't interact with either the HUD or the GUI if any GUIs are still open (they shouldn't, but they wouldn't be interactable even if they were). So yes, you could put a button on the HUD. That isn't useful on desktop systems, but with touchscreen systems, this means that you could make custom control buttons on the HUD.

How do you make custom elements? Pretty easily. There will probably be a gui.inherit(base_class) function that returns an inherited class from another element (or ui.Element by default), and then you just add or override methods in the returned class. As for how to make the custom elements, looking at some of the built in ones will be the easiest method to learn how.

Anyway, I need to wrap this up as it's getting long. But I hope I've clarified somewhat what the API could quite possibly look like. Like always, suggestions, criticisms, and questions are quite welcome.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests