Page 1 of 2

Formspec replacement proposal

Posted: Sun May 27, 2018 13:33
by TeTpaAka
Hello community,

Since there are multiple issues with our current formspec system (see https://github.com/minetest/minetest/issues/6527), I propose a different layouting scheme for a new one. To test my proposals, I modified the current formspec renderer. This, however, comes with multiple limitations:
  • I completely removed any interactivity (for now). Buttons show interaction, but no response is send to the server.
  • This won't compile for Android since I left the Android code in place but removed code it depended on.
  • There will probably be some issues if you try to display an old and a new formspec at the same time.
  • The formspec strings can get pretty long, especially autogenerated float values take up far more space than necessary. A binary format would be preferable.
Concept:
  • We work in normalised coordinates. This means, positions are in the range [0,1] and are relative to a parent object.
  • Everything you see is a rect. Every rect can have multiple appearance changing modifiers (like background color/image...) and up to one behaviour changing modifiers (like button, inventory...)
  • Every rect can have multiple child rects. Coordinates of the child rect are relative to it's parent.
Lua API:
To facilitate creating forms I created a small mod (see below) which offers an object oriented interface to formspecs. To create the initial window, you would call:

Code: Select all

local window = formatter.create()
This creates a window with transparent background that spans the whole screen. Since most menus will only need a smaller area on the screen, you can change it's size and position relative to the screen:

Code: Select all

window:set_size(.5, .5)
window:set_pos(.25, .25)
The aspect ratio of this window is still dependend on the aspect ratio of the screen. This will most likely mess with your menu design. To compensate, the function:

Code: Select all

window:set_aspect(16,9)
will force a specific aspect ratio. To fit the aspect ratio, the renderer shortens one of the sides until the ratio is as specified.
To actually see anything, you have to either set a background color or an image to the window:

Code: Select all

window:set_background_color("#FFFFFF") -- currently only strings are accepted
window:set_image("modname_texturename.png")
If you set a background image, you probably want to align your interface to the background. To facilitate this, you can set the window into a pseudo absolute mode. Internally, the formspec is still specified by relative coordinates. The Lua API transforms pixel sizes to relative offsets.

Code: Select all

window:set_absolute_size(width, height)
Now we can add interface elements to the form:

Code: Select all

window:inventory(spec)
window:button(spec)
window:image(spec)
window:label(spec)
Spec is the specification of the new element it is a table with the following format:

Code: Select all

spec = {
    x = 0, -- position relative to the parent
    y = 0,
    width = 1, -- size relative to the parent
    height = 1,
    image = "modname_texturename.png", -- image spec (image only)
    text = "your text", -- text to display on the element (button and label only)
    name = "b1", -- id of the button (button only)
    location = "current_player", -- inventory location (inventory only)
    list = "main", -- inventory list (inventory only)
    columns = 8, -- number of inventory columns (inventory only)
    rows = 4 -- number of inventory rows (inventory only)
}
I will probably remove spec.name and replace it with a callback once I actually react to button presses.
To display the formspec you would have to call:

Code: Select all

minetest.show_formspec_new(name, "formname", window:generate())
I will probably create a function that autogenerates a unique formname and processes answers of the client and calls appropriate callbacks.

Formspec format
  • general Element (Image/Label ...)

    Code: Select all

    {
        rect = { "x0" : 0, "y0" : 0, "x1" : 1, "y1" : 1 },
        bgcolor = "#FFFFFF",
        image = "modname_texture.png",
        aspect = {"x" : 16, "y" : 9 },
        text = "some text",
        children = [{ ... }, ... ],
        style = { "style_name" : "style_value" }
    }
    
    Create a new rectangle spanning from (x0, y0) to (x1, y1) in relation to it's parent.

    Styles:
    • text_align, specifies the alignment of the text can be one of: center, top, topright, right, bottomright, bottom, bottomleft, left, topleft
    Currently, text is drawn with a fixed color (white). I will make the text color available as style if I continue working on this.
    The text is automaticly word wrapped. Enabling this will become an option later on.
    Also, text clashes with the basic design principles of this new formspec. It doesn't scale with the rest of the gui and may get outside of the rect. Suggestions on how to fix this are welcome.
  • Inventory

    Code: Select all

    {
        ... see above
        "type" = "inventory",
        "columns" = 1,
        "rows" = 1,
        "location" = "current_player",
        "list" = "main"
    }
    
    converts the parent rect into an inventory.
    The inventory completely fills the rect and touches the border at all four sides. Inventory slots are square and have a padding of at least 2/13 inventory slots in x-direction and 1/4 inventory slot in y-direction.
    The old formspec drew a one pixel border on the outside of every slot. Since this clashes with my design, the border is now drawn on the inside and the image is scaled down appropriately.
    Styles:
    • inventory_background_color, specifies the color of the slot background
    • inventory_border_color, specifies the color of the slot border
    • inventory_border_width, specifies the width of the slot border
  • Button

    Code: Select all

    {
        ... see above
        "type" : "button",
        "id" : "unique identifier"
    }
    
    converts the parent rect into a button.
    The button background is draw according to it's style. Background images and colors are ignored for now (this will probably change).
    The id will be set in the answer when the button ist pressed (not implemented yet).
    Styles:
    • button_standard, set it to a texture to change the default appearance of the button
    • button_hover, set it to a texture to change the appearance of the button when the cursor is inside it's rect
    • button_pressed, set it to a texture to change the appearance of the button when it get's pressed
[/list]
Links:
Lua API: https://github.com/TeTpaAka/formatter
Core modifications: https://github.com/TeTpaAka/minetest/tree/formspec

By issueing the chatcommand "/testform" you can see an example gui I created.
Image
The example gui uses images from https://opengameart.org/content/ui-pack-rpg-extension.

TODO:
  • Add

    Code: Select all

    input[name]
    element for text input.
  • port the field sending methods from the old formspec
  • split the gui elements into their own file
  • split the parsing into it's own file
  • add error and sanity checks to core and Lua API
  • Add more elements like checkboxes, dropdowns, radiobuttons, ...
  • Add scrollable rects

Re: Formspec replacement proposal

Posted: Mon May 28, 2018 13:56
by rubenwardy
I'd rather JSON was used as the serialisation format as it will handle children much better

Re: Formspec replacement proposal

Posted: Mon May 28, 2018 15:48
by TeTpaAka
rubenwardy wrote:I'd rather JSON was used as the serialisation format as it will handle children much better
I'm not fixed on the serialisation format. I just used the existing one because it was easier to start with a working format and modifying it to my needs than to incorporate a new one. I can switch to JSON though later on. Switching the underlying format should be rather easy.

Re: Formspec replacement proposal

Posted: Mon May 28, 2018 23:24
by v-rob
First off, I'd like to say that it's awesome that someone has actually tried to do something about this issue with formspecs. To my knowledge, nothing like this has happened before. I have some questions about this, though.
  • 1) I'm not quite sure how the positioning of the formspec windows works. From what I gather, setting the position to 0.5, 0.5 will center the window on the screen, correct? And setting the size to 0.5, 0.5 would make it half the size of the screen?

    2) Shouldn't there be a default aspect ratio? It would make all formspecs that don't define one work properly in all resolutions.

    3) In the inventory element in test.lua, it shows width, height, rows, and columns. But why is width and height necessary when rows and columns are already defined? Wouldn't that make inventory slots bigger or smaller?

    4) Is it possible to show images on inventory slots?

    5) Are you planning on making images for when you hover over buttons and such?

    6) Will there be default images for everything in minetest\textures\base\pack? I think this would make formspecs better because people wouldn't define their own images for everything and texture packs could change things easier.

    7) How is interaction planned? Will interaction be in a different function like the current formspecs, or will the function be part of the element specification?

    8) Can a child element have another child element?
I'm really glad that someone's trying to do something about formspecs, and I'm sorry for all the questions.

Re: Formspec replacement proposal

Posted: Tue May 29, 2018 11:00
by TeTpaAka
v-rob wrote:First off, I'd like to say that it's awesome that someone has actually tried to do something about this issue with formspecs. To my knowledge, nothing like this has happened before. I have some questions about this, though.
  • 1) I'm not quite sure how the positioning of the formspec windows works. From what I gather, setting the position to 0.5, 0.5 will center the window on the screen, correct? And setting the size to 0.5, 0.5 would make it half the size of the screen?
Positions are relative to the top left corner of the screen. So, for a window with size(0.5, 0.5) setting the position to (0.25, 0.25) would center it. I probably should add a convenience function to the API that does these calculations, like

Code: Select all

window:center()
or something like that.
v-rob wrote:
2) Shouldn't there be a default aspect ratio? It would make all formspecs that don't define one work properly in all resolutions.
That does seem like a good idea. I will think about it. Probably (1, 1) would be suitable.
v-rob wrote: 3) In the inventory element in test.lua, it shows width, height, rows, and columns. But why is width and height necessary when rows and columns are already defined? Wouldn't that make inventory slots bigger or smaller?
First part: I changed the layouting scheme from the previous formspec definitions. Now, it no longer treats inventory slots as (sort of, it wasn't even consistent with that) basis. This way, the inventory is always scaled to the full size of the rect. So:
Second part: Yes, it does make the inventory slots bigger/smaller.
It also adds appropriate padding around them to keep them square. I will probably add a style option to make them stretch fo fit instead of padding them, if the modder wants.
v-rob wrote: 4) Is it possible to show images on inventory slots?
You can overlay an Image over the inventory, if this is what you mean. Form elements are drawn in order they are defined in the formspec string.
v-rob wrote: 5) Are you planning on making images for when you hover over buttons and such?
You can already define images for hovering over buttons and for clicking them. If you suggest more, I can add more different style options.
v-rob wrote: 6) Will there be default images for everything in minetest\textures\base\pack? I think this would make formspecs better because people wouldn't define their own images for everything and texture packs could change things easier.
I currently don't have plans for that, sorry. Maybe later.
v-rob wrote: 7) How is interaction planned? Will interaction be in a different function like the current formspecs, or will the function be part of the element specification?
Internally I will probably use something similar to the way formspecs report events to mods right now. But I will provide callbacks to the user, that get called when an event occured. e.g:
local button = window:button({
x = ...,
y = ...,
width = ...,
height = ...,
text = "...",
callback = function(playername)
end
})
v-rob wrote: 8) Can a child element have another child element?
Yes. But there are some issues in the Lua API currently with buttons and formspecs I will resolve later. For images it works without a problem.
v-rob wrote: [/list]
I'm really glad that someone's trying to do something about formspecs, and I'm sorry for all the questions.
Don't hesitate to ask. This is in a very early state of development and suggestions are always welcome. I might not incorporate all of them, but I will explain why I don't.

Re: Formspec replacement proposal

Posted: Tue May 29, 2018 23:44
by v-rob
Wow. From what I can see, this is exactly the kind of formspecs I've been wanting. Keep up the good work!

Re: Formspec replacement proposal

Posted: Wed May 30, 2018 18:46
by texmex
I just want to add my support to this project as it seems to have the potential to overcome major formspec issues. Kudos to tetpaaka for taking on this project when no-one else seems to be willing. :)

Re: Formspec replacement proposal

Posted: Sun Jun 03, 2018 17:37
by TeTpaAka
I switched the formspec format to a JSON based format.

Re: Formspec replacement proposal

Posted: Mon Jun 04, 2018 23:50
by v-rob
So, why haven't any developers commented on this? The only one who has said anything is rubenwardy. That's one way to make someone discouraged really fast. Seriously, it's a very important issue, and yet no one seems to care very much except for myself and a few others.

Re: Formspec replacement proposal

Posted: Mon Jun 04, 2018 23:56
by rubenwardy
Contradiction there - "no devs have commented" "rubenwardy has" :P

Re: Formspec replacement proposal

Posted: Tue Jun 05, 2018 00:04
by v-rob
rubenwardy wrote:Contradiction there - "no devs have commented" "rubenwardy has" :P
Besides you. But my question remains unanswered.

Re: Formspec replacement proposal

Posted: Tue Jun 05, 2018 00:07
by rubenwardy
Devs have real lives and may not check the forums. They also may not have looked into it enough to respond

Re: Formspec replacement proposal

Posted: Tue Jun 05, 2018 07:41
by TeTpaAka
I knew that posting it on the forum wouldn't attrackt dev attention. If I wanted to propose it to the development team, I would have opened a pull request on github. But this feature is not yet in a state I'm comfortable with presenting to the dev team. I will draw attention to it once I consider it ready for further discussions.

Re: Formspec replacement proposal

Posted: Tue Jun 05, 2018 16:25
by csirolli
This is pretty cool. I wish there was some way I could help.

Re: Formspec replacement proposal

Posted: Sun Jun 24, 2018 21:26
by Sokomine
TeTpaAka wrote: The text is automaticly word wrapped. Enabling this will become an option later on.
Also, text clashes with the basic design principles of this new formspec. It doesn't scale with the rest of the gui and may get outside of the rect. Suggestions on how to fix this are welcome.
In general, your approach looks very promising. However, the main issue I have with formspecs is with adding texts. A solution for that problem would be very welcome.

Re: Formspec replacement proposal

Posted: Thu Jun 28, 2018 21:28
by TeTpaAka
Sokomine wrote:
TeTpaAka wrote: The text is automaticly word wrapped. Enabling this will become an option later on.
Also, text clashes with the basic design principles of this new formspec. It doesn't scale with the rest of the gui and may get outside of the rect. Suggestions on how to fix this are welcome.
In general, your approach looks very promising. However, the main issue I have with formspecs is with adding texts. A solution for that problem would be very welcome.
Text is hard. The current approach is to let the modder define an area where text should be and try to fill as much of it as possible with the default font. This takes design elements out of the modders hand but should result in text that should be readable (at least, if it is readable now).

Other approaches would be to either let the modder define the fontsize relative to the screensize or define the whole UI in units based on the fontsize. The second approach would clash with my basic design and the first, while possible, probably wouldn't result in good readable text for the player.

Any suggestions on how to do it better are always welcome.

Re: Formspec replacement proposal

Posted: Fri Jul 13, 2018 22:29
by GamingAssociation39

Code: Select all

local window = formatter.create()
#html = formatter.create() --same effect as in actual html when called website will be created
#banner
	image=
	text=
	text_style = { "style_name" : "style_value" }
#header = get_label 
	scale= 
	align=
	text=
	text_style = { "style_name" : "style_value" }
#navigation = 
	item-link=
	item-icon=
#paragraph = get_paragraph
	wraparound= 
	text=
	text_style = { "style_name" : "style_value" }
#footer = get_foot
	title=
	sub-title=
	home-page=
	
	
=====================================================
	
Proposed functions for the new formspecs:

window:text(spec)
window:set_text_color
window:set_text_size
My proposal for some additional things and the API for my laptop mod's web browser MineBrowse.

Re: Formspec replacement proposal

Posted: Fri Jul 13, 2018 22:34
by rubenwardy
Would it be possible to design this in such a way that you could reuse components between the HUD and GUI windows? Ideally they should be exactly the same, just different roots and layoutters

Re: Formspec replacement proposal

Posted: Fri Jul 13, 2018 22:59
by TeTpaAka
GamingAssociation39 wrote:

Code: Select all

local window = formatter.create()
#html = formatter.create() --same effect as in actual html when called website will be created
#banner
	image=
	text=
	text_style = { "style_name" : "style_value" }
#header = get_label 
	scale= 
	align=
	text=
	text_style = { "style_name" : "style_value" }
#navigation = 
	item-link=
	item-icon=
#paragraph = get_paragraph
	wraparound= 
	text=
	text_style = { "style_name" : "style_value" }
#footer = get_foot
	title=
	sub-title=
	home-page=
	
	
=====================================================
	
Proposed functions for the new formspecs:

window:text(spec)
window:set_text_color
window:set_text_size
My proposal for some additional things and the API for my laptop mod's web browser MineBrowse.
If I understand you correctly, you want the ability to have colored text, right?
That should be possible, since chat messages can be annotated with color information. I will look into it later.
The font size on the other hand could be slightly harder. I could add the ability to set the font size directly, but that would potentially make text unreadable on high resolution screens. This is not likely to come in the near future. I can think about it later, when the rest of the design works.
rubenwardy wrote:Would it be possible to design this in such a way that you could reuse components between the HUD and GUI windows? Ideally they should be exactly the same, just different roots and layoutters
Ideally, the HUD would not be hidden when the GUI gets shown. That way, you could add p.e. buttons or usable inventory slots to the HUD.
I thought about doing this after I finished the Formspecs (at least the basics). This would probably require a replacement of the current HUD, too.

Re: Formspec replacement proposal

Posted: Sat Jul 14, 2018 14:50
by GamingAssociation39
TeTpaAka wrote:
GamingAssociation39 wrote:

Code: Select all

local window = formatter.create()
#html = formatter.create() --same effect as in actual html when called website will be created
#banner
	image=
	text=
	text_style = { "style_name" : "style_value" }
#header = get_label 
	scale= 
	align=
	text=
	text_style = { "style_name" : "style_value" }
#navigation = 
	item-link=
	item-icon=
#paragraph = get_paragraph
	wraparound= 
	text=
	text_style = { "style_name" : "style_value" }
#footer = get_foot
	title=
	sub-title=
	home-page=
	
	
=====================================================
	
Proposed functions for the new formspecs:

window:text(spec)
window:set_text_color
window:set_text_size
My proposal for some additional things and the API for my laptop mod's web browser MineBrowse.
If I understand you correctly, you want the ability to have colored text, right?
That should be possible, since chat messages can be annotated with color information. I will look into it later.
The font size on the other hand could be slightly harder. I could add the ability to set the font size directly, but that would potentially make text unreadable on high resolution screens. This is not likely to come in the near future. I can think about it later, when the rest of the design works.
So that's somewhat what I was talking about because I want to implement a way to properly create a website for my mod in MT and can't do so effectively without images as the title and such.

Re: Formspec replacement proposal

Posted: Fri Dec 21, 2018 19:52
by GamingAssociation39
How's this project going along??? Where are we going with this??

Re: Formspec replacement proposal

Posted: Sun Jan 13, 2019 20:49
by TeTpaAka
GamingAssociation39 wrote:How's this project going along??? Where are we going with this??
Sorry, I have too much to do IRL atm, so I haven't been able to work on it. I will update the thread when I come around to completing it.

Re: Formspec replacement proposal

Posted: Mon Jan 14, 2019 09:15
by Pyrollo
I just discovered this post. Unfortunately I haven't seen it before.

I'm working on a formspec mod I hope I can publish soon.

My mod aims to make it much easier to use formspecs with:
  • A table format form specification
  • Triggers associated to elements
  • Automatic security checks
And much more...

Much work has yet be done on it, but I still have some issues to deal with. I hope to release something in the next month.

If some of you are interrested, code is here : https://github.com/pyrollo/nofs_modpack

Master branch should be buggy but yet usable but I'm working on a rewrite (dev-context branch) that will replace most of master branch code.

Re: Formspec replacement proposal

Posted: Tue Jan 15, 2019 07:16
by Gael de Sailly
Pyrollo wrote:I just discovered this post. Unfortunately I haven't seen it before.
Same for me. I seem to always miss the good topics. Also I'm surprised not to see more interest in this proposal. It looks awesome!

Re: Formspec replacement proposal

Posted: Mon Feb 11, 2019 21:22
by orwell
Hi,

This proposal looks very promising, and I really appreciate that formspecs are to be replaced. However, I have (at the moment) a far bigger problem with how formspec interactions are handled. Examples:
- The state of a checkbox is not available on a button submit, only directly on change
- Same with textlists
- Dropdown needs to set a "selected index", but fields return the text in there, making ugly conversion hacks necessary (esp. when strings are translated)
This is a question to all devs in here: If formspecs are about to be replaced, will this also be improved? And should this go into a separate thread?