Variable length param2 - concept of a new map format

For people working on the C++ code.
Post Reply
juhdanad
New member
Posts: 6
Joined: Tue Jan 03, 2017 21:42
GitHub: juhdanad
IRC: juhdanad
In-game: juhdanad

Variable length param2 - concept of a new map format

by juhdanad » Post

One annoying thing about param2 is that it is only good for one thing - the level of a node, its orientation, or color. In some cases more of these are required (like colored stairs), in other cases param2 is not used at all (many regular nodes). Here I propose a new format, where the mod maker can explicitly define how much data each node requires (up to 16 bits), without increasing the memory consumed by a map node (so it's still 4 bytes).

Example code with this format:

Code: Select all

minetest.register_node("wool:wool_stair",{
    description = "Wool stair",
    tiles = "default_wool_white.png",
    paramtype = "light",
    data = "rraccccccccc", -- 2 bit rotation, 1 bit axis (only up and down are possible), 9 bit color
    drawtype = "nodebox",
    node_box = stair_mesh -- declared before
})
What changes?
Currently there is a 16 bit number describing the type of a node, and a 8 bit param2. This allows registering 65536 nodes, each having 256 variants (16777216 total). This implementation mixes the two numbers: there are still 16777216 total node variations, but each node occupies as many of them as needed for its variants. So you can register:
  • 16777216 regular nodes or
  • 524288 facedir nodes (each having 32 variants) or
  • 32768 colored nodes with 512 colors or
  • 1024 colored facedir nodes with 512 colors (14 bits) or
  • 256 nodes with 16 bits of custom data for your mod (without using metadata)
or some combination of the above.
To achieve this, instead of paramtype2 each node definition can contain a variant describing string (like "rraaa??") which tells how many bits are used, and how are they used. Each letter tells the function of one bit (r is rotation, c is color, ? is custom mod data etc.)
There will be some helper methods, too:
  • minetest.get_node_color_bits(node_name / node): replaces paramtype2=="color", number of color bits
  • minetest.get_node_color(node): returns an integer
  • minetest.set_node_color(node, color): if "color" is greater than the largest allowed value, it gets wrapped around
  • minetest.get_node_axis_bits(node_name / node)
  • minetest.get_node_axis(node)
  • minetest.set_node_axis(node, axis)
  • ... (for all current paramtype2 options)
  • minetest.get_node_custom_data(node)
  • minetest.set_node_custom_data(node, data)
Drawbacks:
  • No backwards compatibility: this is a new format, only old->new conversion is supported. Old clients can not connect to new servers, as they have different node IDs.
  • Loss on speed: on the c++ side, it will require more computations to get the ContentFeatures for a node ID, and extract the param2 information. On the Lua side, extracting param2 information requires computations. But I hope that the change will be barely noticeable.
  • Small increase in saved file's size and used memory: of course the variant description string requires some space, but not as much as adding a new param3 to nodes.
  • Possible breakage of mods: I think mods will not break, but I'm not a professional modder, and since the changes are huge, there might be something that I have not thought of.
Steps of implementation:
  1. Implement variable-size param2: this won't break mods, as backwards compatibility can be ensured.
    • paramtype2 will be interpreted as the corresponding variant string ("leveled" as "??llllll" etc.)
    • If paramtype2 is set, then each time param2 is requested or set, it will be the value of the corresponding node property.
    • Node IDs can be larger than 65536, but I don't think that it will cause much trouble.
  2. Rearrange axis directions and rotation directions to unify facedir and wallmounted. New ordering: rotation=0 always corresponds to the wallmounted direction. Axis directions: up, down, north, east, south, west (so one axis bit is for stairs, two bits are for symmetric logs/pillars, three bits are for all rotations). Maps could be automatically converted based on the node definitions, but there is no way to convert unknown nodes.
  3. maybe move meshoptions to the node definition
And finally, the technical details of the implementation:
The new layout of MapNode: u8 sector, u16 data, u8 param1.
There are 256 "sectors" to store node definitions, each sector has its own data layout: so there might be a sector for nodes that use no variants (stone, air etc.), another sector for facedir nodes etc... So for each sector, there is a layout containing variant length, color bit position, color bit mask, rotation bit position, rotation bit mask etc., and of course a vector of (maximum 65536) ContentFeatures. Then getting the ContentFeatures for a MapNode is like:

Code: Select all

// In CNodeDefManager
NodeDefSector *sector = sectors[node.sector];
u16 id = node.data >> sector->variant_length; // the content part
return (sector->node_defs.size() > id) ? sector->node_defs[id] : CONTENT_IGNORE;
Or for example getting the color of a node:

Code: Select all

// In CNodeDefManager
NodeDefSector *sector = sectors[node.sector];
return (node.data >> sector->color_position) & sector->color_mask;
When registering nodes, if one sector gets full, a new sector will be started.

I write this post to make sure that modders will welcome the change before starting to actually implement something. I'm also open to suggestions. Please let me know if this worth the effort or not.

User avatar
Hybrid Dog
Member
Posts: 2828
Joined: Thu Nov 01, 2012 12:46
GitHub: HybridDog

Re: Variable length param2 - concept of a new map format

by Hybrid Dog » Post

https://github.com/minetest/minetest/bl ... #L736-L737
Currently the node ids, then the param1 values and after that the param2 values are saved. So if lots of contiguous param2 values are 0, the mapblock can be compressed efficiently.
What is the impact on compression efficiency when using variable length param2?

I think if you would add a rarely used param3 for modders, the param3 values are almost always 0, so in the mapblock data the 16^3 B = 4 KiB, which are set to 0, disappear when compressed.

To test it I've made a 4 KiB text file with 4096 times 0 (the ascii character '0') written in it (and newline at the end).
I compressed it to a tar.gz file which takes 124 Bytes.
I've wrote two times as much 0s into the text file and compressed which resulted in 130 Bytes.
To figure out the overhead of the tar.gz file, I've written one 0 into the file and compressed it: 108 Bytes.
I've changed a few of the 4096 0s to another character, compressed it and got 140 Bytes (a few more characters changed resulted in 157 Bytes).

‮‪‮
‮‪‮
‮‪‮
‮‪‮
‮‪‮
‮‪‮
‮‪‮
‮‪‮
‮‪

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests