[Mod] Node-IO API [node_io] WIP

[auouymous]

This mod attempts to provide a single API for transfer mods (hoppers and pipeworks) to access inventory nodes (chests and furnaces). Any inventory node that supports the API will work with any transfer mod that also supports the API. It provides many utility functions for inventory nodes to implement the API and for transfer mods to access them. Inventory nodes that lack standard inventories can also use the API to work with transfer mods. The API currently supports item and liquid inventory nodes, but could be extended to support anything that can be transfered between nodes (signals, power, ...).

Are any inventory mod or transfer mod authors interested in using something like this? Do you have suggestions to improve the API and make it more likely to be adopted?

Dependencies: default?
License: CC0-1.0
Download: https://github.com/auouymous/node_io/archive/master.zip
Source: https://github.com/auouymous/node_io

[Sokomine]

There's the pipeworks mod, and there are other similar mods. Perhaps you ought to advertise your library in those threads so that it will get widely used? It's a pity that libraries are so often forgotten about.

[IcyDiamond]

Hello, auouymous.

I am the developer of the up-and-coming mods Elepower, which, among other things, deals with fluid transfer and fluid_lib that deals with fluid inventory nodes. I would be interested in coming up with a solution to integrate our APIs seamlessly, because Elepower absolutely requires fluid inventories and transfer between them. The other option would be that this API becomes the universal API for all things related to *transfer and I simply make Elepower support it. I would like to hear from you.

[auouymous]

I am the developer of the up-and-coming mods Elepower, which, among other things, deals with fluid transfer and fluid_lib that deals with fluid inventory nodes.

I was given your name on irc and added support for millibuckets after looking at fluid_lib. But I then discovered a small issue with the way quantities are handled in room_for_* and put/take_* in the API. I was going to contact you and others as soon as I resolve this issue.

[auouymous]

The API has been fixed and updated.

• The node_io_room_for_* functions return a number upto the value requested instead of a boolean.
• The node_io_room_for_item function now has a count parameter and ignores the count in itemstack.
• New node_io_get_*_stack functions return a copy of the itemstack (or a fake itemstack) with count always set to 1.
• The want_item parameter for node_io_take_item function now accepts a string or itemstack.
• New node_io_accepts_millibuckets function returns true if the liquid inventory node supports millibuckets. All liquid quantities are always in millibuckets but nodes that don't support millibuckets require increments of 1000.

[ChimneySwift]

This looks very interesting, I like the idea of a standardized library for node IO like this, there are a lot of different pipeworks-like mods emerging and supporting them all gets complicated. I'll be adding support for this to fancy_vend when I get a chance.

[rubenwardy]

Looks good though

[joe7575]

Hi,

I am the developer of the Tubelib/TechPack, a transfer mod similar to pipeworks and I like the idea of a standardized node API. Unfortunately for me it comes one year to late.
But perhaps we still can bring both APIs together....
FYI, you will find my API here

[orwell]

Hello, too,
I seem to be the first one from the other side, inventory receivers, namely loading and unloading rails from advtrains. I've seen an issue with your mod and reported it on github, otherwise I really like the idea.
I really hope this will be accepted in h the community.

[Desour]

The pipeworks api might be of interest.

Have you thought of using one `node_io` table containing all the callbacks in node definition?

[IcyDiamond]

Have you thought of using one `node_io` table containing all the callbacks in node definition?

I +1 this

[auouymous]

I've seen an issue with your mod and reported it on github, otherwise I really like the idea.

The helper functions now support detached inventories.

Have you thought of using one `node_io` table containing all the callbacks in node definition?

Do you mean grouping the callbacks in a table instead of prefixing each one with "node_io_"? The helper functions would require an additional check (slowdown) for the node_io table, and I imagine most transfer mods would use them instead of faster raw access. Is there a good reason for doing this?

[Desour]

Have you thought of using one `node_io` table containing all the callbacks in node definition?

Do you mean grouping the callbacks in a table instead of prefixing each one with "node_io_"? The helper functions would require an additional check (slowdown) for the node_io table, and I imagine most transfer mods would use them instead of faster raw access. Is there a good reason for doing this?

Yes, I mean.
I don't think, this would be a significant slowdown.
By the way, is it slower to access from the not array part of a big table compared to of a small one?
Does it even make sense adding such convenience functions to call the callbacks? Laziness for the price of efficiency?
A reason would be that the node definition would be more structured. Structure is convenient, to write and to read.
Also pipeworks (and mesecons and digilines) does the same, so modders might be used to do it like this. Ergo more mods will use the api.

[orwell]

I would appreciate this too.

[auouymous]

I don't think, this would be a significant slowdown. By the way, is it slower to access from the not array part of a big table compared to of a small one? Does it even make sense adding such convenience functions to call the callbacks?

The main table and sub-tables are both hash tables, they only slow down when the buckets become unbalanced. Hashing a longer field name with "node_io_" prefix is faster than hashing the "node_io" table name querying it from main table, hashing field name and then querying it from the node_io sub-table. The other issue is an extra branch check for the ndef.node_io table. Transfer mods using raw access only need to do this one time per transfer, but mods using helper functions would end up doing it numerous times per transfer. Of course the helper functions also do many other things numerous times per transfer already.

The helper functions are nice for quickly adding support to transfer mods, but really shouldn't be used long-term in any serious mods. I will mention this in the API file.

A reason would be that the node definition would be more structured. Structure is convenient, to write and to read.

Readability is a good reason, I'll probably push out the change in a day or two, along with a patch that adds full node-io support to pipeworks and maybe some additional API breaking changes. While I'm breaking the API, is there anything else I should change?

[Desour]

While I'm breaking the API, is there anything else I should change?

• The side is a string (NESWDU). Is this more useful than a vector? Should there be a side string or vector that respects the rotation of the node? I don't know. Are there helper functions to make a side to a vector and vice versa? And maybe helpers to help mods with the rotations of their nodes?
  -
• The callbacks need more parameters, some optional. Eg. pipeworks has putter and itemstack also for the can callbacks.
  --
• Also pipeworks has connected_sides. Maybe rules like the one from mesecons would make sense (think of nodes that span several nodes and co.).
  -
• I don't really like the unit "millibuckets". (It's actually mm³ ("millicubicmeters") which is the same as liters. (If 1 node = 1m)) mc has buckets in the default game that you can not deactivate afaik. Minetest has buckets only in its subgames (see mtg). Should this api be specialized to mtg? Also "millibuckets" is long to write.
  ---
• pipeworks uses return_input_invref or decides on its own to choose from what inventory to put. Nodes should choose on their own where to put the item they get to. But if another node takes from a node, the taking node (or any other taker) might want to decide on its own from where to take. There should be a callback to give an invref and a listname to help the taker.
  -
• Perhaps the callbacks should get custom string parameters where mods can add some information that might be handled by the getter or taker. (Think of a fuel putter device that adds "fuel".) Just don't use the 4th or whatever parameter and call it "custom".
  --
• The putter is a fake player it's documented. I first thought that was the equivalent to pipeworks' `owner` which is important. What is the putter for? Would it make sense to add a helper to create a fake_player?

[IcyDiamond]

I don't really like the unit "millibuckets". (It's actually mm³ ("millicubicmeters") which is the same as liters. (If 1 node = 1m)) mc has buckets in the default game that you can not deactivate afaik. Minetest has buckets only in its subgames (see mtg). Should this api be specialized to mtg? Also "millibuckets" is long to write.

• Millibuckets makes perfect sense for a game like Minecraft/Minetest. It is not meant to be scientifically accurate and technology mods need a fluid unit that's not equal to a bucket.
• In what way is millibuckets longer to write than millicubicmeters? Also, it's abbreviated to mB which is much better than mm³, imo.

[joe7575]

The side is a string (NESWDU). Is this more useful than a vector?

I like the side approach, but from the point of view of the node. In this case it is independent on the rotation of the node.

In Tubelib I use the side parameter, too. It simplifies the API to the node. Here an example to adapt the default furnace to Tubelib. The side parameter in this example is used to distinguish between fuel and other items.

Spoiler

Code: Select all

--                        Up    Back
--                        |    /
--                        |   /
--                     +--|-----+
--                    /   o    /|
--                   +--------+ |
--          Left ----|        |o---- Right
--                   |    o   | |
--                   |   /    | +
--                   |  /     |/
--                   +-/------+
--                    /   |
--                 Front  |
--                        |
--                      Down

tubelib.register_node("default:furnace", {"default:furnace_active"}, {
	on_pull_item = function(pos, side)
		local meta = minetest.get_meta(pos)
		return tubelib.get_item(meta, "dst")
	end,
	on_push_item = function(pos, side, item)
		local meta = minetest.get_meta(pos)
		minetest.get_node_timer(pos):start(1.0)
		if side == "D" then
			return tubelib.put_item(meta, "fuel", item)
		else
			return tubelib.put_item(meta, "src", item)
		end
	end,
	on_unpull_item = function(pos, side, item)
		local meta = minetest.get_meta(pos)
		return tubelib.put_item(meta, "dst", item)
	end,
})	

BTW: This API follows the KISS principle (Keep it simple, stupid) to simplify the adaption for other mods. You should keep this in mind, when you define your Node-IO API.

[Desour]

In what way is millibuckets longer to write than millicubicmeters? Also, it's abbreviated to mB which is much better than mm³, imo.

millibuckets <-> liters
mB <-> l

(mB == milliByte??¿¿)

I'm not totally against millibuckets and not for liters (and of course not for millicubicmeters or cubicdecimeters).
What about unit for energy = eu => unit for volume = vu => millivolumeunits (mvu)? Does mvu sound worse than mB?
Or what about mn (millinode)?



To the side thing:

Maybe rules like the one from mesecons would make sense

If this was done, sides wouldn't be enough anymore, a vector would be needed.



I'm sorry for being so penetrating but an api that everybody should use has to perfect.

[auouymous]

Thank you! I was already looking into the first two and your feedback has reinforced my decision.

The side is a string (NESWDU). Is this more useful than a vector? Should there be a side string or vector that respects the rotation of the node? I don't know. Are there helper functions to make a side to a vector and vice versa? And maybe helpers to help mods with the rotations of their nodes?

I originally chose the NESWDU string for performance reasons, but I don't remember what I was benchmarking it against. I am benchmarking some new ideas that can be rotated with facedir and easily check sides by translating to top,bottom,right,left,front,back.

The callbacks need more parameters, some optional. Eg. pipeworks has putter and itemstack also for the can callbacks.

I noticed the pipeworks version of can_put_item() takes itemstack and returns a value, which is identical to node-io's room_for_item(). It also requires more checks to test for presence of can_put_item and room_for_item, so I am merging the two callbacks into one. The can_put_* callbacks will have optional stackitem and count parameters. When stackitem item is nil, the callback works as it currently does, otherwise it works like room_for_item. This allows a pipe to test if items can be put in the inventory and display a connection, even if the inventory is full.

Also pipeworks has connected_sides. Maybe rules like the one from mesecons would make sense (think of nodes that span several nodes and co.).

Could you elaborate? Pipeworks only appears to put items in inventories with its pipe. My patch calls can_put_item and displays the connection, so no need for a connected_sides setting. And pipeworks connected_sides doesn't appear to support facedir, but can_put_item would. If a pipe can put or take, it just tests can_put_item and can_take_item, and liquid pipes would use the liquid callbacks.

I don't really like the unit "millibuckets". (It's actually mm³ ("millicubicmeters") which is the same as liters. (If 1 node = 1m)) mc has buckets in the default game that you can not deactivate afaik. Minetest has buckets only in its subgames (see mtg). Should this api be specialized to mtg? Also "millibuckets" is long to write.

My opinion on this is the same as IcyDiamond's.

pipeworks uses return_input_invref or decides on its own to choose from what inventory to put. Nodes should choose on their own where to put the item they get to. But if another node takes from a node, the taking node (or any other taker) might want to decide on its own from where to take. There should be a callback to give an invref and a listname to help the taker.

Node-IO was designed to put the inventory node in control, because some inventories don't use an invref. When a transfer node puts an item in a side, it is up to the inventory node where that items goes. Taking anything is the same as putting, you just ask for something from a side and the inventory chooses what you get. But a transfer node with a filter or inventory needs to know what it will get to either filter the take or verify the take will fit in its own inventory. So Node-IO has read-only callbacks to query the contents of fake inventories attached to a given side. The transfer node can then call take_item with a stackitem that filters exactly what it receives.

I was thinking about adding named inventories though. A transfer node would query available named inventories and then request a specific inventory to put or take. A smart pipe could be connected to any side of a furnace and the player could configure the pipe to filter fuels to the fuel inventory, ores to the src inventory and take from the dst inventory. These named inventories don't represent invrefs, they can be fake or have different names than the invrefs. And the inventory remains in control, if the furnace only wants to receive fuel from top it will only expose the named fuel inventory on that side. Dumb pipes would operate as they currently do without named inventories.

Is this something I should add?

Perhaps the callbacks should get custom string parameters where mods can add some information that might be handled by the getter or taker. (Think of a fuel putter device that adds "fuel".) Just don't use the 4th or whatever parameter and call it "custom".

I'm not sure what you are asking for.

The putter is a fake player it's documented. I first thought that was the equivalent to pipeworks' `owner` which is important. What is the putter for? Would it make sense to add a helper to create a fake_player?

Putter and taker is for logging and allow_metadata_whatever callbacks. I don't see the point of pipeworks' owner parameter, it is only for putting but not taking. Hoppers allow putting in locked chests but won't take from them, makes sense if the locked chest isn't protected and you want someone to place a hopper and fill it up. Pipeworks' owner could be used by locked chest to only allow a pipe placed by the chest owner to insert items, but it doesn't pass owner when removing which could have been used to allow an injector to take if both have same owner. I don't think node-io should care who owns what, it doesn't matter for single player, and you should be using protection mods on multiplayer. And multiple players sharing a protection will have issues if one of them places inventories and the other places the pipes. Let me know if this is wrong and there is a valid reason to pass owner.

A utility function to create fake players isn't needed. A mod either has a real player or it passes nil. And any mod that stores owner in nodes probably already has a way to create fake players for its own logging/allow_metadata_whatever.

[auouymous]

Forgot to answer these in my wall of text.

In this case it is independent on the rotation of the node.

A transfer node only knows about NESWDU and only the inventory node can translate NESWDU to FBLRDU because it might not use a standard facedir. I'll be adding a helper function so an inventory node can specify which NESW direction its front is and then translate the NESWDU side to FBLRDU.

I'm not totally against millibuckets and not for liters (and of course not for millicubicmeters or cubicdecimeters). Or what about mn (millinode)?

Most Americans don't know what a liter is or even what milli means. All players do know what a bucket is and it makes perfect sense to use it as the measurement of liquids. Modders are more likely to understand milli or can easily learn. It is also easy to tell everyone that 1000mB is 1 bucket and 500mB is half a bucket. The mB unit name has been around for years in minecraft, and probably the same in minetest, you can't change it now.

What about unit for energy = eu => unit for volume = vu => millivolumeunits (mvu)?

Energy can be registered like a liquid, transfered in liquid pipes and stored in liquid tanks or buckets. Doing so forces it to use mB, because that is what liquid nodes display.

Or node-io could get an energy API identical to its liquid API. The liquid and millibuckets parameters become energy and quantity/count/whatever. The energy name would be something like "buildcraft:MJ" but nodes that display energy would strip off the colon and everything before it, displaying "50 MJ". "buildcraft:MJ" and "industrialcraft:EU" could be transfered over the same energy pipes, just as lava and water can over liquid pipes, but not at the same time. Every mod that creates its own energy gets to name it, so MJ for buildcraft, EU for industrial craft and RF for thermal expansion. I could create a machine and have it optionally depend on multiple energy mods and accept energy from any of them. But MJ might be easier to create than EU, so each operation requires 75MJ or 50EU. Energy is like liquid and food, lots of flavors. Try telling mod authors that only one food type is needed in the game... ;)

I'm sorry for being so penetrating but an api that everybody should use has to perfect

Don't be sorry, we all want a perfect API, and feedback is required for that to happen.

[Wuzzy]

Without API documentation this mod is useless.

[orwell]

Without API documentation this mod is useless.

File "API"?

[Wuzzy]

Oops, I did not see it. The existance of this file should be mentioned in README.md, however.

EDIT: Oh, man. The API documentation is horribly written. Half of the functions are not described, and it is very unstructured. Terms like “millibuckets” appear out of nowhere.

[joe7575]

@Wuzzy: I assume, this is highly under construction. He in in a sandwich situation, pressure from both sides, a) to keep the API as simple as possible, but b) with the possibility to support all kinds of transportation/technic mods...