plugins (by sorcerykid)
Today I am proud to announce the initial beta release of Pluggable Helpers with the goal of encouraging the cooperative development of helper functions in Minetest mods and games. Pluggable Helpers automates the entire process from downloading to installation, including management of dependencies, within a distributed architecture backed by the BitBucket Cloud for secure version control.
I developed this mod in order to address several of the concerns that were raised here and here.
Repository:
https://bitbucket.org/sorcerykid/plugins
Download Archive (.zip)
Download Archive (.tar.gz)
Dependencies:
None.
Source Code License:
The MIT License
Installation:
- Unzip the archive into the mods directory of your game.
- Rename the markup-master directory to "plugins".
- Add "plugins" as a dependency to any mods using the API.
Modularity is extremely important when it comes to maintaining large scale code-bases such as games and mods in Minetest. Helper classes and methods and even libraries serve this purpose. But so often they are re-implemented over-and-over again since nobody wants to rely on external dependencies in their mods and games. Eventually some helper functions may be integrated into the engine, but even that is often a lengthy review process with its own share of obstacles and pitfalls.
This is where Pluggable Helpers comes into the picture.
- "The core philosophy of Pluggable Helpers is to empower the community to create an evolving game-development API through the use of a jointly maintained public repository of helper classes, methods, and libraries that can be downloaded and installed on-the-fly with no intervention required by the end-user."
The remainder of this document pertains to mod and game developers only. End-users should never need to be concerned with these implementation details. The Pluggable Helpers mod is literally plug and play.
Getting Started
Let's begin with a simple helper definition. We'll define a new "math.square" helper method, that returns the square of any given number:
Code: Select all
author = "sorcerykid"
version = "1.0"
license = "MIT"
depends = { }
imports = { }
return function ( v )
return v ^ 2
end
Now, on my local machine, I can easily incorporate this helper function into my mod as follows:
Code: Select all
plugins.require( "math.square" )
print( math.square( 2 ) )
On subsequent server startups, of course, the helper will be loaded directly from my local repository, so there is no requirement to even be connected to the Internet. The entire process is very efficient, almost imperceptible from an end-user perspective.
Supported Helper Types
There are three types of helpers available: methods, classes, and libraries. Each is intended for a specific purpose:
- Helper Methods
A helper method extends one of the builtin global libraries, whether it be "minetest", "string", "math", "table", etc. with an additional custom method. The earlier example of "math.square" would be considered a helper method since it extends the global math library. Once a helper method is required within any mod, then it is made available to all other mods. If another mod requires the same helper method, then it will not be loaded twice.
A helper method must be named in snake_case notation. Only lowercase letters, underscores, and numerals are permitted.
Helper Classes
A helper class is a class constructor that returns a new instance of an object with its own methods and properties. It is usually stateful in natture, and may even be derived from a base class. Once a helper class is required within a mod, then will be available to all other mods just as with helper methods.
A helper class must be named in CamelCase notation. Only letters and numerals are permitted.
This is an example of a helper class definition called "StaticSet", for easily working with immutable membership sets.
Now, adding a membership set to a mod is as simple as a few lines of code.Code: Select all
author = "sorcerykid" version = "1.0" license = "MIT" depends = { } imports = { } return function ( list ) local data = { } local self = { length = 0 } for i, v in ipairs( list ) do data[ v ] = true end self.length = #list self.exists = function ( elem ) return data[ elem ] ~= nil end end
Helper LibrariesCode: Select all
plugins.require( "StaticSet" ) local ranks = StaticSet { "basic", "staff", "admin", "owner" } print( ranks.exists( "basic" ) )
Helper libraries are a special type of persistent helper class that is instantiated at server startup. They can serve a dual purpose of either being a barebones library for consolidating related helper methods or else a fully-fledged module that encapsulates various public and private methods and properties. Unlike helper methods and classes, libraries are always included rather than required.
A helper library must be named in CamelCase notation. Only letters and numerals are permitted.
This is an example helper library called "DebugLog" that provides several methods for posting messages to the Minetest debug log, including statistics of how many warnings were posted.
Placing this code into a mod and running Minetest will print "Total warnings: 2" to the console:Code: Select all
author = "sorcerykid" version = "1.0" license = "MIT" depends = { } imports = { } prototype = "" return function ( this ) local warning_count = 0 this.post_warning = function ( message ) minetest.log( "warning", message ) warning_count = warning_count + 1 end this.post_action = function ( message ) minetest.log( "action", message ) end this.get_warning_count = function ( ) return warning_count end end
Code: Select all
local logger = plugins.include( "DebugLog" ) logger.post_warning( "You made a boo boo!" ) logger.post_action( "We're doing something now" ) logger.post_warning( "Oops, I did it again!" ) print( "Total warnings: " .. logger.get_warning_count( ) )
Conflict Resolution of Helper IDs
One of the primary reasons that I developed Pluggable Helpers was to resolve the very obvious potential for code duplication whenever the Minetest API is expanded. A good example of this situation was the introduction of vector.dot and vector.cross in Minetest 5.0.
Up until recently, any mods that needed these helper functions had no choice but to define them manually. Of course they are now both redundant. Thankfully, Pluggable Helpers provides for conflict resolution in this situation.
At startup, each required helper ID is checked against the global environment. If the method or class is already defined by the builtin Lua or Minetest API, then that function will be used. Otherwise, the corresponding helper will be downloaded from the remote repository. This provides a seamless means to backport newer API functions for legecy versions of Minetest.
Let's say that I wrote a fancy new mod that depends on minetest.encode_base64, yet I just realized that Minetest 0.4.14 does not support that API function even though that's among my target userbase. I could simply write a Lua variant of the Base64 encoder and submit it as a helper method. Now it takes just one line of code to ensure my script will run properly regardless of the Minetest version.
Code: Select all
plugins.require( "minetest.encode_base64' )
print( minetest.encode_base64( "Wow! This will even work in Minetest 0.4.14!" ) )
For security reasons, helper functions are always executed within a restrictive environment. The Pluggable Helpers sandbox includes the builtin Lua functions shown below in addition to all of the core Lua and Minetest libraries (such as math, string, vector, io, etc.), and any dependencies.
- next
- pairs
- ipairs
- assert
- error
- dofile
- loadfile
- loadstring
- getmetatable
- setmetatable
- pcall
- rawequal
- rawget
- rawset
- select
- tonumber
- tostring
- type
- unpack
- dump
- plugins.register_class( name, ref )
Adds the given library or class constructor to the global helper environment, so that it is accessible to all helper functions. The "ref" parameter must be a table reference in the case of libraries or a function reference in the case of class constructors. The "name" parameter is the name or an alias by which to make the library or class constructor available within the helper environment. It need not be the original name, but it must adhere to the naming conventions previously described.
Using Helpers in Mods
The following API functions should only ever be called at startup, otherwise they will raise an exception. Hence, they are best placed at the head of your script.
- plugins.include( id )
Returns a reference to the helper library with the given ID. If the library is not found within the local repository, then it will be asynchronously downloaded and installed.
Note that helper libraries are purposefully not added to the global environment. You should therefore store the result into a local variable. There is no penalty for including the same helper library in multiple Lua scripts. In fact, that is the correct way to use this API function.
plugins.require( id )Code: Select all
-- Correct usage in init.lua: local mylib = plugins.include( "MyGreatLibrary" ) -- Incorrect usage in init.lua mylib = plugins.include( "MyGreatLibrary" )
Returns a reference to the helper method or helper class with the given ID. If the class or method is not found within the local repository, then it will be asynchronously downloaded and installed.
As mentioned previously, helper methods and helper classes are both available globally. It is entirely acceptable to ignore the return value of this API function and use the helper class or method globally. But you can localize the reference for efficiency. Both of the following examples are the same, but the latter takes advantage of the shorthand.
Code: Select all
plugins.include( "math.square" ) local square = math.square print( square( 2 ) ) local square = plugins.include( "math.square" ) print( square( 2 ) )
All helpers must adhere to the SemVer standard, with the first numeral designating a major release and the second numeral, a minor release. This makes it possible to automatically distinguish between version changes.
To require or include a specific version of a helper, simply suffix the ID with a slash and the version tag. For example if only version 1.0 of "math.square" is installed in the local repository and your mod requires version 1.1, then the latest version will be downloaded and installed from the remote repository. For this reason, helpers must always be backward compatible to avoid breakage, as only a single helper can be installed at one time.
For convenience, both of the API functions above allow for splitting the ID and version into a two element array. So all three of these examples are acceptable:
Code: Select all
plugins.require( "math.square/1.1" )
plugins.require "math.square/1.1"
plugins.require { "math.square", "1.1" }
plugins.require( { "math.square", "1.1" } )
One of the central tenets of Pluggable Helpers, is to streamline the review process so that additional functionality can be made available to games and mods within a reasonable timeframe. Obviously, I want to provide developers the greatest degree of creative freedom, but without unduly sacrificing the security and integrity of games and mods that rely on this shared code-base.
All helper methods, classes, and libraries will therefore be vetted for compliance with several core guidelines prior to being published. The goal is to limit any chance of an untrusted helper going rogue.
- Helpers must be useful
A helper function must have some degree of utility, and it must do exactly what its name implies without unanticipated side-effects. For example, a helper called "math.add" that subtracts two numbers and prints the result will be rejected. - Helpers must be generic
Generally speaking a helper function should be as generic as possible, unless it is intended to perform a common and repetitive task that would be best consolidated into a function. For example, a helper called "math.square" that squares a given number may be generic enough for inclusion. But, a helper called "math.multiply" that returns the product of 5 x 4 would be rejected. - Helpers must be secure
It should go without saying that all helper functions must respect the security and integrity of the target installation environment. Any attempt to defeat the Pluggable Helpers sandbox or to override the builtin security model of the Minetest engine (client or server) or to abuse the Pluggable Helpers infrastructure will be grounds for rejection. - Helpers must be open-source
In order to build an effective community-focused API for Minetest, it is essential that every helper function be objectively compatible with the principles of FOSS. Any attempt to obfuscate or encrypt or copy-protect source code or to impose a copyright clause that inhibits free distribution and adaptation will be grounds for rejection - Helpers must be concise
The goal of a helper method or class is to perform a small task and to do it in the most efficient and effective way possible. Embedding an entire game into a single function is obviously impractical. Note that this limitation does not apply to helper libraries, which may be as sophisticated as the target application demands (see above)
Also keep in mind that no single coding style is imposed on submissions (other than the aforementioned naming conventions). But, please try to ensure that your source code is easily readable so thatit can be jointly maintained by the community.
The Lua Code Guidelines for Minetest are a good starting point. While not enforced in this project, they are strongly encouraged.
Unit-Testing of Helpers
Prior to submitting any helper for review, it is important to unit-test it first. This can be accomplished by creating a helper definition and manually installing it into the local repository. The "source" subdirectory within the plugins mod directory is the default location for the local repository, but this may be changed to a symbolic link if needed.
A helper definition must be a plain-text file with read and write permissions by the Minetest process. It is named exactly as the helper ID and consists of a Lua snippet in the following format:
Code: Select all
author = "testuser"
version = "1.2"
license = "GPLv3"
depends = { "math.min", "math.max" }
imports = { "math.min", "math.max" }
return function ( v, v_min, v_max )
return min( v_max, max( v_min, v ) )
end
Important: Be certain to require or include any test helpers in at least one mod (or else list them as dependencies) before launching Minetest, otherwise they will deleted from "source" subdirectory during startup.
Layout of a Helper Submission
A helper submission is a plain-text file describing the proposed helper. It is the precursor to a helper definition which is the Lua snippet that the Pluggable Helpers Mod downloads and installs from the remote repository.
A valid helper submission consists of 5 fields and an anonymous function at the end of the file.
- Author - This is author's registered username on the Content DB
- License - This is a valid FOSS license as listed on the Content DB
- Version - This is the SemVer tag corresponding to the current release
- Depends - This is a list of helper methods, helper classes, or helper libraries that must be installed prior to this helper being required or included (optional)
- Imports - This is a list of helper methods, helper classes, or builtin library methods that are to be imported into the helper's environment when it is required or included (optional)
- Prototype - This is a special sixth field that is needed only in the case of helper libraries. It allows for specifying a base class for the constructor function to extend at startup.
Code: Select all
Author:
sorcerykid
License:
MIT
Version:
1.0
Depends:
Imports:
function ( )
end
Project Goals
In the short-term, I would really like to bring on-board a team of trusted community members to assist with the review of new submissions. Obviously, Rubenwardy, as the maintainer of the ContentDB itself, and the participation of other core developers and active contributors would be most welcome and encouraged for this project to be successful.
Eventually, it may be worth migrating the remote repository and the submission server to a subdomain on minetest.net, at least if it becomes popular enough. That would not only make it more official but also ensure that it truly is a community endeavor. Of course this need not happen for the interim. I am more than happy to maintain the repository on a volunteer basis in much the same way as sofar hosts the Public Remote Media Server Project independent of Minetest.
Right now I am also working on bringing a Wiki online, so that all approved helper functions can be easily documented. It will be located at http://plugins.mytuner.net/wiki/ when ready.