Package management in Minetest

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

Package management in Minetest

by MarkTraceur » Post

All right, it's high time we talked about this as a community.

1. It's 2012, why am I downloading mods from forum post attachments and shitty filesharing sites?
2. It's 2012, why are the packages I'm installing that support this game depending only on one or two mods, with dependency trees that look more like dependency pancakes?

We have the power to make an amazing package management system for Minetest. But we need to talk about a few things first.

1. What should it look like? Should it be a graphical thing? In-game (this means it would be in C++ or, if we first write a mod system for the main menu, Lua)?

2. Could we use an existing system? aptitude is a powerful one, and is used widely already, we could come up with a script that packages mods and a place to host them, then write or port an apt-get client as part of the minetest build process, or even write/port an external utility (c.f. Cydia) for the purpose of package management.

3. Where/by whom/how should it be hosted? Is a personal server (like the one at marktraceur.info) good enough? Would we need a dedicated server? Would we want a proper domain name, or would an IP address be good enough?

4. How should packages be tracked, reviewed, and maintained? Is the current system of "post whatever you'd like and it's in the annals forever" be enough of a gate to potentially malicious packages? Should there be MOTU-type people who review new packages and help maintain existing ones? Should this infrastructure also host git repositories and/or bug trackers for the mods? Should it also include an automatic package builder script, so a modder doesn't even need to package their own mods, and the server can handle it?

These are just thoughts, but if you have some opinion on this, we could use the help figuring it out. If you want to talk more one-on-one, I'm marktraceur on irc.gnu.org (which is the same as irc.freenode.net) and you should PM me.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

User avatar
RAPHAEL
Member
Posts: 627
Joined: Tue Nov 01, 2011 09:09
Location: Earth

by RAPHAEL » Post

NOTE: It's late, I'm tired, there may be issues with this post. You have been warned.

I agree there should be SOME FORM of mod package management. However it would likely be a lot of work. I remember before my almost year long leave some people were working on such a thing. Sad nothing came of it.

My personal suggestions is there should be some mod if possible or external program that could be run to check for updates to mods and install them if updates exist. However it should always keep a backup of one version below current update (in case new mod borks things). The utility should also make a proper backup of the world (in case mod update borks something). The utility should also check to see if mod package works with minetest version being used somehow (maybe package contain a mineversion file containing the known latest version of minetest that it works with?).

Once a utility and working test setup made, standards should be written on packaging mods. If a mod developer wants their mod added to repo, they need to post such in their mod post and some reputable users should verify the mod as "safe" and "working". That being said mod developers should also state the known latest version their mod works with if it is known.

Also there should be some form of GUI eventually. It also has to work on the platforms minetest works on. This is 2012 (almost 2013), console based update utility should be avoided if it is not needed. However free to offer compatibility with console for those who like using console.

Overall SOMETHING needs to be done as there are a LOT of mods out there now and theres a lot that need updating/doesn't work right/could use improving. More mods will come. The end user currently is responsible for first finding the mods they want, downloading, figuring out how to install, checking if its malicious or not, installing, firing up minetest and hoping it doesn't fail. A proper package system could help smooth over this ugly side of mods.

Also I think there should be a team of volunteers that find outdated mods and offer to update them if the developer has abandoned them. I have myself updated a few mods that I use personally and I'm not even overly familiar with lua yet. If I can do it I'm sure there can be others to help as well. For this I suggest a new forum board just for "mods updated but not by the developer" (meh not good with titles) where those volunteers can post the updated mods and be original post starter (so they can keep updating) or maybe keep current board setup and append maybe [updated] to post title. If append of new post title, a forum mod should mod outdated mod post with maybe [outdated].

For standards I suggest there be a VERSION file in mods containing the mod version number only in a form of maybe 0.0.0

There should also be maybe a MVERSION file with the latest version of minetest that the mod is known to work with in a form of maybe 0.4.4 (example).

There should also be some kind of structure setup for inside package. Should the package contents be in a folder with mod name or not? I suggest mod contents be in a folder. What format of archive? zip? 7z? tgz? tar.bz2? (I suggest 7z personally).

ALL standards concerning packages of mods should be spelled out in a single document somewhere. Everything down to naming of textures and files to include etc.

Anyway I've probably missed something but oh well. Post is done for now.
"Before you speak, ask yourself: Is it kind, is it true, is it necessary, does it improve upon the silence?"
My mods: http://goo.gl/n4kpn
(Currently Various, Industrial, Fakeblocks, Jail, MoarCraft, Christmas, Replicator, minetest dev installer for linux, bash mod installer, windows mod installer)

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

"Someone": Me! It wasn't a year ago, but here: http://minetest.net/forum/viewtopic.php?id=1928

I think some of the suggestions you've brought up are good, but it seems like we can do a bit better than using forums for package maintenance, bug reports, etc.. Setting up a Bugzilla instance, and maybe some code hosting, wouldn't be all that difficult.

As for stability of a package...I'd imagine that we could tag releases of mods to work with specific versions rather easily, the same way most GNU/Linux distributions will have different versions of a package for different releases of the distro. Backports happen for major bugs, but otherwise, if you want the new versions, you upgrade the entire system. The same thing could be done for the base Minetest system. Mostly this would foist the burden of testing for stability off on early adopters--people who run the unstable minetest will also run the unstable mods, and they'll be able to report bugs (again, hopefully not on a forum). This could all be easily spelled out in the control files of the packaging software, so that's a consideration we'll have to make while we're choosing whether to write something new or port something to our uses.

I agree the GUI should be secondary, since a GUI would leave out major servers from the user base, which is problematic.

7-zip seems like a nice project, but bzip2 and gzip are far more standard for free projects....at the very least, we should offer all three. Actually, GNU packages now offer .xz files, which are "a stripped-down version of 7-zip" according to the Wikipedia article. Maybe tbz2, tgz, and xz are the three we should offer? https://en.wikipedia.org/wiki/Xz
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

User avatar
Calinou
Moderator
Posts: 3169
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

by Calinou » Post

MarkTraceur wrote:aptitude is a powerful one, and is used widely already, we could come up with a script that packages mods and a place to host them, then write or port an apt-get client
aptitude/apt-get would be too much for a simple game like Minetest. Remember, not everyone uses GNU/Linux too so you'll have to port your whole thing to Windows (and maybe OS X) if you want it to be popular.
MarkTraceur wrote:Setting up a Bugzilla instance
Bugzilla is terrible, better use something like Trac or Redmine. 8)
MarkTraceur wrote:I agree the GUI should be secondary, since a GUI would leave out major servers from the user base, which is problematic.
We should have an optional GUI, else you can say "good bye" to end users using it. I understand using the command line is not that hard, but some people just run away whenever they have a command to type in a command line.


See http://minetest.net/forum/viewtopic.php?id=2568 -- I think an in-game mod manager would be the best solution.
As a compression format, why not use xz?

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

by rubenwardy » Post

I have made a Minetest Package online UI which I call "Minetest Extensions"

It is available here: http://www.multa.bugs3.com/minetest/forum/

Forum Topic

It supports 3m fully, and if sfan did some work to his mod manager, it would support that as well (the api is working on my thing, his thing can not connect to it properly).

As to package managers, it would be good to do that.

There are already a few mod managers:

Sfan's Python MM
Jeija's Ingame MM
3M C++ MM

But none of them are perfect (python & 3m are console based, ingame has a wierd gui)
Last edited by rubenwardy on Wed Dec 05, 2012 16:00, edited 1 time in total.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

OK, I'm glad that there's something like an in-game package manager already. I guess we just need to change it, or rewrite it, to our purposes.

Before that, it's important to write a good backend for the thing. This should almost certainly be included in minetest core, and should be built by default.

Calinou, apt-get et. al. are relatively easy to build on other systems, and if not, we can use Cydia's work to port it to other systems with relative ease. Even using some other system is fine, but I'd like to avoid the work of dependency resolution and similar bugaboos ourselves, when it's been solved before.

Bugzilla doesn't need to be a final solution. Trac is good, but not the best. How about Roundup? See an example instance at https://openhatch.org/bugs/ . But I've grown rather fond of Bugzilla in the past 6 months or so, I'd be happy to have it around for this purpose.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

mauvebic
Member
Posts: 1550
Joined: Fri Jan 27, 2012 11:32

by mauvebic » Post

MarkTraceur wrote: Should there be MOTU-type people who review new packages and help maintain existing ones?
Except where the license doesn't allow for it. Personally i dont want anyone else "maintaining" (ruining) my code lol weather or not im still active. But i am curious what role (if any) licensing will play. Cuz the present system seems geared towards protecting the community (cosmetic knockoffs, reposts) more than it protects the actual authors (who's interests should be paramount). I could use a system i can have more faith in (which is why i agree hosting mods here is a terrible idea). Perhaps then modders will stop removing all their stuff when they have disagreements with the management.
Last edited by mauvebic on Wed Dec 05, 2012 17:42, edited 1 time in total.

User avatar
RAPHAEL
Member
Posts: 627
Joined: Tue Nov 01, 2011 09:09
Location: Earth

by RAPHAEL » Post

mauvebic wrote:
MarkTraceur wrote: Should there be MOTU-type people who review new packages and help maintain existing ones?
Except where the license doesn't allow for it. Personally i dont want anyone else "maintaining" (ruining) my code lol weather or not im still active. But i am curious what role (if any) licensing will play. Cuz the present system seems geared towards protecting the community (cosmetic knockoffs, reposts) more than it protects the actual authors (who's interests should be paramount). I could use a system i can have more faith in (which is why i agree hosting mods here is a terrible idea). Perhaps then modders will stop removing all their stuff when they have disagreements with the management.
I think all mod devs should consider putting in their mod posts permission for the community to UPDATE their mods should they become absent. They could put restrictions on and define what is an update.

Me personally with my mods if I'm absent and my mod is out of date (doesn't work) and people enjoy it, I have no issue about it being updated to work. However taking over development of the mod may be an issue as I will never be absent forever from minetest (that I can see of anytime soon).
"Before you speak, ask yourself: Is it kind, is it true, is it necessary, does it improve upon the silence?"
My mods: http://goo.gl/n4kpn
(Currently Various, Industrial, Fakeblocks, Jail, MoarCraft, Christmas, Replicator, minetest dev installer for linux, bash mod installer, windows mod installer)

mauvebic
Member
Posts: 1550
Joined: Fri Jan 27, 2012 11:32

by mauvebic » Post

Image
this one is missing its original author too :p

Lemme try to understand, you want to grant a group of people the right to rewrite mods they judge to be abandoned (doesnt say if the user has disappeared, or left with his material, or been banned). Who and how these people are chosen, would i still want my name on the finished product, all good questions lol but it seems like your basically giving your mods and authorship to said committee. Instead of maintaining old mods you'll probably get more fights, more departures, and everything will become generic/unified/boring.

User avatar
Menche
Member
Posts: 1001
Joined: Sat Jul 02, 2011 00:43
IRC: Menchers
In-game: Menche
Location: An island in a lava lake.

by Menche » Post

RAPHAEL wrote: I think all mod devs should consider putting in their mod posts permission for the community to UPDATE their mods should they become absent. They could put restrictions on and define what is an update.

Me personally with my mods if I'm absent and my mod is out of date (doesn't work) and people enjoy it, I have no issue about it being updated to work. However taking over development of the mod may be an issue as I will never be absent forever from minetest (that I can see of anytime soon).
Don't the licenses of most existing mods allow other people to take up development or fork without special permission?
An innocent kitten dies every time you top-post.

mauvebic
Member
Posts: 1550
Joined: Fri Jan 27, 2012 11:32

by mauvebic » Post

CC BY-NC-ND doesn't allow for derivative works. some GPLs (and variations of it) also have explicit instructions in the areas being discussed here.

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

mauvebic, if you want to develop and release non-free mods, it's up to you to distribute them, I'm not going to help you.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

mauvebic
Member
Posts: 1550
Joined: Fri Jan 27, 2012 11:32

by mauvebic » Post

CC's and GPLs are all free licenses, some of them might not be as free as you need for this project, but thats the point of asking. My main concern is making sure mods that dont belong there don't show up, either by accident or not.
Last edited by mauvebic on Thu Dec 06, 2012 01:20, edited 1 time in total.

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

mauvebic, CC-ND and CC-NC are non-free, because they restrict the user from doing something. *GPL* are non-free in a sense, but only restrict people who potentially want to deny freedoms to others. If I have anything to say about it, NC and ND won't be allowed any more than a binary-only mod.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

User avatar
RAPHAEL
Member
Posts: 627
Joined: Tue Nov 01, 2011 09:09
Location: Earth

by RAPHAEL » Post

This is my preferred license for my mods:
http://sam.zoy.org/wtfpl/COPYING
"Before you speak, ask yourself: Is it kind, is it true, is it necessary, does it improve upon the silence?"
My mods: http://goo.gl/n4kpn
(Currently Various, Industrial, Fakeblocks, Jail, MoarCraft, Christmas, Replicator, minetest dev installer for linux, bash mod installer, windows mod installer)

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

by rubenwardy » Post

There should be made a standard with all this.

IE: Mods should use github or similar.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
Calinou
Moderator
Posts: 3169
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

by Calinou » Post

MarkTraceur wrote:mauvebic, if you want to develop and release non-free mods, it's up to you to distribute them, I'm not going to help you.
+1 -- if we have an official mod management thingy, it must include mods under a free license only (see how SuperTuxKart does things, it does not allow non-free licenses too). If non-free mods are allowed, then Minetest would not be allowed in the main repository of Debian packages. ;)

Also, mauvebic, why use the NC and ND clauses? Why would someone even attempt making money with your mod, and why is that bad to you? Same goes to derivative works.

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

OK, this is turning into the wrong set of questions.

If you'd like to chime in on package management, here are the few things I really need:

1. Are there existing systems we can use?

2. Would those systems work for all target platforms?

3. Would a self-rolled system work best for everyone?

4. Would a bug tracker and code hosting solution be a good idea on top of all that, or is it too ambitious?
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

mauvebic
Member
Posts: 1550
Joined: Fri Jan 27, 2012 11:32

by mauvebic » Post

Calinou wrote: Also, mauvebic, why use the NC and ND clauses? Why would someone even attempt making money with your mod, and why is that bad to you? Same goes to derivative works.
NC - because i did it for free and should stay as such.

ND - 2/13 mods - because i write mods to be enjoyed by the players. If i thought people would improve the code and not just rebrand part of it then i wouldn't need the clause. I also wasted a lot of time trying to differentiate multinode, then worldedit had a rewrite which cancelled out those efforts. This time i'd like it to stay different. I dont know what else you'd have me do to maintain some measure of uniqueness, it was a moderator who suggested using a different license in the first place.
Last edited by mauvebic on Thu Dec 06, 2012 18:39, edited 1 time in total.

streondj
Member
Posts: 11
Joined: Wed Dec 05, 2012 08:06

by streondj » Post

mauvebic wrote:
Calinou wrote: Also, mauvebic, why use the NC and ND clauses? Why would someone even attempt making money with your mod, and why is that bad to you? Same goes to derivative works.
NC - because i did it for free and should stay as such.
Ya, I kinda agree with that, look at how Apple ripped off freeBSD,
using open source to further their proprietary ends,
with all their nets to catch jumpers and all..

That was due to lack of a share-alike clause however.

Commercial can be good certainly.
I like GPL since it allows for charging for the medium,
I think that's the best thing, like for books, CD's and that kinda stuff.

That way for instance can have subscription MMORPG minetest servers that charge for bandwidth.
ND - 2/13 mods - because i write mods to be enjoyed by the players. If i thought people would improve the code and not just rebrand part of it then i wouldn't need the clause. I also wasted a lot of time trying to differentiate multinode, then worldedit had a rewrite which cancelled out those efforts. This time i'd like it to stay different. I dont know what else you'd have me do to maintain some measure of uniqueness, it was a moderator who suggested using a different license in the first place.
How about your children, aren't they a derivative work?
And you certainly wouldn't force them to have the same name as you.
Besides your always unique in your own way.

I look at ND, proprietary and copyrights as voluntary sterilization.
Indeed look at Bill Gates, going around sterilizing with vaccines.

In conclusion,

CC-BY-SA is the best
Last edited by streondj on Thu Dec 06, 2012 20:34, edited 1 time in total.

User avatar
RAPHAEL
Member
Posts: 627
Joined: Tue Nov 01, 2011 09:09
Location: Earth

by RAPHAEL » Post

To get back on topic, how about two volunteer teams. One for Linux one for Windows (and who knows another any any other os that support is for). Each team focus on a package manager for their os. OR everything get together and work on an updater/installer mod?
"Before you speak, ask yourself: Is it kind, is it true, is it necessary, does it improve upon the silence?"
My mods: http://goo.gl/n4kpn
(Currently Various, Industrial, Fakeblocks, Jail, MoarCraft, Christmas, Replicator, minetest dev installer for linux, bash mod installer, windows mod installer)

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

RAPHAEL, I think it's feasible to build a system that will work with both platforms, probably something built in C++ that just fetches packages and reads them locally. It shouldn't be too difficult to then let that be tied into from a main menu mod.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

by rubenwardy » Post

Like 3m, but with a GUI *

Minetest Extensions is the best repo for mods at the moment.

* (graphical user interface)
Last edited by rubenwardy on Fri Dec 07, 2012 13:26, edited 1 time in total.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

MarkTraceur
Member
Posts: 103
Joined: Sat Dec 03, 2011 05:41
Location: San Francisco, CA
Contact:

by MarkTraceur » Post

"best" meaning....?

I mean, it's nice, and it could be useful, but it doesn't seem to have any API, it's (apparently) a totally new system, and from what I can tell it's not free software. One of the better options still seems to be a debian-style package manager and repository.
Mods: https://gitorious.org/marktraceur-minetest-mods
IRC: marktraceur on freenode

User avatar
Jeija
Member
Posts: 686
Joined: Fri Dec 23, 2011 21:46
Location: Nürtingen, Germany

by Jeija » Post

I may have solved some of your questions for my ingame modmanager (that rubenwardy mentioned already): http://minetest.net/forum/viewtopic.php?id=2568
1. What should it look like? Should it be a graphical thing? In-game (this means it would be in C++ or, if we first write a mod system for the main menu, Lua)?
I thought it must be graphical so that people can understand it. On the other side, Lua is more customizable. Therefore I splitted the code up, GUI in C++, the actual function in Lua, This also solves the other problems/questions:
3. Where/by whom/how should it be hosted? Is a personal server (like the one at marktraceur.info) good enough? Would we need a dedicated server? Would we want a proper domain name, or would an IP address be good enough?
I think the person who writes the "Game" (like minetest_game, minimal) should also host the mod server or link to a mod server. I made it as customizable as possible: The lua script just has to support the basic functionality like refereshing the modlist, (un)installing a mod, ... Therefore People can just use the system they like the best as long as they can acces it in lua.
4. How should packages be tracked, reviewed, and maintained?
My proposal also solves this problem: By the game owner. For the default minetest_game I'd say there should be a small number of devs that work on the package management.

Problems: Someone needs to port the mod manager to windows + Mac OS, I can't do that since I don't own these OS. I guess it shouldn't be too hard for someone that is used to these OS.
I can do all the rest like auto-update and resolving dependencies.

I hope these ideas could help you. See the official modmanager thread (link above) for more information.
Last edited by Jeija on Fri Dec 07, 2012 22:26, edited 1 time in total.
Redstone for minetest: Mesecons (mesecons.net)

Post Reply

Who is online

Users browsing this forum: No registered users and 12 guests