[Tool] GUI for installing and updating mods from GitHub

[Morn76]

In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

It would be great if more modders could add their mods there. Installing from the mmdb is pretty easy.

Yes, installation is easy, but you never get updates, do you? Mmdb mods are distributed as ZIP files.

[dgm5555]

Mmdb mods are distributed as ZIP files.

zips irritates me too, one of the reasons I like MT-GitSync

[Morn76]

Mmdb mods are distributed as ZIP files.

zips irritates me too, one of the reasons I like MT-GitSync

And that's also why I think the approaches of mmdb and MT-GitSync are fundamentally incompatible. Pretty much all mods in active development are on GitHub already and those which are not in active development will probably stop working after the next API change. So why not use GitHub for distribution?

Also, there are situations where mods interfere with each other, but at least you can only hope for an update to fix that. E.g. I'm still waiting for 3d_armor and wieldview to work together properly. :-) So I think updates are as important for mods as for the main game.

[webdesigner97]

In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

It would be great if more modders could add their mods there. Installing from the mmdb is pretty easy.

Yes, installation is easy, but you never get updates, do you? Mmdb mods are distributed as ZIP files.

I do use it :D My StreetsMod just waits for an update :P

[Morn76]

I do use it :D My StreetsMod just waits for an update :P

Great, but how are users going to get that update without downloading a new ZIP file? Not very convenient if you ask me.

[philipbenr]

I don't know about you all, But I used to have a simple script (I forgot whether python or something else) that updated all the files that were on GitHub. I will take a look around and see If I can find it.

[dgm5555]

I don't know about you all, But I used to have a simple script (I forgot whether python or something else) that updated all the files that were on GitHub. I will take a look around and see If I can find it.

Am I missing something - MT-GitSync *is* an easy tool to update all mods on git-hub (just click Update All)

[Morn76]

I don't know about you all, But I used to have a simple script (I forgot whether python or something else) that updated all the files that were on GitHub. I will take a look around and see If I can find it.

Am I missing something - MT-GitSync *is* an easy tool to update all mods on git-hub (just click Update All)

It would be even funnier if that "simple script" turned out to be MT-GitSync and he would post it here. :-)

[webdesigner97]

I do use it :D My StreetsMod just waits for an update :P

Great, but how are users going to get that update without downloading a new ZIP file? Not very convenient if you ask me.

Modders should be able do do http requests, then you can at least notify the users about a new version...

[Morn76]

Modders should be able do do http requests, then you can at least notify the users about a new version...

Simply having MT-GitSync update everything that needs updating still sounds far easier to me. :-)

[webdesigner97]

Not everyone has Python installed :/

[Morn76]

Not everyone has Python installed :/

It's installed by default on OS X and probably on many Linux distributions (even though the Tkinter bindings might be missing on some.)

And the worst part about installing on Windows is finding an .exe on the slightly confusing python.org site, which is why I've put a direct link to the 3.4.0 download page in my first post. So all things considered I'd say running a basic Python script on the big three OSes is not that hard.

[webdesigner97]

Not everyone has Python installed :/

It's installed by default on OS X and probably on many Linux distributions (even though the Tkinter bindings might be missing on some.)

And the worst part about installing on Windows is finding an .exe on the slightly confusing python.org site, which is why I've put a direct link to the 3.4.0 download page in my first post. So all things considered I'd say running a basic Python script on the big three OSes is not that hard.

Well, I was able to install Python on my Windows, but for other it might not be worth it... sadly...

[Morn76]

Well, I was able to install Python on my Windows, but for other it might not be worth it... sadly...

I think they need to add a smart download button to the python.org homepage, so people can download the latest version for their OS with a single click. Right now you have to click a few times, scroll to the bottom of a page, click again… It's far more difficult than it has to be.

[dgm5555]

Would it be difficult to extend MT-gitsync to also download from gitorious
(eg for moreblocks and moreores)
https://github.com/Calinou/moreblocks/ no longer exists (which took me a while to figure out why gitsync was 'freezing - github asks for passwords if the resource doesn't exist
https://gitorious.org/calinou/moreblocks/

Actually if you are doing an edit, would it be possible to add a column for dependencies so they are automatically installed at the same time?

PS it still amazes me the minetest 'modshop' is so woeful, and can't even list out of date mods. Yours is still the best mod for this purpose.

[rubenwardy]

The Mod Store is broken. It is being fixed though.

[Morn76]

@dgm5555: I've just added support for other URLs in MT-GitSync 3.15. Calinou's two entries in modtable.csv now point to Gitorious instead of GH.

Dependency resolution is interesting but solving the general case (where there can be endless loops) is difficult I think. Plus I'm not aware of many dependency issues with MT mods apart from str_helpers.

[dgm5555]

Wow thanks!!
A couple of small issues since the change.
Clicking on browse mods caused the window to open behind all the other windows, so took me a while to realise it was there (using Ubuntu 14.04), and ditto going back from database window, clicking doesn't bring the main window forward. What about a pop-out list, so it's only a single window?
It doesn't seem to be able to move directories from it's home sub to the mod.. Oh just remembered by default it selects your mod directory, so fails as there is no martin on my system...
Also just a thought, if you autosaved any new mods to a file, it would make it easier for us to upload and add to your list).
I wonder if it would be possible to access it direct from the game. One simple, way would be easy to create a gitsync mod which ran gitsync from a button. If you had command-line arguments you could have it run directly, but even without, you could have the gitsync window popup direct from minetest. I think the lua forms for the front page are under /usr/local/share/minetest/builtin/mainmenu, so could be worth a play to integrate it directly. Lua code to run the python script would be rather like this:

Code: Select all

	if fields.gitsync then
		-- run the python script to generate the map (could be run in background by appending an '&' to the string, but texture might be corrupted or out of date for current load)
		local osx = "python \"/home/david/.minetest/mods/MT-GitSync/gitsyncpy\" --updateall y --moddir \"/home/david/.minetest/mods/\"  --with arguments
		local osx = "python \"/home/david/.minetest/mods/MT-GitSync/gitsyncpy\" --without
		os.execute(osx)
		print("mapit: Updated map for ".. worldName)
		minetest.chat_send_player(mapitPlayerName, "Map Updated. Please exit and re-enter world to reload the texture", false)
	end

[dgm5555]

I was thinking about this a bit more.
If you put the gitsync button on the main modstore window would be very convenient
Then you could use lua code to overwrite gitsync from a separate syncronised directory after exiting (if this is needed), then it would be kept up to date
Also if you set up a separate repo for just a 'newmodtable.csv' and gave it a collaborator password, you could safely code the password into gitsync and if it successfully downloaded a new repo, it could push the address to the newmodtable.csv. Griefing wouldn't be a problem as you'd still have your core database list, and could manually update it occasionally with any new repos from the list.
Id still argue for a one-window gitsync though. Ubuntu handling of the two windows is terrible.

[Morn76]

Thanks for your suggestions, gdm5555! I think I'll look into the single-window interface thing first. The Lua stuff is interesting too of course if paths to the gitsync.py directory can be implemented in a cross-platform way.

Maybe there should be a listbox above the mod list that toggles between a builtin core mod list and a remote contributed mod list. Or a checkbox, "show core mods only".

edit #1: The single-window interface is done (v3.17).

edit #2: Mod URLs that are not in the database are now saved to new_mods.csv (v3.21).

[dgm5555]

Great job. I like the one window look, but I tried to install a couple of mods (and also create a separate gitsync directory for a live version then clone the github to MT-Gitsync), andgot the following error:-
python ./gitsync.py

Code: Select all

Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1489, in __call__
    return self.func(*args)
  File "./gitsync.py", line 114, in add_repo
    nmfile = open(newmod_fn, 'ba')
ValueError: mode string must begin with one of 'r', 'w', 'a' or 'U', not 'ba'

I take it ba would be a typo meaning append binary, which should be 'ab' (to make it work)

[Morn76]

Oops, you are right, "ab" is correct. I've pushed a fix. I simply did not remember Python 2 was this picky when it came to file flag order. Not very Pythonic but more C-like if you ask me. :-)

[Morn76]

I've added depency checks in v3.23, i.e. installing TNT will also pull in str_helpers. Other dependencies can be added via the dependencies.csv file.

There are no checks for infinite loops though, so situations where mod X depends on mod Y and vice versa have to be avoided. :-)

[dgm5555]

This is a csv list of 1200ish gits in the forum with the post name and forum address
Unfortunately I didn't have a way of excluding gits in signatures, so they have sometimes become listed as possible repos for the mod. Also not all mods are included (for some reason some seem to have been missed), but it's a starter for a much more comprehensive database.

https://dl.dropboxusercontent.com/u/212 ... umMods.csv

This is a short sample...

Code: Select all

modName,gitAddress,forumAddress
[RELEASE] [closed] outgame intervention [outgame_intervention] - Minetest Forums,https://github.com/HybridDog/outgame_intervention,https://forum.minetest.net/viewtopic.php?p=165495
[RELEASE] [closed] Superpickaxe [superpick] - Minetest Forums,https://github.com/HybridDog/superpick,https://forum.minetest.net/viewtopic.php?p=51650
[RELEASE] [Mod] [GitHub] CreativeChanger - per-player gamemode [creativechanger] - Minetest Forums,https://github.com/Traxie21/CreativeChanger/,https://forum.minetest.net/viewtopic.php?p=80793
[RELEASE] [Mod] [Lib] Smart Formspecs [1.1 -!- inventory_plus support][smartfs] - Minetest Forums,https://github.com/rubenwardy/smartfs,https://forum.minetest.net/viewtopic.php?p=116028
[RELEASE] [Mod] 3-biome mapgen [noisetest] - Minetest Forums,https://github.com/SmallJoker/noisetest,https://forum.minetest.net/viewtopic.php?p=142510
[RELEASE] [Mod] 3D animals [1] [animals] - Minetest Forums,http://cornernote.github.io/minetest/,https://forum.minetest.net/viewtopic.php?p=36007
[RELEASE] [Mod] 3D Furniture [1.0] [3dforniture] - Minetest Forums,https://github.com/tony-ka/tonyka-minetest-mods,https://forum.minetest.net/viewtopic.php?p=28730
[RELEASE] [Mod] 3D noise floatlands [0.2.0] [fracture] - Minetest Forums,https://github.com/paramat/fracture,https://forum.minetest.net/viewtopic.php?p=139386
[RELEASE] [Mod] 3D Players [3d_player] - Minetest Forums,https://github.com/PilzAdam/3d_player,https://forum.minetest.net/viewtopic.php?p=50243
[RELEASE] [Mod] 3D Players [3d_player] - Minetest Forums,http://cornernote.github.io/minetest/,https://forum.minetest.net/viewtopic.php?p=50430
[RELEASE] [Mod] 3D Players [3d_player] - Minetest Forums,https://github.com/Sokomine/3d_player,https://forum.minetest.net/viewtopic.php?p=50430
[RELEASE] [Mod] Above sea level pools in map

[dgm5555]

There are no checks for infinite loops though, so situations where mod X depends on mod Y and vice versa have to be avoided. :-)

Brilliant. As a simple safety net, you could have a very basic stack count, and only go (eg) 3 or 4 layers deep, failing to complete all downloads. An alternative simple check would be to see if a dependency was already in the mods directory and ignore it if it was.

I've also attached a list of more mods with gits you could consider for your db.

Finally have you seen the mod search: viewtopic.php?f=14&t=10149
It would be a really powerful merge with gitsync