[Tool] GUI for installing and updating mods from GitHub

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

@izzyb: Thanks, I'll take a look. Maybe I should also create a maintenance script that checks for duplicates on the list.

@TheReaperKing: At the moment it is not possible to select multiple entries. Clicking an item puts its address in the "selected mod" textbox on the left, which is then read when installing (so you can also type the address). So multiple selection would not work with that. But perhaps I can think of a way to make it work. Maybe selected mod(s) should be a listbox too.

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

Hopefully it wouldn't be too much work but that'd be great!! Just with as you mentioned the over a thousand mods it will take me forever to select them all! :) Thanks for your hard work on this project. I love collecting mods so a project like this is a major blessing!
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Morn76 wrote:@izzyb: Thanks, I'll take a look. Maybe I should also create a maintenance script that checks for duplicates on the list.
There are quite a number of duplicates, but most look like multiple versions of the same mod in different repositories. Not sure if they are actually different versions or not. Would be nice to get more details on each of them. I was thinking about going through and verifying if each of the mods actually work.

It'd be nice to have some additional fields to work with in the lists. I'll probably want to change or add to it, but here are some fields that'd be nice to have:

1) Type - mod, game, texture pack. I see a couple of games in the list, and have games to add. would be nice to flag them as games, not mods.
2) Version - Place for a version number. should probably have one for version of minetest needed
3) Owner - Name of repository owner to help with duplicate versions of the same mod
4) Status - WIP, development, released, old, broken, or whatever
5) popularity - some sort of like/dislike flag or favorite marker. Would be nice to have this just locally, but having a means to centralize the counts would be nice. Just having an additional .csv file that tracks your favorites would be nice - especially if it auto highlights the dependencies for you.

Also, a way to handle links for tarballs would be nice. :)

EDIT:
6) Class - Mob, tool, food, plants,etc.
cdb_d26897136a4c

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

I noticed the same thing, sometimes the same mod name (and more importantly folder name) but a different mod author. Right at the top it has 3Dplayer and awards as examples. Perhaps especially for folders with the same name the folder could be followed by the name of the repo owner? For example awards(rubenwardy).

Also would it be possible to make it so just doing a double click on the list would download the repo?

Thank you so much for your hard work on this. I love seeing the collection of mods, makes life so much easier!

Take care.
-Mike
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

Somehow double posted by accident, sorry! :)
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

Thanks for the great suggestions, everyone. I'll see what I can do. :-)
TheReaperKing wrote: Also would it be possible to make it so just doing a double click on the list would download the repo?
Probably not, Tkinter is a bit limited. It can only get the active list item once in a while and check if it has changed, but a listbox doesn't really register clicks or double clicks themselves like other GUI toolkits do.

@izzyb: Adding columns to the table is possible of course. But the question is who will fill in and maintain the information? Repo type might be something you can scrape automatically, but e.g. status or version are not really something you can determine automatically.

As for ratings, I wonder if we could get them from Krock's database: http://krock-works.16mb.com/MTstuff/modSearch.php

I agree that a clear distinction between mods, games, and texture packs would be nice.

I have a nice sortable table in PySide/Qt here: https://github.com/mdoege/lightningmf , so I'm almost a bit tempted to use Qt instead of Tk. Except it would probably make things a bit difficult for Windows users. But table sorting is another one of those things that would be nice to have but are not supported by standard Tk widgets I think. Tk is nice for users because there is nothing to install, but you quickly reach its limits…

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

That Krock database looks like a nice start. Didn't know about it before. It would be nice to have a standard formate that everyone can agree on to help automate collection of data in tools like this. Something that allows the mod creators and maintainers to contribute to the data would be ideal. A simple file in the mod directory with standardized parameters for authors to include would do it. It could be used in game also, either when selecting mods, or maybe for in game help or mods that display details about a node.

Perhaps it's something everyone creating modes like this should discuss and work out an agreed format for such a document with all the fields that might be useful to each of you.

Also, perhaps adding the ability to like a mod in your app could be fed back to Krock to help automate the collection of that data.
cdb_d26897136a4c

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

Krock's database has its own thread (viewtopic.php?f=14&t=10149), so it's hard to overlook. :-)

The problem with it is that Krock's database is not available for download and he apparently does not have any intention to cooperate with authors of other tools like mine. And I don't think there is an API either so e.g. fetching ratings would have to use HTML scraping.

So right now everyone has to do their own scraping of the forums, creating independent databases with different features. Not a great situation, but that's how it is at the moment unfortunately.

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

Have you seen this post? Perhaps you two could work together, if nothing else for comparing mod lists:
viewtopic.php?f=14&t=13051

Also this post on the thread seems relevant too:
viewtopic.php?p=225992#p225992

Hope it helps! I appreciate your commitment to this :)
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

Thanks, ReaperKing; I think Ruben Wardy's mod list looks very interesting, and at least you can download it as JSON. So he is probably a bit more open-minded than Krock regarding cooperation with other tools.

At the very least I could import his list from time to time and check that all his repos are in my list too.

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

Ok, so in the meantime Krock has contacted me and given me access to his JSON (thanks, Krock!). I have added the missing GitHub repos from his list, so now there are (after izzyb's deletions) 1,342 mods in total.

Krock's JSON also includes up- and downvotes, so these are now displayed in the list as e.g. [+4/-2], which would mean 4 votes in favor and 2 against. Note that only 251 repos have votes, so this is a fairly small fraction right now.

Still, lots of delicious new input… :-)

Image
Last edited by Morn76 on Mon Sep 05, 2016 10:22, edited 1 time in total.

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Awesome! I'll have to go through Krock's database and add my votes.

I have some more entries to clean up. I'm not sure which entry is causing these, but I'm getting user/password prompts when doing an update. This is the output on the console - I just hit enter for the prompts. Could you maybe output the table entry being processed on the console to help identify the entry causing errors like this. I have another one that thinks I've modified something locally. I likely have, but I don't know what mod is generating the error.

Code: Select all


Username for 'https://github.com': 
Password for 'https://github.com': 
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/everamzah/dmobs/'

Username for 'https://github.com': 
Password for 'https://github.com': 
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/captainLAD/gemsverison2.0/'

Username for 'https://github.com': 
Password for 'https://github.com': 
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/vitaminx/memorandum/'

Username for 'https://github.com': 
Password for 'https://github.com': 
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/vitaminx/pizza/'

cdb_d26897136a4c

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

They are removed/dead gits/links
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

So perhaps the GUI should check if the repository still exists on GitHub before it tries to pull from it…

Update: OK, so I've added that check now. Deleted repositories should no longer be a problem.

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Do they get purged from the local list? Maybe have a script check all the repositories before a release?

This presents a second problem. I have repositories I've downloaded using an outdated link. When I try to download the same mod using one of the remaining links, it can't because the folder already exists. I know I can manually go and delete the folder, but it'd be nice if it could prompt to replace the existing install. That'd come in handy for trying different versions of the same mod from different repositories as well.

Also, would it be possible to highlight or somehow mark the mods that are downloded?
cdb_d26897136a4c

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

So far it only checks during updates (because this is where the bug was reported), but maybe it could check before installing too. I check the list (https://github.com/mdoege/gitsync-devel ... ck_urls.py) from time to time too of course, but still. I also would like to update votes for each mod when it is selected, but I'd have to ask Krock how that would work. So far I can only fetch the JSON for all mods.

Deleting repos from the app is a bit dangerous. Mod developers have their development mods (that are not necessarily pushed to GitHub) in that folder, so there is a danger that valuable work could get lost by accident. Forum users tend to be mod developers too and I do not want to cause data loss. Once an app can delete stuff, sooner or later it will delete the wrong thing. I think TheReaperKing had this idea upthread that the author could be part of the folder name, so you can e.g. have different "mobs" mods installed. This might be safer than deleting or replacing.

Highlighting downloaded mods is a good idea. Added to my to-do list.

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by rubenwardy » Post

TheReaperKing wrote:I noticed the same thing, sometimes the same mod name (and more importantly folder name) but a different mod author. Right at the top it has 3Dplayer and awards as examples. Perhaps especially for folders with the same name the folder could be followed by the name of the repo owner? For example awards(rubenwardy)
There is only one mod called awards. The other is a github fork used by someone to contribute a patch - This list should not contain that fork, at least not by default
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

rubenwardy wrote:
TheReaperKing wrote:I noticed the same thing, sometimes the same mod name (and more importantly folder name) but a different mod author. Right at the top it has 3Dplayer and awards as examples. Perhaps especially for folders with the same name the folder could be followed by the name of the repo owner? For example awards(rubenwardy)
There is only one mod called awards. The other is a github fork used by someone to contribute a patch - This list should not contain that fork, at least not by default
So I should keep minetest-mods/awards and remove Traxie21/awards, right? What about rubenwardy/awards though?

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by rubenwardy » Post

rubenwardy/awards was where the repo originally was before it was moved to minetest-mods. So minetest-mods is the only one that should remain
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

I have updated the code today: Now mods that are already downloaded are highlighted in the list (prepended with "0>>>", so they are right at the top). Remote repo existence is checked before installing. Also, if a directory name is already taken, the user name is added, so e.g. bacon might be installed as "bacon_by_davisonio".

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Awesome! Thanks. I'll check it out now.
cdb_d26897136a4c

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Thanks for the update. It's working great! :) One thing that would be nice to see in the status line is a count of installed mods. I know you can get it doing an update, but would be nice to see it all the time. I'm up to 295 mods installed and managed by MT-GitSync. :)

Adding the _by_author works out perfectly for my needs! :) I have mods downloading into one directory then create symlinks in .minetest/mods to choose the active version. Working out great! :)

Also loving that you can search the developer name to find all their mods. I'm having trouble getting the mobs_redo mods by tenplus working and see something is wrong with the packages being downloaded. I'll post details once I have it sorted out.
cdb_d26897136a4c

izzyb
Member
Posts: 35
Joined: Thu May 07, 2015 22:41

Re: [Tool] GUI for installing and updating mods from GitHub

by izzyb » Post

Tenplus1's mobs_redo mods

These are the existing entries in the list. They're both bad entries.

Code: Select all

Mod Releases • Re: [Mod] Mobs Redo [1.30] [mobs],tenplus1/mobs_animal,https://forum.minetest.net/viewtopic.php?t=9917&p=228759#p228759
Mod Releases • Re: [Mod] Mobs Redo [1.14] [mobs],http://www.github.com/tenplus1,https://forum.minetest.net/viewtopic.php?t=9917&p=185521#p185521
The Main forum link is viewtopic.php?f=11&t=9917
It consists of 4 mods, each with it's on repo. Assuming I Understand everything correctly, Here's how it should look. There's more links there to compatible mobs I'll go through.

Code: Select all

Mod Releases • Re: [Mod] Mobs Redo API[1.31][Mobs],tenplus1/mobs_redo,https://forum.minetest.net/viewtopic.php?f=11&t=9917
Mod Releases • Re: [Mod] Mobs Redo Animals (mobs_animal) [1.31][Mobs],tenplus1/mobs_animal,https://forum.minetest.net/viewtopic.php?f=11&t=9917
Mod Releases • Re: [Mod] Mobs Redo Monsters (mobs_monster)[1.31][Mobs],tenplus1/mobs_monster,https://forum.minetest.net/viewtopic.php?f=11&t=9917
Mod Releases • Re: [Mod] Mobs Redo Simple NPC and Trader (mobs_npc)[1.31][Mobs],tenplus1/mobs_npc,https://forum.minetest.net/viewtopic.php?f=11&t=9917
cdb_d26897136a4c

Morn76
Member
Posts: 659
Joined: Sun Feb 16, 2014 19:45

Re: [Tool] GUI for installing and updating mods from GitHub

by Morn76 » Post

Thanks, izzy, I've updated the code and mod table again. Now the git repo count is shown in the description box.

GitHub really needs to be more like Google Docs and allow editing files directly in the browser. That would make maintaining the mod list collaboratively a lot easier.

User avatar
TheReaperKing
Member
Posts: 531
Joined: Sun Nov 22, 2015 21:36
Contact:

Re: [Tool] GUI for installing and updating mods from GitHub

by TheReaperKing » Post

Can't you? I remember modifying text files before right on the github webpage or do you mean something else? Also maybe putting the mod list in a google doc would work?
Become A Real Life Superhero - http://SuperheroHill.com
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com
Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com
Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests