New Mod/Content Database by May

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

New Mod/Content Database by May

by rubenwardy » Post

It's here! See the news post

Spoiler
Hi all. As part of my course I am doing a web technologies module. We need to create a website (server and front end) from scratch. My partner and I have decided that making a content database would be a good project to work on, due to the wide range of features and technologies needed.

The backend will be finished by May - or else I'll fail the module. Hopefully I'll then be able to work on readding a mod store dialog to the main menu of Minetest, in time for 0.5.0.

Technical Details

We'll be using Python and Flask. Users will log in to the backend using their forum account. Users will then be able to add their own content, or the content of others. The backend will expose a JSON API to allow Minetest and other applications to download mod information.

Time line

Design
We'll be submitting a design document in the next few weeks. We will ask the community for to comments on our design proposal.

Development
We aim to have an initial prototype showing most of the minimal viable product features by 16th April.
The project will be finished by the 21st of May.
During development, I'll make sure to host public beta versions.

Release
The code will be released after the course is submitted and marked.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
octacian
Member
Posts: 597
Joined: Mon Dec 21, 2015 22:18
GitHub: octacian
IRC: octacian
In-game: octacian
Location: Canada

Re: New Mod/Content Database by May

by octacian » Post

Interesting, it'll be really nice to see a project like this completed for Minetest. Now, I'm curious. Why Python? And why Flask?

While I'm not familiar with Flask, I am familiar with Django. Personally though, I've concluded that Node.js and Express are a more efficient platform in several ways. So, could you elaborate on why exactly you are using these technologies?
MicroExpansion, Working Computers, All Projects - Check out my YouTube channel! (octacian)
I'm currently inactive in the Minetest community! So if I don't respond, that's why.

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

Any solution with JavaScript on the server would be rejected by other developers. I've done a fair amount with NodeJS and express, although I prefer Koa as it works nicer with ES7 features. The module actually requires you to use JavaScript, I had to get an exception to use Python
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
Devy
Member
Posts: 133
Joined: Sat Jan 21, 2017 02:31
GitHub: DevyHeavy
In-game: devy

Re: New Mod/Content Database by May

by Devy » Post

I'm very excited about this!

sofar
Developer
Posts: 2146
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: New Mod/Content Database by May

by sofar » Post

Randomly thinking out loud. You'll have 2 UI's: The minetest lua code is going to be using json API's entirely. And then there's the part where users can add or maintain content. That part will be webpages, and no matter how hard you're going to try, I don't think you'll be able to get the flexibility and features you'd want your users to have without doing a little bit of JS here and there, since that makes it a lot easier to do certain things. Having every webpage POST more JSON is costly and often not needed, and you want to do things like download a JSON blob of data once into the browser and let the client navigation be painted by JS to make it fast. It's not that I wouldn't want to avoid it, but just like CSM in minetest, some things just work way more efficient if the client doesn't need to round-trip to the server for every user interaction. So my advice is: Don't avoid JS too fanatically, and if you do JS, limit it to very basic things so that you don't need to pull in node.js and other insane frameworks.

May seems awfully soon :)

Skulls
Member
Posts: 108
Joined: Thu Dec 21, 2017 17:41
In-game: Skulls

Re: New Mod/Content Database by May

by Skulls » Post

Do you have to do some "real" JavaScript or will slapping on something like the Bootstrap boilerplate be sufficient?

I recommend having the content served from GitHub and the Python application act like a curator. Content publishers would log in with their forum account (that will be interesting...) and set up a deploy profile which gives an overview and a GitHub URL (probably a release build). This way you can dump a lot of the code security concerns onto the big dogs and focus on the mechanics.

Even if you don't get it integrated into Minetest client code right away you could easily make a nifty system where players could download a custom blob of mods, have the system check compatibilities and dependencies, and then just extract a zip bundle.

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: New Mod/Content Database by May

by Linuxdirk » Post

rubenwardy wrote:Users will log in to the backend using their forum account. Users will then be able to add their own content, or the content of others.
Will it allow providing Git links to the master tarball or ZIP?
sofar wrote:Randomly thinking out loud. You'll have 2 UI's: The minetest lua code is going to be using json API's entirely.
Focus on the API and expose it via HTTPS to the WWW so everyone can build whatever UI they want.

If the API is complete build a JS-only application with all the fancy stuff nowadays JS allow. If the API has the features to POST or GET the stuff you want it's all good.

ptvirgo
Member
Posts: 55
Joined: Thu May 26, 2016 22:18
GitHub: ptvirgo
In-game: Mox Wos
Contact:

Re: New Mod/Content Database by May

by ptvirgo » Post

Cool idea, it's inspiring me to imagine how I'd implement the same concept.

There's an older "Minetest Bower" site, maybe worth looking into whether that format / api has anything to offer. GraphQL also offers some nice features over REST, though that probably isn't realistic if your teacher is serious about the term "from scratch" (then again, Flask isn't exactly from scratch ...)

If you're at all moved by the Unix claim that it's best that each program "Do one thing and do it well," it might make sense to break the Database / API, Front end, and Mod manager into separate (or at least very loosely coupled) programs.
{-| Visiting Minetest worlds and making up stories: www.codeforyendor.xyz -}

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

I created the first mod database ages ago - Minetest Extensions, written in PHP. Since then there's been moddb, bower, and a few smaller ones. I'm making sure to learn from their mistakes, and improve on their designs. As for using bower, I will be reading the information from bower.json if available

REST/JSON is a lot simpler to implement, and I won't need most of the features that GraphQL.
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

ptvirgo
Member
Posts: 55
Joined: Thu May 26, 2016 22:18
GitHub: ptvirgo
In-game: Mox Wos
Contact:

Re: New Mod/Content Database by May

by ptvirgo » Post

Didn't know about the first database, but I've seen some of your work, so - you're Minetest famous. Cool idea, like I said it's fun thinking about the possibilities.
{-| Visiting Minetest worlds and making up stories: www.codeforyendor.xyz -}

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

Update: Our design has been approved, so we've been working on it for the last week or so. Here is our progress:

Image
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

Started working on readding the mod store to Minetest:

Image Image

That's a functional UI, although not a pretty one.

Must do before release:
  • add screenshot and short description to the package list
  • add support for games and texture packs
  • uninstall package
Extra features:
  • detect out of date packages and offer to update
  • add tags
  • add screenshots in the package screen
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
ANAND
Member
Posts: 335
Joined: Sun Jan 28, 2018 12:54
GitHub: magnetar47
IRC: MAGNETAR ANAND
In-game: ANAND and all names with Tomato
Location: India
Contact:

Re: New Mod/Content Database by May

by ANAND » Post

I'm really excited to see this - a neat little way to install mods right from within... :D
My Mods

cdb_NXKPOcRlgUGdMyf8uLoUQvnMGIfuaHmp

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

Image
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
texmex
Member
Posts: 1753
Joined: Mon Jul 11, 2016 21:08
GitHub: tacotexmex
In-game: tacotexmex

Re: New Mod/Content Database by May

by texmex » Post

Looking good, rubenwardy.

User avatar
ANAND
Member
Posts: 335
Joined: Sun Jan 28, 2018 12:54
GitHub: magnetar47
IRC: MAGNETAR ANAND
In-game: ANAND and all names with Tomato
Location: India
Contact:

Re: New Mod/Content Database by May

by ANAND » Post

rubenwardy wrote:Image
That's perfect! But how are they classified into the categories to the left?
My Mods

cdb_NXKPOcRlgUGdMyf8uLoUQvnMGIfuaHmp

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

Basic code has been merged. Additional features such as updating, screenshots, and tags will be added later.

New content tab:

Image

Here's the road map:

Image
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

yoan31
Member
Posts: 15
Joined: Tue Apr 10, 2018 14:43

Re: New Mod/Content Database by May

by yoan31 » Post

It looks great !

User avatar
ANAND
Member
Posts: 335
Joined: Sun Jan 28, 2018 12:54
GitHub: magnetar47
IRC: MAGNETAR ANAND
In-game: ANAND and all names with Tomato
Location: India
Contact:

Re: New Mod/Content Database by May

by ANAND » Post

rubenwardy - I like the new content tab, simple and neat! But one small issue though:

Image

The buttons block all the descriptions... Some sort of a word-wrap implementation would help fix this.
My Mods

cdb_NXKPOcRlgUGdMyf8uLoUQvnMGIfuaHmp

User avatar
Krock
Developer
Posts: 4649
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker
Location: Switzerland
Contact:

Re: New Mod/Content Database by May

by Krock » Post

ANAND wrote:The buttons block all the descriptions... Some sort of a word-wrap implementation would help fix this.
Another solution would be to move the text to a tooltip, which is shown when moving over the image or the title label. That's however less practical, because you can't read all of them directly.
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: New Mod/Content Database by May

by Linuxdirk » Post

Why not something like this?

Image

User avatar
ANAND
Member
Posts: 335
Joined: Sun Jan 28, 2018 12:54
GitHub: magnetar47
IRC: MAGNETAR ANAND
In-game: ANAND and all names with Tomato
Location: India
Contact:

Re: New Mod/Content Database by May

by ANAND » Post

Linuxdirk wrote:Why not something like this?
Nice, although the smaller buttons might make it difficult for touch-screen users...

The short description should be separated from the title, and could be placed where the long description currently is, and the long description can be viewed by pressing the 'View' button.
My Mods

cdb_NXKPOcRlgUGdMyf8uLoUQvnMGIfuaHmp

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

The problem is that labels don't have a size, so I have no way of limiting their length correctly. Currently I just crop to 70 characters, which is the correct length for 1080p at a standard dpi and default font.

Also, the window needs to be landscape as Minetest will predominantly be played in landscape. Having variable size items in the list is not a good idea
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: New Mod/Content Database by May

by Linuxdirk » Post

ANAND wrote:Nice, although the smaller buttons might make it difficult for touch-screen users...
Of course they should not be smaller than the currently used buttons. Maybe have view and install below each other to use the full width.
rubenwardy wrote:Also, the window needs to be landscape as Minetest will predominantly be played in landscape.
The scribble is in portrait mode only to show what I mean. The actual look would be like this, but with scrollbar:

Image
rubenwardy wrote:Having variable size items in the list is not a good idea
I am uncertain about this. First rule of course is "do not move navigation elements around". But on the other hand those can be seen as multiple entries in a list and those entries vary in length and thus moving entries down when the previous entry gets longer doesn't feel that bad to me since the navigation elements are always at the same location relative to the entry.

But since the design is limited by the implementation I guess we have to take what we get if no-one wants/can add more/better UI features ... :)

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: New Mod/Content Database by May

by rubenwardy » Post

I'd definitely prefer to have a scrolling list for this, but that's not possible with formspecs currently
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

Locked

Who is online

Users browsing this forum: No registered users and 6 guests