Page 1 of 2
New Mod/Content Database by May
Posted: Wed Mar 07, 2018 17:09
by rubenwardy
It's here! See the
news post
Spoiler
Hi all. As part of my course I am doing a web technologies module. We need to create a website (server and front end) from scratch. My partner and I have decided that making a content database would be a good project to work on, due to the wide range of features and technologies needed.
The backend will be finished by May - or else I'll fail the module. Hopefully I'll then be able to work on readding a mod store dialog to the main menu of Minetest, in time for 0.5.0.
Technical Details
We'll be using Python and Flask. Users will log in to the backend using their forum account. Users will then be able to add their own content, or the content of others. The backend will expose a JSON API to allow Minetest and other applications to download mod information.
Time line
Design
We'll be submitting a design document in the next few weeks. We will ask the community for to comments on our design proposal.
Development
We aim to have an initial prototype showing most of the minimal viable product features by 16th April.
The project will be finished by the 21st of May.
During development, I'll make sure to host public beta versions.
Release
The code will be released after the course is submitted and marked.
Re: New Mod/Content Database by May
Posted: Wed Mar 07, 2018 19:30
by octacian
Interesting, it'll be really nice to see a project like this completed for Minetest. Now, I'm curious. Why Python? And why Flask?
While I'm not familiar with Flask, I am familiar with Django. Personally though, I've concluded that Node.js and Express are a more efficient platform in several ways. So, could you elaborate on why exactly you are using these technologies?
Re: New Mod/Content Database by May
Posted: Wed Mar 07, 2018 19:38
by rubenwardy
Any solution with JavaScript on the server would be rejected by other developers. I've done a fair amount with NodeJS and express, although I prefer Koa as it works nicer with ES7 features. The module actually requires you to use JavaScript, I had to get an exception to use Python
Re: New Mod/Content Database by May
Posted: Fri Mar 09, 2018 03:19
by Devy
I'm very excited about this!
Re: New Mod/Content Database by May
Posted: Fri Mar 09, 2018 04:44
by sofar
Randomly thinking out loud. You'll have 2 UI's: The minetest lua code is going to be using json API's entirely. And then there's the part where users can add or maintain content. That part will be webpages, and no matter how hard you're going to try, I don't think you'll be able to get the flexibility and features you'd want your users to have without doing a little bit of JS here and there, since that makes it a lot easier to do certain things. Having every webpage POST more JSON is costly and often not needed, and you want to do things like download a JSON blob of data once into the browser and let the client navigation be painted by JS to make it fast. It's not that I wouldn't want to avoid it, but just like CSM in minetest, some things just work way more efficient if the client doesn't need to round-trip to the server for every user interaction. So my advice is: Don't avoid JS too fanatically, and if you do JS, limit it to very basic things so that you don't need to pull in node.js and other insane frameworks.
May seems awfully soon :)
Re: New Mod/Content Database by May
Posted: Fri Mar 09, 2018 15:02
by Skulls
Do you have to do some "real" JavaScript or will slapping on something like the Bootstrap boilerplate be sufficient?
I recommend having the content served from GitHub and the Python application act like a curator. Content publishers would log in with their forum account (that will be interesting...) and set up a deploy profile which gives an overview and a GitHub URL (probably a release build). This way you can dump a lot of the code security concerns onto the big dogs and focus on the mechanics.
Even if you don't get it integrated into Minetest client code right away you could easily make a nifty system where players could download a custom blob of mods, have the system check compatibilities and dependencies, and then just extract a zip bundle.
Re: New Mod/Content Database by May
Posted: Fri Mar 09, 2018 15:29
by Linuxdirk
rubenwardy wrote:Users will log in to the backend using their forum account. Users will then be able to add their own content, or the content of others.
Will it allow providing Git links to the master tarball or ZIP?
sofar wrote:Randomly thinking out loud. You'll have 2 UI's: The minetest lua code is going to be using json API's entirely.
Focus on the API and expose it via HTTPS to the WWW so everyone can build whatever UI they want.
If the API is complete build a JS-only application with all the fancy stuff nowadays JS allow. If the API has the features to POST or GET the stuff you want it's all good.
Re: New Mod/Content Database by May
Posted: Sun Mar 18, 2018 21:21
by ptvirgo
Cool idea, it's inspiring me to imagine how I'd implement the same concept.
There's an older "Minetest Bower" site, maybe worth looking into whether that format / api has anything to offer. GraphQL also offers some nice features over REST, though that probably isn't realistic if your teacher is serious about the term "from scratch" (then again, Flask isn't exactly from scratch ...)
If you're at all moved by the Unix claim that it's best that each program "Do one thing and do it well," it might make sense to break the Database / API, Front end, and Mod manager into separate (or at least very loosely coupled) programs.
Re: New Mod/Content Database by May
Posted: Sun Mar 18, 2018 22:23
by rubenwardy
I created the first mod database ages ago - Minetest Extensions, written in PHP. Since then there's been moddb, bower, and a few smaller ones. I'm making sure to learn from their mistakes, and improve on their designs. As for using bower, I will be reading the information from bower.json if available
REST/JSON is a lot simpler to implement, and I won't need most of the features that GraphQL.
Re: New Mod/Content Database by May
Posted: Mon Mar 19, 2018 00:08
by ptvirgo
Didn't know about the first database, but I've seen some of your work, so - you're Minetest famous. Cool idea, like I said it's fun thinking about the possibilities.
Re: New Mod/Content Database by May
Posted: Fri Mar 23, 2018 17:39
by rubenwardy
Update: Our design has been approved, so we've been working on it for the last week or so. Here is our progress:
Re: New Mod/Content Database by May
Posted: Sat Mar 24, 2018 23:36
by rubenwardy
Started working on readding the mod store to Minetest:
That's a functional UI, although not a pretty one.
Must do before release:
- add screenshot and short description to the package list
- add support for games and texture packs
- uninstall package
Extra features:
- detect out of date packages and offer to update
- add tags
- add screenshots in the package screen
Re: New Mod/Content Database by May
Posted: Mon Mar 26, 2018 08:28
by ANAND
I'm really excited to see this - a neat little way to install mods right from within... :D
Re: New Mod/Content Database by May
Posted: Sun Apr 08, 2018 17:54
by rubenwardy
Re: New Mod/Content Database by May
Posted: Sun Apr 08, 2018 21:15
by texmex
Looking good, rubenwardy.
Re: New Mod/Content Database by May
Posted: Mon Apr 09, 2018 06:27
by ANAND
rubenwardy wrote:
That's perfect! But how are they classified into the categories to the left?
Re: New Mod/Content Database by May
Posted: Thu Apr 19, 2018 20:04
by rubenwardy
Basic code has been merged. Additional features such as updating, screenshots, and tags will be added later.
New content tab:
Here's the road map:
Re: New Mod/Content Database by May
Posted: Thu Apr 19, 2018 20:07
by yoan31
It looks great !
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 08:29
by ANAND
rubenwardy - I like the new content tab, simple and neat! But one small issue though:
The buttons block all the descriptions... Some sort of a word-wrap implementation would help fix this.
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 10:24
by Krock
ANAND wrote:The buttons block all the descriptions... Some sort of a word-wrap implementation would help fix this.
Another solution would be to move the text to a tooltip, which is shown when moving over the image or the title label. That's however less practical, because you can't read all of them directly.
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 10:55
by Linuxdirk
Why not something like this?
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 11:49
by ANAND
Linuxdirk wrote:Why not something like this?
Nice, although the smaller buttons might make it difficult for touch-screen users...
The short description should be separated from the title, and could be placed where the long description currently is, and the long description can be viewed by pressing the 'View' button.
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 12:01
by rubenwardy
The problem is that labels don't have a size, so I have no way of limiting their length correctly. Currently I just crop to 70 characters, which is the correct length for 1080p at a standard dpi and default font.
Also, the window needs to be landscape as Minetest will predominantly be played in landscape. Having variable size items in the list is not a good idea
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 14:41
by Linuxdirk
ANAND wrote:Nice, although the smaller buttons might make it difficult for touch-screen users...
Of course they should not be smaller than the currently used buttons. Maybe have view and install below each other to use the full width.
rubenwardy wrote:Also, the window needs to be landscape as Minetest will predominantly be played in landscape.
The scribble is in portrait mode only to show what I mean. The actual look would be like this, but with scrollbar:
rubenwardy wrote:Having variable size items in the list is not a good idea
I am uncertain about this. First rule of course is "do not move navigation elements around". But on the other hand those can be seen as multiple entries in a list and those entries vary in length and thus moving entries down when the previous entry gets longer doesn't feel that bad to me since the navigation elements are always at the same location relative to the entry.
But since the design is limited by the implementation I guess we have to take what we get if no-one wants/can add more/better UI features ... :)
Re: New Mod/Content Database by May
Posted: Fri Apr 20, 2018 14:49
by rubenwardy
I'd definitely prefer to have a scrolling list for this, but that's not possible with formspecs currently