The future of Minetest

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

daret wrote:
Wed Oct 27, 2021 21:54
Minetest engine should support everything and more than Minecraft does.
I strongly disagree, and I believe the devs and most of the modders would too. The MTE should support everything that an arbitrary voxel world needs without any Minecraft specifics. Then a mod on top of it can make it look and work like Minecraft (this is how things are now, see the MCL2 game).
(Note that besides of the technical difficulties, there's a legal issue as well with copying Minecraft features into the engine. Not a good idea.)
daret wrote:
Wed Oct 27, 2021 21:54
This would at least help games like Mineclone.
Yeah, but only that one. It would be very likely a PITA for all the other games.
daret wrote:
Wed Oct 27, 2021 21:54
Dynamic tools / animated - like bows, compass, ... - item picture change without the animation
Not sure what you mean by "without the animation". Item pictures can already change.
daret wrote:
Wed Oct 27, 2021 21:54
Leashes, Fishing - I never saw a good mod for it. I believe, the "rope" needed for fishing rod / leashes is not possible, or?
I believe fishing must be a mod. Maybe the Lua API needs a bit tweaking if that's the reason why there isn't any good mod (I haven't debugged any of the fishing mods myself, so I'm not sure what's missing), which leads us back to one of my points, Lua API needs some care and love.
daret wrote:
Wed Oct 27, 2021 21:54
Dimensions - I know, you can create vertical worlds, but they are still somehow direct connected. You cold use vertical space for some space game and the dimensions for different solar system or something like that ...
Not a bad idea, adding a fourth dimension: X, Y, Z, subworld; where subworld could be 0 = overworld, 1 = nether, 2 = end etc. or solar systems depending on the game. But probably would require way too much modification in the engine to worth it I guess. But good idea indeed!
daret wrote:
Wed Oct 27, 2021 21:54
World size - 65500 (or so) is big, but far from infinite. There was some discussion in 2014, but think this is not changed, or?
That's 2^16-1 (65535). I personally don't think this is a real limitation. I have never wandered off the edge so far :-)

Cheers,
bzt

User avatar
Wuzzy
Member
Posts: 4786
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: The future of Minetest

by Wuzzy » Post

1. no concept for Lua bindings and API, incompatible changes all the time
Every time I update the Minetest Engine, games and mods broke, and I have to face a big pile of random Lua script crashes. I'm not sure why this is happening, but I suspect it's because the lack of (proper) tests and consistent API. Even the Minetest Game is keep crashing after the latest MTE upgrade :-( (something about lua entity on_step handler missing a property).
It really depends. As an user, you MUST check manually if the mod/game author guarantees compability with Minetest version X. The compability breakages used to be much worse tho. Also, many breakages are deliberate and documented in the changelog. You're complaint isn't really helping tho since you're being vague here. Still, I think many breakages are hard to excuse.

On the other hand, as a long-time game and mod dev myself, the breakages are often very easy to fix as long you still maintain the game. I never had catastrophic breakages that weren't my own fault. And paramat is right, it's often when games and mods abuse undocumented feature. That's on them. Suggestion: When a game breaks, complain to the game author.

Although Minetest tries hard to not break things between upgrades, it's not perfect. Ideally, Minetest would not permit ANY API breakage between minor releases. Ideally, a game I coded in 5.0.0 should still work in 5.1244.0 without ANY maintenance on my part, provided I didn't abuse the API (like depending on bugs or undocumented features). Even a "few small little breakages" or behavior changes mean the game author must touch the game again to make it work as expected.

Simple but annoying example: Object crosshairs: It's not a crasher, but this technically broke graphical design because games were now expected to add object_crosshair.png or else the game will fallback to a default crosshair, which might look ugly in your design if you didn't see it coming. Compability COULD have been provided, but was not. This is one of the small things that add up the further versions you go down the line. The fix, of course, is trivial for games, but my point is, it required game authors to touch the game again only to make the crosshair look like in the previous version again.

Therefore, I think Minetest should be a LOT stricter when it comes to breaking things, also not permit "minor breakages", no matter how "easy" to fix, due to the burden it puts to game authors to touch their game after every minor release. Only if there REALLY is no other way, and no compability mode is possible, should a breakage even be considered. Or if it's major version, then everything is allowed, of course. ;)
2. incompatible changes in the UI
I feel like this should just be a normal bug report.

3. it is not possible to update contents consistently
This is weird, because packages DO have a minimum maximum supported version in their data. Maybe a bug?
4. MT engine has no main concept, it's development completely ad hoc
I think the reason behind the unmanageable number of issues (about 1000 open issues on github as of writing!!!) and the insanely long time for PRs to be merged (sometimes years!!!) lies in the lack of a clear concept. If there were a clear vision for MT, it would be easier for the developers to decide if a PR aligns with MT's goal or not (by that I mean both the PR writers' and the PR mergers' life would be simpler). There would be probably less problems with the merged PRs as well, meaning much less open issues in the first place.
No, that won't change anything. The REAL problem is that only a tiny number of people is allowed to give official approvals to PRs (only coredevs, basically), and of those reviewers, only a fraction is active at all. While at the same time, Minetest DROWNING in community contributions.
This is also why approval takes forever, core devs just rarely show up in the first place. It's really no surprise why development has slowed down to a crawl. The bottleneck is the reviewing process, but nobody has approval rights.

And approvals go rarely smoothly, there's always something to nitpick. QA is still important though because without any QA, Minetest will detoriate FAST like Freeminer.
The real solution would either be: More people become official reviewers with approval rights (read: core devs). Or the existing core devs make Minetest their fulltime job (haha, fat chance!).
5. obsolete engine under the hood
This is techically no longer true. Irrlicht got forked and became IrrlichtMt, it's independent now, which is GOOD.

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

Wuzzy wrote:
Thu Nov 04, 2021 03:23
1. no concept for Lua bindings and API, incompatible changes all the time
Every time I update the Minetest Engine, games and mods broke, and I have to face a big pile of random Lua script crashes. I'm not sure why this is happening, but I suspect it's because the lack of (proper) tests and consistent API. Even the Minetest Game is keep crashing after the latest MTE upgrade :-( (something about lua entity on_step handler missing a property).
It really depends. As an user, you MUST check manually if the mod/game author guarantees compability with Minetest version X.
And that's exactly the problem: the UI (specifically on the "Content" and "Browse online content" tabs) you simply cannot see nor check the versions. Maybe you must check the versions, but you cannot. There has been a progress on this, the Update button only appears if it's safe to update the mod I was told (haven't checked, but for a while I got far less issues after upgrading so I guess this is working as it should.)
Wuzzy wrote:
Thu Nov 04, 2021 03:23
Therefore, I think Minetest should be a LOT stricter when it comes to breaking things, also not permit "minor breakages", no matter how "easy" to fix, due to the burden it puts to game authors to touch their game after every minor release.
This is exactly the kind of thing what I meant by "MT engine has no main concept, it's development completely ad hoc".
Wuzzy wrote:
Thu Nov 04, 2021 03:23
2. incompatible changes in the UI
I feel like this should just be a normal bug report.
I'm more interested in a generalized method to handle this, not in solving one by one, because there are SO MANY cases. I guess a "postinstall" hook would be the proper way, where the mods could set (or unset) their settings for the new MTE version, aka. configuration migration. (I have yet another issue with the latest version which I haven't encounter before: biomes aren't generated any more. It turned out there's a setting flag for that. I don't know if this flag existed in the old version at all, but after the MTE upgrade it exists and unchecked for sure breaking world generation. It wasn't trivial nor fun to find that particular setting on the UI at all!)
Wuzzy wrote:
Thu Nov 04, 2021 03:23
3. it is not possible to update contents consistently
This is weird, because packages DO have a minimum maximum supported version in their data. Maybe a bug?
They have now, but when I opened this topic more than a year ago, they hadn't. This has been resolved thanks to the work of the devs, this point is already obsolete.
Wuzzy wrote:
Thu Nov 04, 2021 03:23
4. MT engine has no main concept, it's development completely ad hoc
I think the reason behind the unmanageable number of issues (about 1000 open issues on github as of writing!!!) and the insanely long time for PRs to be merged (sometimes years!!!) lies in the lack of a clear concept. If there were a clear vision for MT, it would be easier for the developers to decide if a PR aligns with MT's goal or not (by that I mean both the PR writers' and the PR mergers' life would be simpler). There would be probably less problems with the merged PRs as well, meaning much less open issues in the first place.
No, that won't change anything.
I disagree. If there were clear concepts (like not breaking the API in minor versions as you've said), it would be easier for contributors to comply, and in turn it would be easier for the devs to merge PRs, which would result in shorter waiting time, which would result in less open tickets. (The small number of devs is an issue, you can either have more devs, or ease their load by making the contributions simpler to merge. You said the obvious solution, more devs. I don't know why that's not doable, and I don't want to question that decision (I don't know the reasons). Therefore I'm suggesting the second best thing, make the PRs easier to merge.)
Wuzzy wrote:
Thu Nov 04, 2021 03:23
And approvals go rarely smoothly, there's always something to nitpick.
Exactly. With a clear concept and development guides, this could be improved significantly. (With a stable Lua API it would be even possible to do automated tests, but I wouldn't go that far because building such a system is no simple task.)
Wuzzy wrote:
Thu Nov 04, 2021 03:23
5. obsolete engine under the hood
This is techically no longer true. Irrlicht got forked and became IrrlichtMt, it's independent now, which is GOOD.
Forking irrlicht did not solve the issue, if anything, it put more work on the MTE devs' shoulders. But since irrlicht isn't maintained any more, this was the right thing to do, and an unavoidable, very important step for the future of Minetest, I agree.

Cheers,
bzt

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: The future of Minetest

by rubenwardy » Post

bzt wrote:
Thu Nov 04, 2021 11:18
This is exactly the kind of thing what I meant by "MT engine has no main concept, it's development completely ad hoc".
Please see the roadmap: https://github.com/minetest/minetest/bl ... rection.md
Non bugfix PRs that aren't on the roadmap need to be concept approved within 7 days, or else they will be closed
bzt wrote:
Thu Nov 04, 2021 11:18
They have now, but when I opened this topic more than a year ago, they hadn't. This has been resolved thanks to the work of the devs, this point is already obsolete.
Package requiring Minetest versions already existed when you opened this topic. Support for it was added to ContentDB in Jan 2019. This is only checked when downloading packages from ContentDB, and there's no way for packages to depend on versions of other packages (this is a hard problem for package managers)
bzt wrote:
Thu Nov 04, 2021 11:18
I disagree. If there were clear concepts (like not breaking the API in minor versions as you've said), it would be easier for contributors to comply, and in turn it would be easier for the devs to merge PRs, which would result in shorter waiting time, which would result in less open tickets. (The small number of devs is an issue, you can either have more devs, or ease their load by making the contributions simpler to merge. You said the obvious solution, more devs. I don't know why that's not doable, and I don't want to question that decision (I don't know the reasons). Therefore I'm suggesting the second best thing, make the PRs easier to merge.)
--
Exactly. With a clear concept and development guides, this could be improved significantly. (With a stable Lua API it would be even possible to do automated tests, but I wouldn't go that far because building such a system is no simple task.)
See the roadmap and also the contribution guidelines. It includes a point on compatibility (but is weirdly worded)

Also, we can't win when it comes to compatibility. Some people are annoyed when we don't break compatibility, others are annoyed when we do. Most of the intentional compatibility breakages since 5.0 are in undefined and undocumented areas. The policy I prefer is to use deprecations to warn for breakages the version or so before it takes place
bzt wrote:
Thu Nov 04, 2021 11:18
Forking irrlicht did not solve the issue, if anything, it put more work on the MTE devs' shoulders. But since irrlicht isn't maintained any more, this was the right thing to do, and an unavoidable, very important step for the future of Minetest, I agree.
Assuming there are developers with enough time to work on this, the plan is to remove Irrlicht entirely and use SDL and OpenGL
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

Hi @rubenwardy,

So many good news, thank you very much!
rubenwardy wrote:Package requiring Minetest versions already existed when you opened this topic. Support for it was added to ContentDB in Jan 2019.
Yes, but it wasn't added to the MTE (as a matter of fact, the PR was merged the very same day I wrote the post :-), and so the feature appeared only later in the next release). Doesn't matter, I'm happy this has been solved!

It's so good to see that all of my concerns has been addressed one way or another, so I'm not worried about the future of Minetest any more! (At least not on the technical side, and I already had hope for this great modder and user community too!)

Thank you again for your hard work on this awesome software,
bzt

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: The future of Minetest

by rubenwardy » Post

bzt wrote:
Thu Nov 04, 2021 15:05
Yes, but it wasn't added to the MTE (as a matter of fact, the PR was merged the very same day I wrote the post :-), and so the feature appeared only later in the next release). Doesn't matter, I'm happy this has been solved!
That's the [Update all] button. Support in the engine for packages depending on Minetest versions was there from the beginning, when ContentDB was added in 5.0. The backend support was added a few months after that

But yeah, it doesn't matter
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: The future of Minetest

by ywwv » Post

in the future we are going to ahve VR and voice chat. thjis will allow it to compete against the metaverse

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

rubenwardy wrote:
Thu Nov 04, 2021 17:36
bzt wrote:
Thu Nov 04, 2021 15:05
Yes, but it wasn't added to the MTE (as a matter of fact, the PR was merged the very same day I wrote the post :-), and so the feature appeared only later in the next release). Doesn't matter, I'm happy this has been solved!
That's the [Update all] button. Support in the engine for packages depending on Minetest versions was there from the beginning, when ContentDB was added in 5.0. The backend support was added a few months after that

But yeah, it doesn't matter
Come on, a backend function is useless unless it is channeled to the UI so that end users can see and use it :-) Btw, I'm sure minimum and maximum MTE version checks wasn't implemented, and it is now, that's a big thing too for user experience. As I've said there are FAR LESS (by magnitudes) issues with the engine / mod upgrades as it were a year ago, and I'm very glad!

Cheers,
bzt

ps: I'm completely selfish here, but my boys learned how to install and update mods on their own, and I haven't heard any "Daddy, it crashed again, fix it!" cries lately :-D Last year when I opened the topic this was a constant issue, I had to do mod maintainance for them (and fix the Lua / settings / whatever issues) all the time. I honestly don't remember if I had any problems for half a year now, so, well done!

User avatar
daret
Member
Posts: 136
Joined: Tue Nov 12, 2019 20:36
GitHub: daretmavi
In-game: Daretmavi

Re: The future of Minetest

by daret » Post

bzt wrote:
Wed Nov 03, 2021 20:57
daret wrote:
Wed Oct 27, 2021 21:54
Minetest engine should support everything and more than Minecraft does.
I strongly disagree, and I believe the devs and most of the modders would too. The MTE should support everything that an arbitrary voxel world needs without any Minecraft specifics. Then a mod on top of it can make it look and work like Minecraft (this is how things are now, see the MCL2 game).
(Note that besides of the technical difficulties, there's a legal issue as well with copying Minecraft features into the engine. Not a good idea.)
You misunderstand me.
I'm not talking about copying Minecraft staff, I'm talking about technical possibilities.

With other words. I you want to copy Minecraft, you should be able to do so. The engine should not limit you or make something impossible.
bzt wrote:
Wed Nov 03, 2021 20:57
daret wrote:
Wed Oct 27, 2021 21:54
This would at least help games like Mineclone.
Yeah, but only that one. It would be very likely a PITA for all the other games.
It would actually expand the possibilities for all games.

User avatar
daret
Member
Posts: 136
Joined: Tue Nov 12, 2019 20:36
GitHub: daretmavi
In-game: Daretmavi

Re: The future of Minetest

by daret » Post

bzt wrote:
Wed Nov 03, 2021 20:57
daret wrote:
Wed Oct 27, 2021 21:54
Dynamic tools / animated - like bows, compass, ... - item picture change without the animation
Not sure what you mean by "without the animation". Item pictures can already change.
I mean this:
https://youtu.be/eVVqdGcnMcM
- look at the compas
- if you change the compas image, then the draw back and up animation is issued

Similar problem is, if you want do simulate bow stretching.
bzt wrote:
Wed Nov 03, 2021 20:57
daret wrote:
Wed Oct 27, 2021 21:54
Leashes, Fishing - I never saw a good mod for it. I believe, the "rope" needed for fishing rod / leashes is not possible, or?
I believe fishing must be a mod. Maybe the Lua API needs a bit tweaking if that's the reason why there isn't any good mod (I haven't debugged any of the fishing mods myself, so I'm not sure what's missing), which leads us back to one of my points, Lua API needs some care and love.
If you find some good fishing mod, then let me know.
bzt wrote:
Wed Nov 03, 2021 20:57
daret wrote:
Wed Oct 27, 2021 21:54
World size - 65500 (or so) is big, but far from infinite.
There was some discussion in 2014, but think this is not changed, or?
That's 2^16-1 (65535). I personally don't think this is a real limitation. I have never wandered off the edge so far :-)
On server with many users, it could be a bit limited, or for some not "Minecraft like" game.
If Minetest is an engine, then game developers should decide, what is enough and if the engine allows more, then it is better so.

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

daret wrote:
Sat Nov 06, 2021 16:00
I you want to copy Minecraft, you should be able to do so. The engine should not limit you or make something impossible.
No, that's not so. You legally can't do that. If you would create an engine that would allow you to create a 100% perfect Minecraft clone, then Microsoft's lawyers would be on your doorstep in no time. You don't want to get in a court fight with one of the richest corp who pays more to lawyers than to its programmers.
daret wrote:
Sat Nov 06, 2021 16:00
if you change the compas image, then the draw back and up animation is issued
I see. This is indeed annoying, but hardly an issue for the future of Minetest. You should open a bugfix request for this.
daret wrote:
Sat Nov 06, 2021 16:00
If Minetest is an engine, then game developers should decide, what is enough and if the engine allows more, then it is better so.
But the engine doesn't allow more, and for a reason. I don't think you realize how big a 65536^3 world is. Just consider this: on the floor level only, taking a one node tall slice, there are over 4 billion nodes. Now imagine you have 65536 times more, as the world expands vertically too. The total volume is something about 281 trillion nodes, that's a pretty huge world.

Cheers,
bzt

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: The future of Minetest

by Blockhead » Post

bzt wrote:
Sun Nov 07, 2021 19:50
If you would create an engine that would allow you to create a 100% perfect Minecraft clone, then Microsoft's lawyers would be on your doorstep in no time
Not quite, they would show up if you actually made the Minecraft clone and not if you made something else entirely that just happens to have the same potential.
bzt wrote:
Sun Nov 07, 2021 19:50
I don't think you realize how big a 65536^3 world is. Just consider this: on the floor level only, taking a one node tall slice, there are over 4 billion nodes. Now imagine you have 65536 times more, as the world expands vertically too. The total volume is something about 281 trillion nodes, that's a pretty huge world.
This is really convoluted way to spin things to make it sound big, and try to excuse engine limitations. Most of the vertical space is wasted here. Hacking vertically separated dimensions in is one way to fill it, but it's still a dirty hack.

Let's run some numbers: if you want to have a fast means of transport such as a train that goes 30 m/s (which the fastest ones in advtrains do), then you can travel from one edge to another without stopping in 35 minutes. If you wanted to make a 1:1 replica of the real world, well, that's not a particularly long train trip in real life, nor a particularly fast one really when high speed rail is easily capable of 60 m/s. Compare this to 60 Mm or 1.5x the circumference of earth at the equator in Minecraft, that's equivalent to 23 days non-stop travel.

Stuff shouldn't be just big enough that nobody will complain until the server is a decade old and all the free land is taken. It should be as big as you want to make it, even if you want to go in a straight line. Yes, we have to break backwards compatibility and refactor a lot of code for this, but it's a hot topic and we can't deny the world size limitation hampers Minetest.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

ShadMOrdre
Member
Posts: 1118
Joined: Mon Dec 29, 2014 08:07
Location: USA

Re: The future of Minetest

by ShadMOrdre » Post

Why not make a route for the fast train that snakes through the other 59 km of wasted space, over and above, bridges and tunnels. Even on a single layer of terrain, there is a possibility of making train tracks that far exceed a single 35 minute trip. It just takes imagination, and the time to lay that much track. ;)

I agree that the vertical space is wasted. How much space could be gained by refactoring the vertical height into layers, whether stacked or by a realignment of the coords system? Even if layers were limited to 256m, 240 60x60km squares don't add up to 60 million meters squared.

The only way to fix that is to recode the engine, coding for backwards compatibility, and future expansion. If only....

The truth is, all software projects have one of two outcomes: Move into the future with a massive "breaking" update that essentially means a new and improved version 2.0 of the project, OR the mass exodus of those involved, whom loose interest and involvement and thus cause the the project to fade. This is a sad truth.

As long as the core dev group for the engine remains a volunteer community effort, things will continue along as more a hobbyist fascination than an outright powerhouse voxel engine. This isn't meant to be disparaging to anyone. It is a simple observation by one hobbyist, about the community of hobbyist with whom he is a member. None of us have the requisite support system available to us that would enable anything more dedicated. And face it, MT doesn't have the name recognition. While maintaining the idea of MTG as a game, and thus MT as a game engine, MT sells itself short of far greater potential.

IMHO, the MT community seems so dedicated to the idea of MTG that it completely misses the big picture idea of a truly open, free, and programmable voxel engine. We all seem stuck on "games", dissing the lack of "games" features while dissing any possible breaking updates, while trying to maintain some resemblance to a 10+ year old proprietary game programmed in Java.

Are my statements going to change anything? No. As much as I believe in what I say, it is only by my action that I can effect change. While speaking a truth is an action, a physical world needs physical effort. We all want something from this project, but only a few do the grunt work. As a "hobbyist" volunteer community effort, this will not change.

I'll still play as long as it's playable on the PCs I have or will have.

I believe that is this singular fact that is what will define the future of MT for all of us.

Shad

User avatar
daret
Member
Posts: 136
Joined: Tue Nov 12, 2019 20:36
GitHub: daretmavi
In-game: Daretmavi

Re: The future of Minetest

by daret » Post

bzt wrote:
Sun Nov 07, 2021 19:50
No, that's not so. You legally can't do that. If you would create an engine that would allow you to create a 100% perfect Minecraft clone, then Microsoft's lawyers would be on your doorstep in no time. You don't want to get in a court fight with one of the richest corp who pays more to lawyers than to its programmers.
You actually can "create a 100% perfect Minecraft clone" with C or Java o Python or anything. This is just a question of effort. If you make an engine, that makes it easier, this doesn't mean that all lawyers are going after you.
Fist you have to create a "perfect Minecraft clone", than the lawyers are going after you.
Maybe Mineclone is in danger, but not Minetest.
bzt wrote:
Sun Nov 07, 2021 19:50
daret wrote:
Sat Nov 06, 2021 16:00
if you change the compas image, then the draw back and up animation is issued
I see. This is indeed annoying, but hardly an issue for the future of Minetest. You should open a bugfix request for this.
See
https://github.com/minetest/minetest/issues/9377 (open)
https://github.com/minetest/minetest/issues/10955 (closed - duplicate)

partially related
https://github.com/minetest/minetest/pull/10411 (closed - due to lack of response)

Something else closed
https://github.com/minetest/minetest_game/issues/1926 - Won't add (initialising these nodes should be done by the mod = annoying)

Some problems are just closed = no support from engine = every mod/game has to deal with it
bzt wrote:
Sun Nov 07, 2021 19:50
But the engine doesn't allow more, and for a reason. I don't think you realize how big a 65536^3 world is. Just consider this: on the floor level only, taking a one node tall slice, there are over 4 billion nodes. Now imagine you have 65536 times more, as the world expands vertically too. The total volume is something about 281 trillion nodes, that's a pretty huge world.
I can't say it better than @Blockhead has.

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

daret wrote:
Thu Nov 11, 2021 16:30
You actually can "create a 100% perfect Minecraft clone" with C or Java o Python or anything.
Yeah, technically you can, but legally you can't.
Blockhead wrote:
Thu Nov 11, 2021 06:34
Not quite, they would show up if you actually made the Minecraft clone and not if you made something else entirely that just happens to have the same potential.
I agree, but that's what daret asked for: "Minetest engine should support everything and more than Minecraft does" Supporting each and every Minecraft features would make it a clone.
Blockhead wrote:
Thu Nov 11, 2021 06:34
This is really convoluted way to spin things to make it sound big, and try to excuse engine limitations.
It's not just the engine, we're talking about machine limitations too. Machines can only work with limited precision and with limited and finite natural numbers, furthermore the size of memory is very limited too.

Have you ever considered that Minecraft is limited to just a fraction of what Minetest can do? 5120 x 5120 x 320 in best case (with "large" world). And that a typical Minecraft world is even smaller, only 864 x 864 x 320? Think about that too.
Blockhead wrote:
Thu Nov 11, 2021 06:34
It should be as big as you want to make it
How? If you want a bigger world, first I would like to hear your opinion on which data structure and indexing algorithm to use for the nodes (Minetest uses octrees). Take it from an expert, it is quite an achievement and an incredible feat that Minetest can handle 65536 x 65536 x 65536 big world in real-time!
ShadMOrdre wrote:
Thu Nov 11, 2021 08:25
I agree that the vertical space is wasted. How much space could be gained by refactoring the vertical height into layers, whether stacked or by a realignment of the coords system?
Let's assume vertical height of 512 is enough. That's 9 bits, meaning we have 7 remaining bits for layers (nether, end, solar systems whatever). That's 2^7 = 128 different layers, not bad I think.

Cheers,
bzt

ThorfinnS
Member
Posts: 311
Joined: Mon Feb 25, 2019 22:05
GitHub: ThorfinnS

Re: The future of Minetest

by ThorfinnS » Post

Huh. I thought Minecraft had a max of something like 30 million blocks per side, and only 256 (or maybe 255) high.

Minetest world is only 65 km on a side. 40 miles. IRL, a three-day hike, maybe two if you are in good shape. That's why I've thought it makes more sense for a pre-industrial world, where 40 miles isn't that much of a restriction.

But I agree recoding everything to long long or whatever is not anything I'd care to do, neither would I find it of any use, because I have no interest in modeling a modern world. If I wanted to model commercial aircraft, that might be an issue, going from one side of the MT world to the other in 6 minutes. Assuming you had a machine butch enough to do the mapgen or even the block loading.

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: The future of Minetest

by Blockhead » Post

Hi bzt, I hope you have some time, strap yourself in, get a drink..
bzt wrote:
Thu Nov 11, 2021 19:34
Blockhead wrote:
Thu Nov 11, 2021 06:34
This is really convoluted way to spin things to make it sound big, and try to excuse engine limitations.
It's not just the engine, we're talking about machine limitations too. Machines can only work with limited precision and with limited and finite natural numbers, furthermore the size of memory is very limited too.
Machines can work with arbitrary precision bounded only by available memory. This is how operations on very very large numbers, like finding the world's largest prime numbers are done. It is not just extensible to the natural numbers, or even all integers. This same process can be applied to extremely accurate fractional parts of a number as well, such as trillions of digits of pi. These arbitrary precision operations are not native to the machine but they are achieved by packing several CPU-native datatypes into contiguous memory and operating on them as if they have place value. GMP is one such library.

Of course, just because we can work with arbitrary precision numbers, that does not mean we should. It becomes quite slow to work on these numbers as their size increases. Thankfully, there is no need to worry because I am not suggesting we make coordinates anywhere near this large. log2(60000000) (60 million) = 25.838, meaning that just 26 bits can store a Minecraft coordinate in one dimension, for a total of 78 bits to store all three dimensions. On a real machine we pad these numbers out to 32 bits each. 32-bit arithmetic is not any slower than 16-bit arithmetic, nor would 64-bit arithmetic be much slower, other than if we need to be doing so much of the longer bitwidth arithmetic that we are loading from main memory too often and this is bottlenecking performance.

The size of main memory (RAM) is small yes, but storage (hard disks, solid state drives) is much, much larger. Hence there are a few things that are important to do in any game engine but particularly block games.

First is that the portion of the entire world that is saved to storage is only a tiny fraction of its theoretical size, because exploring such a large volume (as you established it is extraordinary) takes a long, long time. Another space optimisation that other engines such as Craft make is that only modified chunks are actually recorded to storage. This approach costs time to regenerate chunks though, so for decently complex Minetest worlds, loading from storage is preferred over generating the mapblocks again, due in parts to CPU usage of generating mapblocks but also so CPU isn't hogged doing mapgen over and over again.

Your point about world size actually starts to become a valid one when you realise that given enough horizontal space to expand to, players of block games are well-known to actually go hundreds of thousands or even millions of blocks in one direction, and this gets the world file into the terabytes in size range. Server admins will still want to limit world size according to available storage, including settings that disable mapgen beyond those limits.

A second optimisation that is critical for block game engines is that in case of any real exploration ever having been done on your world, most of those mapblocks are not even loaded at any given time. The server has a configurable radius around each player over which it will simulate what is happening in those mapblocks. Some engines also have ways to simulate more such as 'chunk loaders' which force an area to always be loaded, but this exception proves the general rule.

Thirdly, out of those mapblocks that have been recently loaded, some are kept around, cached in main memory, for a while after they have been loaded even if that area is left. This makes loading recently-travelled-to areas quicker. Minetest then has a configurable maximum number of these mapblocks to keep in memory, and a timeout after which they will be unloaded regardless of the limit.
bzt wrote:
Thu Nov 11, 2021 19:34
Have you ever considered that Minecraft is limited to just a fraction of what Minetest can do? 5120 x 5120 x 320 in best case (with "large" world). And that a typical Minecraft world is even smaller, only 864 x 864 x 320? Think about that too.
I must point out here that there is a massive disclaimer at the top of that page that says that it refers specifically and only to Legacy Console and Nintendo 3DS editions. To claim it applies to anything else like Java or Bedrock editions is a misunderstanding. I can't blame you for clicking on that page if you searched for "Minecraft world size" in your search engine and clicked on that and thought it was what you were after. It's a failing of search engines and the Minecraft wiki in this case that you didn't get the correct information for Java/Bedrock. Now here is a relevant reference that will explain as I said earlier that Minecraft's actual world size is 60 million blocks in the horizontal: World border.
bzt wrote:
Thu Nov 11, 2021 19:34
Blockhead wrote:
Thu Nov 11, 2021 06:34
It should be as big as you want to make it
How? If you want a bigger world, first I would like to hear your opinion on which data structure and indexing algorithm to use for the nodes (Minetest uses octrees). Take it from an expert, it is quite an achievement and an incredible feat that Minetest can handle 65536 x 65536 x 65536 big world in real-time!
I have already established how it doesn't simulate the whole world the whole time, only a tiny fraction.

Indexing mapblocks in Minetest is and shall remain the job of the database system that stores all of them into the database for us. To increase world size then, we must refactor the data structure that holds mapblock coordinates to have larger fields for x and z. This isn't the only possible refactoring, but I will discuss that later.

As is I think that each dimension has 12 bits (log2(30192*2) = 15.88)) of information, and these are packed into a 64-bit integer. My calculation is as follows:

Mapblock coordinates must be multiplied by 16 to retrieve the real coordinates.
The world size is -30912 to +30912. 30912*2 = 61824.
61824 / 16 = 3864
log2(3864) = 11.91
So each dimension has 12 bits of information packed into that data structure.

One naive way to refactor would be to just expand each dimension as far as our 64 bit data structure allows. However, minetest uses a v3s16 (vector of 3 signed 16-bit integers) extensively for its full-coordinate system (this system is the in-engine representation of coordinates as opposed to the in-database mapblock coordinate system I previously described) so expanding beyond 16 bits would require more refactoring. However, quadrupling the size of each coordinate I feel wouldn't add enough space to make the change worth it. A v3s32 would probably be enough though.

64/3 = 21.333 so up to 21 bits in each of these would still fit into a 64-bit packed data type. 2^21*16 = 33,554,432. This would fit snugly into the v3s32 data type as well. So our world would be a bit over half of Minecraft's size in the horizontal and vertical and probably too vast in the vertical, unless we insist on keeping the vertical space for dimension hacks.

You can divide these bits however you want really. Getting more drastic, you could add more horizontal space by sacrificing vertical space (and I think we already have enough vertical space), and end up with a world much larger than Minecraft's. If you wanted to be tricky you could allocate some of the bits to a 4th dimension, but then you'd need to refactor Minetest engine to properly support dimensions. This has been discussed but never followed through with.

One division of the whole 64 bits I might suggest is 23 bits in each of x and z, 12 in y, and 6 in D. This totals horizontal space of 8.3 million in the horizontals, existing 60k horizontal which is generous and backwards-compatible, and 64 dimensions which is generous enough for Minecraft-like games that only have 3 builtin and maybe a dozen modded dimensions. Only if you wanted to do more advanced things like using dimensions as swap space, or having per-user dimensions or procedurally-generated dimensions might you have a problem with this.

I have discussed the problems and potential solutions but I will not insist that one particular is 'correct'. I think if such a big undertaking is actually committed to, it will only be after consensus is established about how to do it. I can't established consensus on my own. The only thing I would say for sure is that we need to move to v3s32 for coordinates and use more bits for at least x and z in the mapblock coordinate data structure, with the exact number of bits extra and whether we allocate bits to a 4th dimension to be decided by consensus.

Edit: One thing I neglected to discuss was what happens to coordinates if we add a fourth dimension. There are a few solutions, either include it in the coordinate vector and make it a v4s32, a specialised v3s32+s8 or similar, or leave it out of most of the code where possible and only where it is relevant provide it as a second function parameter/struct member or similar.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
firefox
Member
Posts: 1709
Joined: Wed Jan 14, 2015 07:34
In-game: Red_Fox
Location: Xanadu

Re: The future of Minetest

by firefox » Post

this is all very interesting, but why?
why would this be a quality that we need to achieve?

a game of chess can potentially last an infinite number of turns, but that doesn't make chess any more interesting.

do you have any purpose for this, other than refusing to change direction?
✨🏳️‍🌈♣️✨

User avatar
Blockhead
Member
Posts: 1622
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: The future of Minetest

by Blockhead » Post

firefox wrote:
Fri Nov 12, 2021 05:56
this is all very interesting, but why?
why would this be a quality that we need to achieve?

a game of chess can potentially last an infinite number of turns, but that doesn't make chess any more interesting.

do you have any purpose for this, other than refusing to change direction?
The chess analogy is bad because it refers to infinite time not space. We also have infinite time available to play Minetest (maybe we wish anyway :). And instead of chess, which we all know is fine as a bounded game, think of Conway's game of life. Sure, if you play it on a 20x20 grid you can see a lot of the interesting things, but unless you scale it up to a much larger scale you can't realise its full potential, such as the fact that's it's Turing complete and you can make life-in-life with it.

As for why: Yes, we can do great things in a 60x60x60 km area, but there are some things we can't do without more space. I mentioned earlier the '35 minute train trip'. The rebuttal from ShadMOrdre was that you shouldn't do it in a straight line. That's a good point for a lot of use cases. Playing creatively within limitations can be great. But as Bill Gates famously never actually said "640K of RAM should be enough for everyone".

Well, as it turns out, 60 km x 60 km is a whole lot smaller than a lot of places in real life. I want to mention real life here because it's been established elsewhere that block games can be representative of real life, quantized down to a cubic metre or so. This has been used for community engagement as in Minecraft Block by Block. I also would like to be able to see biomes at a larger, even if not realistic scale, and more distinct continental landmasses as opposed to amorphous blobs of ocean/land.

Is it ultimately necessary? You're right, it's not. Working within what we have 'should' be enough for everyone. I just think it adds more potential.

Anyway I'm thinking all of this recent flurry of writing should have gone in the infinite world thread, so I'll refrain from keeping on posting here over just one point.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

ShadMOrdre
Member
Posts: 1118
Joined: Mon Dec 29, 2014 08:07
Location: USA

Re: The future of Minetest

by ShadMOrdre » Post

@Blockhead,

Continents? mg_earth.

Better biomes? gal.

I'd like to hear your opinion on both.

Shad

User avatar
daret
Member
Posts: 136
Joined: Tue Nov 12, 2019 20:36
GitHub: daretmavi
In-game: Daretmavi

Re: The future of Minetest

by daret » Post

bzt wrote:
Thu Nov 11, 2021 19:34
daret wrote:
Thu Nov 11, 2021 16:30
You actually can "create a 100% perfect Minecraft clone" with C or Java o Python or anything.
Yeah, technically you can, but legally you can't.
Blockhead wrote:
Thu Nov 11, 2021 06:34
Not quite, they would show up if you actually made the Minecraft clone and not if you made something else entirely that just happens to have the same potential.
I agree, but that's what daret asked for: "Minetest engine should support everything and more than Minecraft does" Supporting each and every Minecraft features would make it a clone.
bzt, we don't understand each other, but we are close :D
I distinguish between engine features and their implementation.

E.g.
If Minetest engine implements the dimensions functionality (different separated worlds). Does it infringing Minecraft copyright? I don't think so.
If you implement unlimited world. Does it infringing Minecraft copyright? I don't think so.

If you create a game with minetest engine, that looks like Minecraft, uses the same textures and all. Does it infringing Minecraft copyright? I would say yes.

If you create a game with minetest engine, that looks similar to Minecraft, uses the diferent textures has is own code (see Mineclone). Does it infringing Minecraft copyright? Not sure, probably not, but lawyers could hassle you. See problems of re3 a reVC, witch are basically clones of GTA III and GTA: Vice City.

bzt
Member
Posts: 217
Joined: Tue Sep 24, 2019 14:26

Re: The future of Minetest

by bzt » Post

Blockhead wrote:
Fri Nov 12, 2021 03:39
Machines can work with arbitrary precision bounded only by available memory.
That's not so. Every CPU has a limited set of finite data types, everything else needs complex algorithms, unsuitable for real-time calculations.
Blockhead wrote:
Fri Nov 12, 2021 03:39
Of course, just because we can work with arbitrary precision numbers, that does not mean we should. It becomes quite slow to work on these numbers as their size increases.
That's what I'm saying, yes. Unsuitable for real-time calculations.
Blockhead wrote:
Fri Nov 12, 2021 03:39
32-bit arithmetic is not any slower than 16-bit arithmetic, nor would 64-bit arithmetic be much slower
That's where you're mistaken. The arithmetic itself isn't slower, true, but since you can't keep the entire world in the memory, you need some indexing algorithm (Minetest uses octrees, as I've already pointed out). And for that indexing algorithm each bit means another additional level in the tree hierarchy, which directly increases the required number of operations to address a node. (Octree's asymptotic behaviour depends on the number of levels in the tree.)
Blockhead wrote:
Fri Nov 12, 2021 03:39
Your point about world size actually starts to become a valid one when you realise that given enough horizontal space to expand to, players of block games are well-known to actually go hundreds of thousands or even millions of blocks in one direction, and this gets the world file into the terabytes in size range.
Exactly.
Blockhead wrote:
Fri Nov 12, 2021 03:39
I can't blame you for clicking on that page if you searched for "Minecraft world size" in your search engine
Yes, I did.
Blockhead wrote:
Fri Nov 12, 2021 03:39
bzt wrote:
Thu Nov 11, 2021 19:34
Blockhead wrote:
Thu Nov 11, 2021 06:34
It should be as big as you want to make it
How? If you want a bigger world, first I would like to hear your opinion on which data structure and indexing algorithm to use for the nodes (Minetest uses octrees). Take it from an expert, it is quite an achievement and an incredible feat that Minetest can handle 65536 x 65536 x 65536 big world in real-time!
I have already established how it doesn't simulate the whole world the whole time, only a tiny fraction.
You don't understand. I'm talking about the algorithm to be used for indexing (both in memory and on storage), and about that algorithm's memory requirements (the housekeeping indexing tables' size) and asymptotic behaviour. With octrees for example, using 16 bit coordinates are twice as fast as using 32 bit coordinates (because the tree is just half tall). There's a limit how much calculation a CPU can do and still keep the game lag-free at 30 fps. Plus with larger coordinates, the size of the meta data you need on storage increases exponentially, meaning slower reading / writing time on the database (increasing from 16 bit to 17 bit means twice the size, increasing to 32 bit means tens of thousand times more, assuming the database is using the fast binary tree index on the key).
Blockhead wrote:
Fri Nov 12, 2021 03:39
64/3 = 21.333 so up to 21 bits in each of these would still fit into a 64-bit packed data type. 2^21*16 = 33,554,432. This would fit snugly into the v3s32 data type as well. So our world would be a bit over half of Minecraft's size in the horizontal and vertical and probably too vast in the vertical, unless we insist on keeping the vertical space for dimension hacks.
Your math is off. You completely forget that that packed 2^64 id must be indexed by sqlite3 (or whatever database engine is used) too to access the data record in the database. Don't mind how you convert the coordinates into a 32 bit or 64 bit integer, focus on how you find the associated data for that integer, that's the bottle-neck. (I've used sqlite3 a lot, it works pretty well if there are no more than 100000 records in a table. It becomes tricky with a few million records, and a billion record is very very problematic if speed matters.)
Blockhead wrote:
Fri Nov 12, 2021 03:39
One thing I neglected to discuss was what happens to coordinates if we add a fourth dimension. There are a few solutions, either include it in the coordinate vector and make it a v4s32, a specialised v3s32+s8 or similar, or leave it out of most of the code where possible and only where it is relevant provide it as a second function parameter/struct member or similar.
Or, you could just chop off a few bits from the height coordinate because we don't need the entire vertical space anyway.
ThorfinnS wrote:
Fri Nov 12, 2021 01:00
Huh. I thought Minecraft had a max of something like 30 million blocks per side, and only 256 (or maybe 255) high.
That wiki page says vertically you can go from -64 to 256 (that's 320 in total), but since Blockhead told me that wiki page isn't correct, I'm not sure about this any more.
daret wrote:
Fri Nov 12, 2021 13:04
I distinguish between engine features and their implementation.
You were talking about "implementing everything that Minecraft does", I was reflecting on that you legally cannot do. Of course you can implement similar features, no problem with that. (To be perfectly correct, you can implement everything exactly as in the original game, but only as long as your engine isn't playable at all without the original game data files, which aren't distributed with your engine of course.)

Cheers,
bzt

User avatar
daret
Member
Posts: 136
Joined: Tue Nov 12, 2019 20:36
GitHub: daretmavi
In-game: Daretmavi

Re: The future of Minetest

by daret » Post

bzt wrote:
Fri Nov 12, 2021 20:07
daret wrote:
Fri Nov 12, 2021 13:04
I distinguish between engine features and their implementation.
You were talking about "implementing everything that Minecraft does", I was reflecting on that you legally cannot do. Of course you can implement similar features, no problem with that. (To be perfectly correct, you can implement everything exactly as in the original game, but only as long as your engine isn't playable at all without the original game data files, which aren't distributed with your engine of course.)
No, I did not. I know, my English is far from perfect, but try to reread my posts.
daret wrote:
Wed Oct 27, 2021 21:54
Maybe some "small" goal could be, that Minetest engine should support everything and more than Minecraft does.
I never said "implementing everything that Minecraft does".

User avatar
PolySaken
Member
Posts: 817
Joined: Thu Nov 09, 2017 05:18
GitHub: PolySaken-I-Am
In-game: PolySaken
Location: Wānaka, Aotearoa
Contact:

Re: The future of Minetest

by PolySaken » Post

I agree with daret here, what they're saying is that minetest should be technically capable of everything the minecraft engine can do, not that minetest should implement any of minecraft's content.
Guidebook Lib, for in-game docs | Poly Decor, some cool blocks | Vision Lib, an all-purpose library.

User avatar
runs
Member
Posts: 3225
Joined: Sat Oct 27, 2018 08:32

Re: The future of Minetest

by runs » Post

I am tired of years and years of endless useless discussions.

The only solution would be to run a fork of at least 10 developers.

Impossible.

Post Reply

Who is online

Users browsing this forum: Majestic-12 [Bot] and 11 guests