minetest needs an RPC api

Post Reply
User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

minetest needs an RPC api

by ywwv » Post

so im thinking about mine test and the eocnomical idea that . we can just host servers on a raspberry pi. but some mods require more complicated server side computation. than just what a raspberry pi can do . this is a cost that the server owners have to burden with . but we can actually fix this . becuasethe clients are running very powerful gaming computers or. comptuers that are more powerfulthan the sererv. as a reasult they can donate their computing power similar to scientific computing distributed supercomputers like folding@home to offload computationally complex tasks on the server side to the clients. eventually the server/client distinction might even melt away as servers become fully decentralized and distributed. so a server is just a "headless node" because it has no . keyboard and mouse. but all of the functionality is there. this would obviously be done using RPC (remote procedure calls) from the server to the client in a master-slave system for now .

so this is a question of. if mine test has an RPC functionality what would you do with it? what games would you . make if you could do this . I am gauging interest from the community so this is kind of like a "poll" to understand how stifled w are byt the lack of this feature

User avatar
Festus1965
Member
Posts: 4181
Joined: Sun Jan 03, 2016 11:58
GitHub: Festus1965
In-game: Festus1965 Thomas Thailand Explorer
Location: Thailand ChiangMai
Contact:

Re: minetest needs an RPC api

by Festus1965 » Post

ywwv wrote:
Thu Sep 02, 2021 08:17
we can just host servers on a raspberry pi
WRONG !

where and what time are you from ?

we, is also me ... Intel CPU 4770 ... with mt-server .. tststs
Human has no future (climate change)
If urgend, you find me in Roblox (as CNXThomas)

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

Festus1965 wrote:
Thu Sep 02, 2021 08:55
ywwv wrote:
Thu Sep 02, 2021 08:17
we can just host servers on a raspberry pi
WRONG !

where and what time are you from ?

we, is also me ... Intel CPU 4770 ... with mt-server .. tststs
thats cool that you can afford a new computer. but we are currently in a comptuer chip shortage. many people are hosting computers on . raspberri pi computers . ive seen it

User avatar
MisterE
Member
Posts: 693
Joined: Sun Feb 16, 2020 21:06
GitHub: MisterE123
IRC: MisterE
In-game: MisterE

Re: minetest needs an RPC api

by MisterE » Post

I host a server on a raspi4 sometimes. works great

User avatar
Festus1965
Member
Posts: 4181
Joined: Sun Jan 03, 2016 11:58
GitHub: Festus1965
In-game: Festus1965 Thomas Thailand Explorer
Location: Thailand ChiangMai
Contact:

Re: minetest needs an RPC api

by Festus1965 » Post

ywwv wrote:
Thu Sep 02, 2021 10:14
thats cool that you can afford a new computer. but we are currently in a comptuer chip shortage. many people are hosting computers on . raspberri pi computers . ive seen it
and WRONG AGAIN !

my own rig was the last system I put from only main new components,
all others in the house
* server on the i7-4770 (just bought RAM and M.2)
* my daughters 5450 ... (RAM and SSD)
and 3 more for the kids when visit here (plus more RAM) (before Corona)

are USED ones, sold out from big companies and was quit cheap and work now 4 years already

open your mind for other options then newest and best,
learn how to bring together software requirements (Minetest is not multi, only mapgen) a server owners experience and the real needs (just fast hdd and a lot of ram) ... beside not to crazy mods
Human has no future (climate change)
If urgend, you find me in Roblox (as CNXThomas)

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

Festus1965 wrote:
Thu Sep 02, 2021 23:12
ywwv wrote:
Thu Sep 02, 2021 10:14
thats cool that you can afford a new computer. but we are currently in a comptuer chip shortage. many people are hosting computers on . raspberri pi computers . ive seen it
and WRONG AGAIN !

my own rig was the last system I put from only main new components,
all others in the house
* server on the i7-4770 (just bought RAM and M.2)
* my daughters 5450 ... (RAM and SSD)
and 3 more for the kids when visit here (plus more RAM) (before Corona)

are USED ones, sold out from big companies and was quit cheap and work now 4 years already

open your mind for other options then newest and best,
learn how to bring together software requirements (Minetest is not multi, only mapgen) a server owners experience and the real needs (just fast hdd and a lot of ram) ... beside not to crazy mods
sorry. but I think you've already been "debunked" by me and the other poster s in this thread. so im going to stop replying to you after this .

the purpose of this idea is to allow "crazy mods". there are limits and they should be . broken

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

an example of how this RPC API would be used is for weather simulation. its not reasonable to run weather simulations on the server at a large scale. also. players can generate their own chunks . this would also lower the CPU load on the server. when exploring part of the map you could generate your own chunks and just upload them.

hole punching could also be used to pass data packets between clients. but that's for later.

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

some of this would happen on the engine side. but lots of mods would use it too. its useful in general

User avatar
orwell
Member
Posts: 958
Joined: Wed Jun 24, 2015 18:45
GitHub: orwell96
IRC: orwell96_mt
In-game: orwell
Location: Raxacoricofallapatorius

Re: minetest needs an RPC api

by orwell » Post

I concur. I think the most critical problem with the MT Lua API is that it's single-threaded.

Advtrains could benefit from this by putting the train movement calculations into an external process, thus not causing lag on the MT server thread. We would only send train positions of trains near players to MT.

If we have a standard API on the engine, we could benefit from a C++ implementation instead of doing it manually in Lua.
Lua is great!
List of my mods
I like singing. I like dancing. I like ... niyummm...

User avatar
rubenwardy
Moderator
Posts: 6972
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: Bristol, United Kingdom
Contact:

Re: minetest needs an RPC api

by rubenwardy » Post

orwell wrote:
Fri Sep 03, 2021 09:04
I concur. I think the most critical problem with the MT Lua API is that it's single-threaded.

Advtrains could benefit from this by putting the train movement calculations into an external process, thus not causing lag on the MT server thread. We would only send train positions of trains near players to MT.

If we have a standard API on the engine, we could benefit from a C++ implementation instead of doing it manually in Lua.
This thread is about sending calculations to the client, which is a bad idea for security and practical reasons

See https://github.com/minetest/minetest/pull/11131 for server side worker threads

For a separate process, you could probably use the insecure API
Renewed Tab (my browser add-on) | Donate | Mods | Minetest Modding Book

Hello profile reader

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

rubenwardy wrote:
Fri Sep 03, 2021 10:38
orwell wrote:
Fri Sep 03, 2021 09:04
I concur. I think the most critical problem with the MT Lua API is that it's single-threaded.

Advtrains could benefit from this by putting the train movement calculations into an external process, thus not causing lag on the MT server thread. We would only send train positions of trains near players to MT.

If we have a standard API on the engine, we could benefit from a C++ implementation instead of doing it manually in Lua.
This thread is about sending calculations to the client, which is a bad idea for security and practical reasons

See https://github.com/minetest/minetest/pull/11131 for server side worker threads

For a separate process, you could probably use the insecure API
its not inscure. if it was the medican industry would have been able to sabotage folding@home with fake inforamtion . unless you think that's why they aren't making progress

ronoaldo
Member
Posts: 177
Joined: Mon Dec 07, 2020 01:04
GitHub: ronoaldo
IRC: ronoaldo
In-game: RonoaldoKakashi
Location: São Paulo, Brasil
Contact:

Re: minetest needs an RPC api

by ronoaldo » Post

RPCs in general (in any form, be it plain text over HTTP or binary serialization like gRPC) are great for distributed/microservices architectures but have you considered that, by issuing an RPC you will end up performing a NETWORK round trip and thus cause less performance?

You can already do something like this from the LUA side as ruben said to any service, using the HTTP API, so this could be implemented by I noticed a HUGE performance decrease just by placing network IO between the game server and the database, I imagine splitting the process can eventually cause even more LAG than less...
Servers: Mercurio | Tools: ContentDB CLI | Mods: minenews

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

ronoaldo wrote:
Thu Oct 14, 2021 19:26
RPCs in general (in any form, be it plain text over HTTP or binary serialization like gRPC) are great for distributed/microservices architectures but have you considered that, by issuing an RPC you will end up performing a NETWORK round trip and thus cause less performance?

You can already do something like this from the LUA side as ruben said to any service, using the HTTP API, so this could be implemented by I noticed a HUGE performance decrease just by placing network IO between the game server and the database, I imagine splitting the process can eventually cause even more LAG than less...
you're confusing latency with throughput. think more deeply. not every computation is realtime

ronoaldo
Member
Posts: 177
Joined: Mon Dec 07, 2020 01:04
GitHub: ronoaldo
IRC: ronoaldo
In-game: RonoaldoKakashi
Location: São Paulo, Brasil
Contact:

Re: minetest needs an RPC api

by ronoaldo » Post

I'm not. The idea you are proposing is going to impose the latency. If the trade-off is not worth (usually I/O is an order of magnitude higher) it is going to make things worse. You can use the insecure HTTP API to make a PoC and prove me wrong.
Servers: Mercurio | Tools: ContentDB CLI | Mods: minenews

rob123
Member
Posts: 105
Joined: Wed Mar 03, 2021 07:15
GitHub: qwerty123a2
In-game: joe and joergeg

Re: minetest needs an RPC api

by rob123 » Post

ywwv wrote:
Sat Sep 04, 2021 07:16
rubenwardy wrote:
Fri Sep 03, 2021 10:38
orwell wrote:
Fri Sep 03, 2021 09:04
I concur. I think the most critical problem with the MT Lua API is that it's single-threaded.

Advtrains could benefit from this by putting the train movement calculations into an external process, thus not causing lag on the MT server thread. We would only send train positions of trains near players to MT.

If we have a standard API on the engine, we could benefit from a C++ implementation instead of doing it manually in Lua.
This thread is about sending calculations to the client, which is a bad idea for security and practical reasons

See https://github.com/minetest/minetest/pull/11131 for server side worker threads

For a separate process, you could probably use the insecure API
its not inscure. if it was the medican industry would have been able to sabotage folding@home with fake inforamtion . unless you think that's why they aren't making progress


it is insecure. code in the game can run unsandboxed.
If my post says something, it is a opinion and not fact unless i say so

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

rob123 wrote:
Mon Oct 18, 2021 20:31
ywwv wrote:
Sat Sep 04, 2021 07:16
rubenwardy wrote:
Fri Sep 03, 2021 10:38


This thread is about sending calculations to the client, which is a bad idea for security and practical reasons

See https://github.com/minetest/minetest/pull/11131 for server side worker threads

For a separate process, you could probably use the insecure API
its not inscure. if it was the medican industry would have been able to sabotage folding@home with fake inforamtion . unless you think that's why they aren't making progress


it is insecure. code in the game can run unsandboxed.
im describing a sandbox. ok?

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: minetest needs an RPC api

by sorcerykid » Post

ywwv wrote:
Fri Sep 03, 2021 01:39
an example of how this RPC API would be used is for weather simulation. its not reasonable to run weather simulations on the server at a large scale.
This is something I had actually briefly considered several years ago while developing the weather simulator for the JT2 server. I had considered the possibility of large-scale whether being handled by an entirely separate process. Of course, the ideal implementation would take advantage of RPC.

viewtopic.php?p=309878#p309878

User avatar
Komodo
Member
Posts: 163
Joined: Tue Jan 11, 2022 13:33
GitHub: MeseCraft
In-game: Komodo
Location: God Bless America
Contact:

Re: minetest needs an RPC api

by Komodo » Post

Rubenwardy gave you a really good answer and you blew it off.

ywwv, I like the idea, its a very creative and thoughtful way to approach the problem you are trying to solve. You should be commended for thinking of that and using your noggin. But, it's not realistic to implement with current technology. The latency to do inter-net computational offloading to clients is not practical or useful for a real-time simulation engine. Rubenwardy mentioned the security aspect of this too, which is a great point.

As for widespread use of using Raspberry Pi's for hosting, I really think it's a bad idea. People bring up Rpis all the time, but they're not very cost effective. For about $100 you get a weak quad core ARM RISC cpu with poor thermal management and design. Currently, you're better off spending that $100 towards something like an older used laptop or used desktop with a dual core CPU with a higher single thread performance rating. x86 CPU's just have more processing power than the Pi's. They can usually have more memory and better storage interfaces too. The Raspberry Pi's are hobbyist tinkerer machines. They're a bit of a novelty and I'm not going to say they're not useful, but there are more effective options available for someone on a budget.

A laptop is probably one of the most cost effective ways to get started. it comes with a keyboard, power, display, ports, sound, battery (UPS), and memory slots. For example, a ThinkPad T430 with a common i5-3320M from 2013 is around $100-150. A Rpi4 with an Arm Cortex A72 @ 1.5GHZ with the accessories you need to get started is going to be around that much. Here is the benchmark comparing these CPU's The x86 is nearly x4 times as performant as the RISC cpu while being about 1x the price.

https://www.cpubenchmark.net/compare/In ... /817vs3917


The link that rubenwardy shared seems to be about the implementation of better worker threads and the possibility of using additional threads/cores for processing. This would be a very big improvement in performance since we can fully use the more common 4, 6, 8 core CPU's in the environment.

Also, as a server operator, I'd never implement your RPC api. I wouldn't trust it to give a good experience to players.
🌎 Website | 🌲 MeseCraft Game | 📰 News | 🖌️ ContentDB

Byakuren
Member
Posts: 818
Joined: Tue Apr 14, 2015 01:59
GitHub: raymoo
IRC: Hijiri
In-game: Raymoo + Clownpiece

Re: minetest needs an RPC api

by Byakuren » Post

folding@home works because the folding solutions are verifiable / scorable without having to find the solution again. It would be difficult to verify something like a weather simulation without running it yourself.
Every time a mod API is left undocumented, a koala dies.

c56
Member
Posts: 307
Joined: Wed Apr 21, 2021 03:05
GitHub: tigercoding56
In-game: bm5 or bemo5 also sell_her_on55

Re: minetest needs an RPC api

by c56 » Post

this sounds like if only 1 player is online they could change output of computations to theyr advantage
this is a signature not a place to post messages also if i could change my username i would change it to sell_her_on55

User avatar
ywwv
Member
Posts: 299
Joined: Mon Jan 18, 2021 11:51

Re: minetest needs an RPC api

by ywwv » Post

Byakuren wrote:
Sat Feb 26, 2022 08:37
folding@home works because the folding solutions are verifiable / scorable without having to find the solution again. It would be difficult to verify something like a weather simulation without running it yourself.
its called a ballot. we can decide thes e things like in athens. in the 1960s "computers" were also random people at their own desks. we solved this by delegating computations multiple times and comparing answers.

ShadMOrdre
Member
Posts: 1118
Joined: Mon Dec 29, 2014 08:07
Location: USA

Re: minetest needs an RPC api

by ShadMOrdre » Post

those distributed models were not real time in the same sense that MT is real time.

Those models used PC downtime towards computing data. IIRC, this had to work offline as well as online, thus, data was sent when connected, or otherwise according to the settings. The amount of data transferred varied, and was never used in real time by the client.

MT would face significant latency issues brought by using client processing.

Instead, the server itself should be able to connect to other servers. Second Life / Opensim is a far better model for making something like this feasible, without the latency. Distributed servers, not distributed computing.


Shad

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest