[Server] VanessaE's Survival Server [5.4.0]

cy
Member
Posts: 66
Joined: Sun Jun 24, 2012 17:25

Re: [Server] VanessaE's Survival Server [0.4.13]

by cy » Post

VanessaE wrote:I have disabled the pipeworks mod and most of technic. They're too ripe for abuse in a multiplayer environment. :-(
Might I ask how they were abused, specifically? It should be safe to say, since they were disabled. The greatest thing I ever enjoyed about minetest was the automation and chain reactions. It's a crying shame that they have to be gutted now.

Was it just non-abusive players trying to use pipeworks, and pipeworks was too heavy a load on the server? The solution for that is what I was hoping for, before everyone suddenly stopped using technic. It's an obvious, but non-trivial solution.

Currently LuaEntitySAO repeatedly sends "update position" messages, and otherwise, client-side, the objects continue at their current velocity (thus stuff drifting out of pipes when it's lagging). What needs to happen is the server needs to send an abstract path through space to the client, and then a single "this entity follows this path over this time interval, ending up here" message. Stuff going through pipes is purely cosmetic, and having the server calculate current velocity vectors is as insane as the server calculating vectors of individual particle velocities. The server should just figure where the entity is going to go (into what chest, or where it'll pop out), and when it will get there, wait that amount of time and then just move it all at once. That'd reduce the messages sent per entity from hundreds to 1 (plus 1 to set up the path when the pipes are placed).

Oh, a hack solution would be to take out the entity animating code from pipeworks, and just have the pipes switch nodes with a "full" pipe when they have an entity in them, instead of using entities at all.

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

It's not really just one thing by itself, it's a collection of things -- the biggest issue came from long multi-kilometer runs of cable, dozens and dozens of nuclear reactors ganged together in places, multiple quarries used together to dig larger holes, and so on. Lots and lots and LOTS of cabling just all over the place causes technic to churn and churn on it, trying to compute its electrical networks. Also, there were large setups in pipeworks that caused similar issues, though smaller than technic. There was enough stuff in use that it sometimes took the server as much as a full minute to start up, and 30-60 second lag spikes whenever loading a chunk of map triggers a large network segment to be loaded up.

I agree with your path idea. I had originally suggested a "move until coordinates foo reached, then stop" approach, but I am not sure if the engine was ever extended to allow for that (or if it was, pipeworks has not been rewritten to use it). Your idea is better.

Switching between "full" and "empty" tubes would destroy the look of the mod though - 90% of that comes from seeing items flowing through them, as it is meant to somewhat mimic similar mods in Minecraft.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

cy
Member
Posts: 66
Joined: Sun Jun 24, 2012 17:25

Re: [Server] VanessaE's Survival Server [0.4.13]

by cy » Post

VanessaE wrote:It's not really just one thing by itself, it's a collection of things
Death by a thousand cuts. Well, that's good though, because it means there wasn't some huge unassailable exploit. So maybe it can get working later.
The biggest issue came from long multi-kilometer runs of cable, dozens and dozens of nuclear reactors ganged together in places, multiple quarries used together to dig larger holes, and so on.
Well, nuclear reactors aren't honestly a huge load are they? There's only one ABM for the reactor core, that doesn't fire too often, and it isn't too expensive to check for nearby blocks at known coordinates. Granted the use of nuclear reactors to power things like massive furnace arrays could slow things down.
Lots and lots and LOTS of cabling just all over the place causes technic to churn and churn on it, trying to compute its electrical networks.
Okay, so... if I understand right, the switching station follows along all the wires it's connected to, and finds all the producers, receviers, and batteries in the network, not only on construction, but every single second via an ABM? That's so fixable! The ABM should just be iterating over a pre-calculated list of machines. Whenever a technic node is broken, if it has a switching station, then the station traverses the network and updates that list. Whenever a technic node is constructed, it looks for a switching station in the same way, and if it finds one, does the network traversal. If not, it's offline.

Heck, there's a second ABM that fires once a second, for every machine, that disables it if it hasn't been checked often enough. That's something you should put in on_construct and on_destruct too! When a technic node is broken, you follow the network to see if removing it would disable any machines, then update the list, disable the machines, and no need to check every second if the network hasn't changed.

It might be a bit tricky to traverse the "network" with a broken node, since that could split it in two, but putting that stuff in construct and destruct is what would keep large, complex technic networks from bogging down the server. Traversing an entire network of connections, every second? Yeesh.
I agree with your path idea. I had originally suggested a "move until coordinates foo reached, then stop" approach, but I am not sure if the engine was ever extended to allow for that (or if it was, pipeworks has not been rewritten to use it). Your idea is better.
Well, sending a "follow path" message as an ActiveObjectMessage packet is easy enough, just have to add logic to look up, set and follow the path in content_cao.cpp. But sending the paths... some of them could be very large and complex. Minetest deletes its content downloaders when the game starts, so you can't send any new media or meshes or anything, so I'm not sure what the right way to do it would be. You could add a "AddPath" packet, to add a path into a list of paths on the client, where it can send objects. But if the path was too big to fit in a single packet... should you continue in the spirit of minetest, and stall at the beginning to download all possible paths, like it does with media and meshes? Then only the server operator add paths, which sort of defeats the purpose of players being able to lay pipes.

Or should some sort of path downloader be persisted, that can take large paths broken down into packets, and reassemble them as they come in? You'd have to save any objects following along unknown paths, so that when those paths come in, you could start animating the objects. But otherwise, it's pretty straightforward. Some limit on how fast path packets can be sent in would take care of hideously complex pipe arrangements, and since it's only once when the nodes are first loaded, it'd be a lot less bandwidth intensive than repeatedly sending the path every second as separate "update position" messages.
Switching between "full" and "empty" tubes would destroy the look of the mod though - 90% of that comes from seeing items flowing through them, as it is meant to somewhat mimic similar mods in Minecraft.
Yeah, thus why I said it'd be a hack. Until that path following is done client side, you won't be able to use technic at all, since servers can't handle calculating the position of all those thousands of entities every time someone starts up a quarry.

If the wires are more problematic though, then nerfing how pipes work won't solve the problem. Need to fix how it keeps re-calculating the same network of wires, first. That seems doable entirely on the server side, though.

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

cy wrote:
VanessaE wrote:It's not really just one thing by itself, it's a collection of things
Death by a thousand cuts. Well, that's good though, because it means there wasn't some huge unassailable exploit. So maybe it can get working later.
Yeah pretty much. Whatever the case, I'm leaving technic to others to work on. It's not my mod anyway. :P
Well, nuclear reactors aren't honestly a huge load are they? There's only one ABM for the reactor core, that doesn't fire too often, and it isn't too expensive to check for nearby blocks at known coordinates. Granted the use of nuclear reactors to power things like massive furnace arrays could slow things down.
As I recall (I haven't looked at the code in a long while), that ABM has to do area checks to make sure the reactor is built properly, and if necessary, trigger a meltdown. So, one or two reactors would be fine, but because technic uses forced loading and has world anchors to keep things alive, when you've got 50 (!) of them in a high capacity HV installation in one spot, another 20 over there, a couple here, 10 on the other end of the map, it all adds up.
Lots and lots and LOTS of cabling just all over the place causes technic to churn and churn on it, trying to compute its electrical networks.
Okay, so... if I understand right, the switching station follows along all the wires it's connected to, and finds all the producers, receviers, and batteries in the network, not only on construction, but every single second via an ABM?
I'm not exactly sure, but I do recall being told that it iterates through not just the map, but its internal tables, reading the map as needed and making/referring to its records of the circuits it has seen since the last restart.
Traversing an entire network of connections, every second? Yeesh.
Not just every second -- every time something significant on network has changed, which on a busy server could be dozens of times a second, depending on what people are doing and what kinds of automation are in place.
Well, sending a "follow path" message as an ActiveObjectMessage packet is easy enough, just have to add logic to look up, set and follow the path in content_cao.cpp.
Sounds reasonable, but good luck getting the core dev team to allow for such a feature. It would have to be something open-ended, generic, handled on-demand as players build. It would need to be able to send just the changes to the client (as well as accepting such from the mod that uses it), i.e. "add object at 123,456,789 to network 9876 and optimize it".
If the wires are more problematic though, then nerfing how pipes work won't solve the problem. Need to fix how it keeps re-calculating the same network of wires, first. That seems doable entirely on the server side, though.
One thing that would help is if there were a proper "cable like" draw type, so that the server wouldn't have to manage so many nodes when adding/removing things from a network segment. sofar has a pull request against the engine that adds "connected" nodeboxes that looks like it would work for this as well as pipeworks' tubes (one of the examples shown therein is mesecon wiring), and est31 has some ideas on better management of those internal network tables.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

cy
Member
Posts: 66
Joined: Sun Jun 24, 2012 17:25

Re: [Server] VanessaE's Survival Server [0.4.13]

by cy » Post

VanessaE wrote:Whatever the case, I'm leaving technic to others to work on. It's not my mod anyway. :P
Sure, no problem. I'm mostly just thinking out loud anyway, not actually expecting you to go reprogram technic for me.
that ABM has to do area checks
Yeah, but--
technic uses forced loading and has world anchors to keep things alive,
D8 ...say no more.
I'm not exactly sure, but I do recall being told that it iterates through not just the map, but its internal tables, reading the map as needed and making/referring to its records of the circuits it has seen since the last restart.
Its own internal tables (called technic.networks) are referred to as "cached". Now that I look at it, they do seem to cache in each network a list of machines, and only traverses the nodes, if the list for the current LV/MV/HV tier at the current position doesn't exist. That would make things a lot more efficient. Thanks I didn't notice that before.

It's still something that should only be calculated on construction or destruction (or if the node changes state in a way that could change the network). It still "touches" all the nodes (resetting their timeout). And it still relies on an ABM to repeatedly check that timeout, instead of a node timer which was sort of designed for that sort of thing. I'm... not sure how inefficient that would be actually, since node timers fire along with ABMs at player proximity.
Not just every second -- every time something significant on network has changed,
Yeah, I didn't see that they had cached their networks so as not to traverse every time.
which on a busy server could be dozens of times a second, depending on what people are doing and what kinds of automation are in place.
The only way to force a re-traversal is to move the switching station. Like with a piston. I've been ...guilty of doing that before. (What? It's great for disabling power hogs when your solar panels are offline!) I certainly would hope that even a busy server didn't have dozens of switching stations being pistoned around every second though. That's something I don't think anyone could expect to work without problems.
Sounds reasonable, but good luck getting the core dev team to allow for such a feature.
Yeah I'm not gonna sell the farm or anything.
open-ended, generic,
On that note, another use for paths might be more efficient mobs. You can animate the models currently, but you still can only set their basic velocity in one direction at a time. That's why mobs run through you instead of stopping sometimes, same as when items drift out of pipes. I could even conceive of "paths" that had some sort of logic to them, telling the client "this is an aggressive mob. Have it chase after the player!" So, yeah there are a lot of applications for "generic" paths.
handled on-demand as players build.
Yes, in the case of pipeworks, the path would be transmitted on construction of the pipe, or on loading of that chunk. Breaking a pipe would require calculating and sending new paths. What wouldn't be required is sending the path to the client, as a sequence of "go left, now go right, now go up, now go down" commands, every single time for all the possibly thousands of entities that ever enter the pipe.
It would need to be able to send just the changes to the client
That might be a good idea. But it might be premature optimization. The real problem is sending all the direction changes for all the entities every time. Even uploading the whole path, every time a pipe gets broke and reconstructed would be orders of magnitude more efficient than that.
One thing that would help is if there were a proper "cable like" draw type, so that the server wouldn't have to manage so many nodes when adding/removing things from a network segment.
Oh, so the server has to store geometry for all the faces of each cable segment? That certainly doesn't seem ideal. I thought only the client calculated the geometry, based on the mesh that the server uploads and then just forgets about.
sofar has a pull request against the engine that adds "connected" nodeboxes that looks like it would work for this as well as pipeworks' tubes (one of the examples shown therein is mesecon wiring),
I might check that one out...

pheonixfire
Member
Posts: 94
Joined: Sun Oct 21, 2012 06:25
In-game: pheonix
Location: Australia

Re: [Server] VanessaE's Survival Server [0.4.13]

by pheonixfire » Post

Hi Vanessa, is the community centre supposed to be a post office or
was it just a convenient place people put their mailboxes?

If it was just a convenient place, I've built a building across from the exchange
that I haven't decided what use for yet that could be used as a post office

pheonixfire

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

It was meant as a sort of multi-purpose building. Not really just one thing, per se.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

jaysned
New member
Posts: 4
Joined: Mon Jul 01, 2013 18:33

Re: [Server] VanessaE's Survival Server [0.4.13]

by jaysned » Post

Is there still a download link to the old map? I dug a bit through the Minetest section on your site and couldn't find a survival world download that wasn't current (new world). I had some buildings and construction strategies that I wanted to review. That, and I want to...sometime...revisit the stuff I made the last couple of years for nostalgic purposes.

User avatar
cheapie
Member
Posts: 316
Joined: Mon May 14, 2012 00:59
GitHub: cheapie
IRC: cheapie
In-game: cheapie

Re: [Server] VanessaE's Survival Server [0.4.13]

by cheapie » Post

jaysned wrote:Is there still a download link to the old map? I dug a bit through the Minetest section on your site and couldn't find a survival world download that wasn't current (new world). I had some buildings and construction strategies that I wanted to review. That, and I want to...sometime...revisit the stuff I made the last couple of years for nostalgic purposes.
It's not on her site, but I have it.

jaysned
New member
Posts: 4
Joined: Mon Jul 01, 2013 18:33

Re: [Server] VanessaE's Survival Server [0.4.13]

by jaysned » Post

Thank You!!!

User avatar
webdesigner97
Member
Posts: 1328
Joined: Mon Jul 30, 2012 19:16
GitHub: webD97
IRC: webdesigner97
In-game: webdesigner97
Location: Cologne, Germany
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by webdesigner97 » Post

Hi VE,
I just saw that you reset the map. Would it be possible to copy my old village onto the new map? I'm not playing that much these days, but maybe I'll find some time in some weeks... Just wanted to say that before you finally delete the old map... I'll give you the coordinates as soon as I find them.

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

Sure, just let me know where the area was. You can use this image to help you find it (X coords across the top, Z coords across the left):

https://daconcepts.com/vanessa/hobbies/ ... -12-15.png

(don't use your browser to look at it - download it instead and open it in The GIMP or something)

Either cheapie or I will get your buildings loaded in from the old map.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

User avatar
webdesigner97
Member
Posts: 1328
Joined: Mon Jul 30, 2012 19:16
GitHub: webD97
IRC: webdesigner97
In-game: webdesigner97
Location: Cologne, Germany
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by webdesigner97 » Post

Ok, thank you. I found my village within this area: (5897; 6359) -> (6012; 6280)
How are you going to do this? Building by building or the whole area at once? The latter option seems to be quite difficult due to the terrain...

mintpick
New member
Posts: 9
Joined: Fri Feb 20, 2015 18:16

Re: [Server] VanessaE's Survival Server [0.4.13]

by mintpick » Post

Hello
I am unable to connect to this server. Client console says that game is trying to download various resources from http://minetest.daconcepts.com/server-media/... but receives 404 response.
Game version: minetest-0.4.13-1474254-win64

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

Works fine for me - cleared my Minetest cache and connected to the server. All media downloaded fine, no 404's at all. A bunch of "dummy" image warnings related to pine wood and steel doors, but those have been there forever and aren't related to what you described.

Check your firewall, anti-virus, and whatever other things you have going on on your machine, and make sure nothing is blocking your access to my site. Some anti-virus programs incorrectly detect my site as being dangerous (not sure why, maybe my SSL certificate?). Check for that also.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

mintpick
New member
Posts: 9
Joined: Fri Feb 20, 2015 18:16

Re: [Server] VanessaE's Survival Server [0.4.13]

by mintpick » Post

Reinstalled game from clean. No antivirus software is present in system. Firewall is disabled. Still no luck.
Here's one of these strings:

Code: Select all

2016-02-25 19:49:21: ERROR[CurlFetch]: http://minetest.daconcepts.com/server-media/_re.png not found (HTTP response code said error) (response code 404)
http://minetest.daconcepts.com/server-media/_re.png gives me 404 even from browser.

EDIT: My friend is also unable to retreive that png image, from another computer and network.

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

Indeed the link you gave does not work. Looks like that particular texture is part of Jeija's digilines mod. Of course, not being on the media server, it probably was downloaded the "old" way, via Minetest's older, internal media transfer protocol.

It's still odd that I don't get any errors while retrieving that file when I started the client. Of course, files stored on a remote_media server are not named plainly like that, but rather with a hash value for the filename.

I have no idea why it isn't working for you, unless it's a bug in your build or in your cURL library maybe.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

mintpick
New member
Posts: 9
Joined: Fri Feb 20, 2015 18:16

Re: [Server] VanessaE's Survival Server [0.4.13]

by mintpick » Post

It was sfan5's build fault, connected from official msvc build just fine. Thank you.

What is the hardware this server runs at? I would like to host Dreambuild-based game in the future, so it will be good to know.

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

This describes the server's hardware: https://www.soyoustart.com/us/offers/e3-sat-3.xml
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

mintpick
New member
Posts: 9
Joined: Fri Feb 20, 2015 18:16

Re: [Server] VanessaE's Survival Server [0.4.13]

by mintpick » Post

Thank you!

User avatar
qwrwed
Member
Posts: 326
Joined: Sun Jul 22, 2012 20:56
In-game: qwrwed or Nitro

Re: [Server] VanessaE's Survival Server [0.4.13]

by qwrwed » Post

I'm also having trouble viewing the downloaded ~10GB old server map. I've set the mg_name to singlenode in the env_meta.txt, but there is nothing around me whatsoever when I first spawn in, and when I teleport to the spawn coordinates there is only this:
Spoiler
Image

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

Are you sure you're using the right backend? My maps are leveldb format. If you use the wrong backend (i.e. don't have leveldb support), MT will think it needs to create a new map and will ignore what's already there.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

User avatar
qwrwed
Member
Posts: 326
Joined: Sun Jul 22, 2012 20:56
In-game: qwrwed or Nitro

Re: [Server] VanessaE's Survival Server [0.4.13]

by qwrwed » Post

How do I make sure I'm using the right backend?

User avatar
VanessaE
Moderator
Posts: 4655
Joined: Sun Apr 01, 2012 12:38
GitHub: VanessaE
IRC: VanessaE
In-game: VanessaE
Location: Western NC
Contact:

Re: [Server] VanessaE's Survival Server [0.4.13]

by VanessaE » Post

Firstly make sure your Minetest build is compiled with support for it ("-DENABLE_LEVELDB=true" among the other cmake flags, if you build it yourself). In the world.mt file, find the backend line and set it to, "backend = leveldb" if it isn't already.

Make sure the permissions and ownership of the whole tree are correct, and make sure Minetest can read from the map.db folder therein, then the world should load.
You might like some of my stuff: Plantlife ~ More Trees ~ Home Decor ~ Pipeworks ~ HDX Textures (64-512px)

User avatar
qwrwed
Member
Posts: 326
Joined: Sun Jul 22, 2012 20:56
In-game: qwrwed or Nitro

Re: [Server] VanessaE's Survival Server [0.4.13]

by qwrwed » Post

I am using the stable 0.4.13 32-bit download (64-bit crashes with Dreambuilder) from the minetest.net page, the backend is set to leveldb, and the Minetest folder isn't anywhere that would need special permissions to access.
I have also tried sfan5's 32-bit build, but that also doesn't work; the console/debug.txt is filled with tens of thousands of lines of

Code: Select all

2016-04-02 03:56:21: WARNING[Server]: saveBlock: LevelDB error saving block (-3,-1,3): Corruption: corrupted compressed block contents

Post Reply

Who is online

Users browsing this forum: No registered users and 9 guests