Better support for multicore CPUs

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Better support for multicore CPUs

by burli » Post

As far as I know games like minetest are more limited to the CPU than the GPU. Thats why I am a little bit disappointed, because just 50% of my quadcore CPU is used. I know that multithreading is not that easy, but it could improve the game.

I tryed num_emerge_threads = 2/3/4, but it has no effect to the performance or the CPU load.

Are there any plans for that?

User avatar
Calinou
Moderator
Posts: 3164
Joined: Mon Aug 01, 2011 14:26
GitHub: Calinou
IRC: Calinou
In-game: Calinou
Location: Troyes, France
Contact:

Re: Better support for multicore CPUs

by Calinou » Post

num_emerge_threads sets the number of threads to use for C++ map generation only, it won't affect mods and other areas of the engine.

gregorycu
New member
Posts: 3
Joined: Wed Jan 28, 2015 12:52
GitHub: gregorycu
IRC: gregorycu
In-game: gregorycu

Re: Better support for multicore CPUs

by gregorycu » Post

I'm always looking for ways to improve performance in ways that has the most positive impact on end users.

A question, are you noticing poor performance? Or are you just noticing minetest is not using 100% CPU?

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: Better support for multicore CPUs

by burli » Post

gregorycu wrote:A question, are you noticing poor performance? Or are you just noticing minetest is not using 100% CPU?
Well, if the game get's more content, like more plants, the performance drops. If you are just walking and a few blocks ahead is nothing because the mapgen lacks, thats annoying.

I don't have high end hardware, but a quad core with 16GB of RAM and a Nvidia GTX750 should be enough to run MT acceptable

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

You're saying that minetest is taking 50% of your CPU. That's likely 50% of one core. If you'd split that workload to multiple cores, then that would still consume (in total) the exact same amount of CPU cycles, but it would require extra locks, and those are expensive, and generally make performance worse.

So, making things more parallel isn't always better for performance.

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: Better support for multicore CPUs

by burli » Post

I say, the CPU runs at 200% (or less) of 400%. Highest value I ever saw was 240%, so I guess there are more than 2 tasks right now.

I know that it could be tricky to deal with threads. But one suggestion might be to use LevelDB instead of SQLite. As far as I know all operations are asynchronous. So LevelDB could run as task, if it is not already doing.

And I think, the Mapgen and Lua are the bottlenecks. I don't know exactly how Mapgens work, but maybe each mapchunk can be generated in a separate task. I don't know how much nodes are connected to the neighbors

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

Mapgen is already a separate thread. There are two mapgen threads to start by default.

User avatar
rubenwardy
Moderator
Posts: 6301
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

Re: Better support for multicore CPUs

by rubenwardy » Post

sofar wrote:Mapgen is already a separate thread. There are two mapgen threads to start by default.
[citation needed]

As far as I know, mapgen is done on the server thread as we need to support lua mapgens.
The emerge thread is for other things, I think it is for sending blocks to clients, not sure.

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

rubenwardy wrote:
sofar wrote:Mapgen is already a separate thread. There are two mapgen threads to start by default.
[citation needed]

As far as I know, mapgen is done on the server thread as we need to support lua mapgens.
The emerge thread is for other things, I think it is for sending blocks to clients, not sure.
ah, yes, you're correct. emerge threads are for loading, compressing and sending mapblocks to clients.

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: Better support for multicore CPUs

by burli » Post

Thread doesn't necessarily mean, that this thread runs on another core. Didn't look at the code

User avatar
Fixer
Member
Posts: 898
Joined: Sun Jul 31, 2011 11:23
IRC: Fixer
In-game: Fixer
Location: Ukraine

Re: Better support for multicore CPUs

by Fixer » Post

A question, are you noticing poor performance? Or are you just noticing minetest is not using 100% CPU?
IIRC, minetest does not have true multicore support, it mostly uses 1 CPU, for example my CPU load is 25% (on 4 cpu logical cores).

User avatar
burli
Member
Posts: 1643
Joined: Fri Apr 10, 2015 13:18

Re: Better support for multicore CPUs

by burli » Post

If you play in singleplayer 2 cores are used. One for server, one for client

User avatar
Lejo
Member
Posts: 715
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo

Re: Better support for multicore CPUs

by Lejo » Post

I checked the Threads of a minetestserver and get the number 6.
One for Map Save.
Two for Mapgen.
But what is the reset, is there more then one for lua mods?
My Problem is that one of my Cores is often at 100%, but the other at 20%.
Can I split one Thread into two?

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

Lejo wrote:I checked the Threads of a minetestserver and get the number 6.
One for Map Save.
Two for Mapgen.
But what is the reset, is there more then one for lua mods?
My Problem is that one of my Cores is often at 100%, but the other at 20%.
Can I split one Thread into two?
Yes, and, No.

Workload division is often not easy to do and balance, and it's often not actually faster.

All lua workloads run in the same (main) thread.

slopsbucket
Member
Posts: 199
Joined: Fri Oct 31, 2014 22:56
Location: Top End, Australia

Re: Better support for multicore CPUs

by slopsbucket » Post

Well, if the game get's more content, like more plants, the performance drops. If you are just walking and a few blocks ahead is nothing because the mapgen lacks, thats annoying.
Just my own observation here, through several different versions of Minetest over several generations of computer. For the most part I've had very little performance troubles running in Linux but I did notice a definite drop in performance whenever there's a large area of jungle weeds in front of me, turn around and face away from them and performance picks up again. Use worldedit to replace the weeds with air and there's no more performance drop.
I don't have high end hardware, but a quad core with 16GB of RAM and a Nvidia GTX750 should be enough to run MT acceptable
It sound like I currently have a similar computer to Burli:
3.2 Ghz Kaby Lake Core i5
16 Gb ram at 2133 Mhz

I only have the built in intel graphics but it seems pretty good considering what it is.

I have noticed a strange thing since I upgraded to Fedora 28 however. Every few minutes the game freezes for half a second, just long enough to make you put a block in the wrong place. Very annoying. I believe this must be a fedora problem however because it's the same version of minetest I was running before I upgraded and it wasn't doing it then.

Cheers,

Andrew.

User avatar
Lejo
Member
Posts: 715
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo

Re: Better support for multicore CPUs

by Lejo » Post

Is it possible to make to group of mods and each group runs in a extra thread??
As lua can not handle multiply threads to the same time, this would be a solution.
You can put mods for blocks in one and the mobs in the other thread.

User avatar
Joseph16
Member
Posts: 308
Joined: Tue Dec 06, 2016 05:35
In-game: Joseph16

Re: Better support for multicore CPUs

by Joseph16 » Post

slopsbucket wrote:
I have noticed a strange thing since I upgraded to Fedora 28 however. Every few minutes the game freezes for half a second, just long enough to make you put a block in the wrong place. Very annoying. I believe this must be a fedora problem however because it's the same version of minetest I was running before I upgraded and it wasn't doing it then.

Cheers,

Andrew.
Try deleting the config file. When I updated from win7 to win10 the game lagged terribly and deleting the file fixed it.
Testin' mines since 1989

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

Lejo wrote:Is it possible to make to group of mods and each group runs in a extra thread??
As lua can not handle multiply threads to the same time, this would be a solution.
You can put mods for blocks in one and the mobs in the other thread.
While technically something like this is possible, there is a big cost to this. Either you'd have to implement locking to prevent race issues, or you'd simply lose functionality and flexibility.

There are other solutions you can do that may help, though. For instance, you can use IPC to offload tasks to different processes if they're heavy computational, and use asynchronous methods to defer work that way.

This is a method I use on Inside The Box, for instance. The score/rank code is much too complex to run in Lua, it would stall the server for minutes, so I offload the task to a completely separate program that calculates the results and passes those results back to the server when it's updated. This does not require any modification to the server, and there are several ways of doing this (luasockets, HTTP).

User avatar
TalkLounge
Member
Posts: 315
Joined: Sun Mar 26, 2017 12:42
GitHub: TalkLounge
In-game: TalkLounge
Location: Germany

Re: Better support for multicore CPUs

by TalkLounge » Post

completely separate program that calculates the results and passes those results back to the server
That works, yes. But its not that effective as it would be done in Lua. The processor will run in both ways. But in the direct comparison I think Lua would have a much better performance, because you dont need to deliver it back to Lua.

I found something interesting: https://dev.minetest.net/Engine_structure#Threads
Subgames Server: Sky World Subgames German Survival Server: Wildes Land 2 E-Mail: talklounge@yahoo.de

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

TalkLounge wrote: That works, yes. But its not that effective as it would be done in Lua. The processor will run in both ways. But in the direct comparison I think Lua would have a much better performance, because you dont need to deliver it back to Lua.
Doing it in lua would be cumbersome, as I'd have to manipulate large data sets (millions of records) in Lua and it's not necessarily well adopted to do that. Other languages properly grow arrays and stacks optimistically which is far more efficient, and the language I am using (golang) to process the data is well adapted at handling the amount of data efficiently. You don't want Lua to go out of memory either, since that crashes the server context.

User avatar
TalkLounge
Member
Posts: 315
Joined: Sun Mar 26, 2017 12:42
GitHub: TalkLounge
In-game: TalkLounge
Location: Germany

Re: Better support for multicore CPUs

by TalkLounge » Post

manipulate large data sets (millions of records) in Lua
Yes, then its useful to use Java(I have programmed already in this language) etc.
My message was pointed at Lejos "make to group of mods and each group runs in a extra thread". That arent big tasks, but many little tasks. If every mod would run in an extra thread, you have a better performance. And you can check, which thread uses the most processor time. I know that there is a profiler, but it doesnt show me real values. The profiler show me, how long a function needs, until its finish. That arent real values. I made the experience, that I can´t look into the profiler and trust him. When I have a permanent lag from 1 but count together all values from the profiler, I am far away from one second. In theory, when I have maximal 99% processor load, there isnt a max_lag higher than 0.1, when every mod have its own thread. But there is. Currently I have a max_lag from 1.4 but on each of my two processors not more than 15% processor load(670 MB from 3 GB RAM is used). There is more capacity, but minetest doesnt use it. Minetest produces lag himself. And much servers have those lag problems.
Subgames Server: Sky World Subgames German Survival Server: Wildes Land 2 E-Mail: talklounge@yahoo.de

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

TalkLounge wrote:If every mod would run in an extra thread, you have a better performance.
Oversimplification, and, likely false.

Mods all manipulate the same basic environment. Therefore, if they run in threads, they'd be manipulating the same data and would have to be modified to do this in a safe way. This is done using locks, atomics, semaphores etc, and, those are not cheap. You'd make the overall performance cost of running Lua code much heavier, and you'd get little to no benefit out of it. In fact, it would likely be really hard to do this properly and you'd end up with a very unstable engine for a very long time. Plus, concurrency is really, really difficult to get right. Debugging locking issues is incredibly difficult.

So "just" putting mods in a thread isn't something any dev in the team even wants to do. Just write better lua code.

843jdc
Member
Posts: 352
Joined: Tue Jan 19, 2016 16:46
GitHub: jdc843
IRC: jdc843
In-game: 843jdc
Location: USA

Re: Better support for multicore CPUs

by 843jdc » Post

Minetest is slow.
Anyone that says otherwise is talking trash.

I'm reading a lot of forum messages discussing how things CAN"T be done, but relatively few on possible ways in which things CAN be done

I have a Intel i7-3820 cpu, 16 GB RAM, Nvidia GTX560 (now a GTX 660), HDD & a SSD. A now dated system but still plenty fast for running Windows 10 and Linux applications.

When I was running Cash's World, the big bottleneck was with the HDD. I switched to the SSD, installed Redis, and the bottleneck moved to the network. I upgraded my network speed and tinkered with Linux's networking code. Except for initial client connections which maxed out my sending speed, that moved the bottleneck to the CPU. The i7-3820 is quad-core. One core was working at 100% running the server.

Even logging in to the server on the same machine was laggy! I started tinkering with the LUA code drastically changing how the mods operate in an attempt to speed things up. (My changes were never completed. Admittedly a butcher job.) That didn't work to speed things up.

With my computer mostly idling except for the server process and occasional Redis writes to the map file, I can only conclude that the Minetest server code is slow as a bug in molasses.

Here's one thought:
1 - Have the server concentrate on what is in the world. Nothing else. It doesn't care how something works or reacts but only that something is there. A chest is only a chest. the server don't care what's in it. Et cetera
2 - Player connections get offloaded to a separate process to log players in and out. Network priority needs to be given to current players so they don't experience lag.
3 - Player movements, machine operations, tree growth, etc are handled by separate processes that inform the server of any needed changes to the map.
4 - Conflicts. Hmm, that's a toughie. I'll leave this for someone else to solve.
5 - Requests for multi-threaded and GPU coding changes go into the TODO file. Maybe someone will step up and work on something. Maybe not haha.

Baby steps. Start small. Have only a map server, one process to log players in and out, and one process to handle everything else. Then players can log in fast and not cause the server to stall while it sends map data and images to the new player. Later, that one big process that handles "everything else" can then be split up as needed.

I haven't played Minetest much in the past 2? years much less studied the server's code. So I don't know what the current state is. I just know reading posts from the developers complaining "We can't do ____ because of ____." is a total cop-out.

The developers have done a marvelous job writing this FREE game for anyone to enjoy. You have my thanks and gratitude for your hard and un-paid work. I agree to not make changes when someone asks for a fundamental change to how the game plays. But complaints about how the game currently runs should be worked on. Quit answering complaints with a complaint of your own.
Gripe Mode off. I'm outa here.

sofar
Developer
Posts: 2132
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: Better support for multicore CPUs

by sofar » Post

843jdc wrote:Minetest is slow.
Anyone that says otherwise is talking trash.

I'm reading a lot of forum messages discussing how things CAN"T be done, but relatively few on possible ways in which things CAN be done

I have a Intel i7-3820 cpu, 16 GB RAM, Nvidia GTX560 (now a GTX 660), HDD & a SSD. A now dated system but still plenty fast for running Windows 10 and Linux applications.

When I was running Cash's World, the big bottleneck was with the HDD. I switched to the SSD, installed Redis, and the bottleneck moved to the network. I upgraded my network speed and tinkered with Linux's networking code. Except for initial client connections which maxed out my sending speed, that moved the bottleneck to the CPU. The i7-3820 is quad-core. One core was working at 100% running the server.
I can't agree with this assessment. My server (Inside the Box) barely uses 3-4% CPU for the first player. I've never seen it above 40% (10 players or so). It runs on a KVM server, so not even real hardware. Only one core, too.

Whatever mods you put on the server are ruining it. The engine, with well written mods, can do much better. I'm not saying it couldn't be better, but I disagree with your assessment that because your server is slow that multicore processing will solve your problem.

because that won't fix bad mods. They will just have more cores to eat all the CPU power from. nom nom nom.

843jdc
Member
Posts: 352
Joined: Tue Jan 19, 2016 16:46
GitHub: jdc843
IRC: jdc843
In-game: 843jdc
Location: USA

Re: Better support for multicore CPUs

by 843jdc » Post

And the rebuttals proving me as talking trash begins haha :) I almost deleted my post before sending it. I decided to wait and see what it invoked.

I got a reply half typed out. I'm stuck on deciding something and am thinking in circles. Not good.
GIST: What one player does should not affect others and vice versa. Define 'quality gameplay' I'm stuck on that one. I think I'll sleep on it. GN :)

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests