A good idea to improve the sub game loading time.
A good idea to improve the sub game loading time.
Now an MT client must get lots of subgame images files from the server or a HTTP server, and load them one by one(lots of file open operation). This will cost a lot of time.
Here is my idea:
Server: Merge all the images to some big images. When you finish your subgame, run a script to merge all the images and put them in a directory.
Client: Only get those merge big images and each image's coordinate info.
Why this can save the loading time a lot?
1. Transfer time. Only transfer the merged big images, instead of hundreds or thousands of small images through the internet.
2. Loading files time. Only open the big merge files, save a lot of the files open time.
Leave your comment :)
Here is my idea:
Server: Merge all the images to some big images. When you finish your subgame, run a script to merge all the images and put them in a directory.
Client: Only get those merge big images and each image's coordinate info.
Why this can save the loading time a lot?
1. Transfer time. Only transfer the merged big images, instead of hundreds or thousands of small images through the internet.
2. Loading files time. Only open the big merge files, save a lot of the files open time.
Leave your comment :)
- Linuxdirk
- Member
- Posts: 3217
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: A good idea to improve the sub game loading time.
Yes, all media should be archived and send as one compressed single file/stream instead of loading every media individually.
- Krock
- Developer
- Posts: 4649
- Joined: Thu Oct 03, 2013 07:48
- GitHub: SmallJoker
- Location: Switzerland
- Contact:
Re: A good idea to improve the sub game loading time.
So we need some stats about how long a file opening operation takes (Windows with path hooks, and Linux). An option would be to save them into an sqlite3 database, but there you'll have to deal with the row lookup time.
I proposed the same idea a few year ago, but can't find the log entry. However, sfan5's point was (roughly): "Nothing beats the filesystem. How should a database be faster than that?".
I proposed the same idea a few year ago, but can't find the log entry. However, sfan5's point was (roughly): "Nothing beats the filesystem. How should a database be faster than that?".
Look, I programmed a bug for you. >> Mod Search Engine << - Mods by Krock - DuckDuckGo mod search bang: !mtmod <keyword here>
Re: A good idea to improve the sub game loading time.
Maybe one simple way is just zip the subgame assets and transfer one zip file to the client.
Linuxdirk wrote:Yes, all media should be archived and send as one compressed single file/stream instead of loading every media individually.
Re: A good idea to improve the sub game loading time.
Save the internet transfer time will save a lot, I think. Zip the subgame assets and send one zip file to the client :>.
The assets loading time in the client side may not save a lot(all files in a database VS all files in the filesystem), because we just load a few hundreds or thousands of small file. That’s a small job for a model CPU, I think. [Need some stats]
The assets loading time in the client side may not save a lot(all files in a database VS all files in the filesystem), because we just load a few hundreds or thousands of small file. That’s a small job for a model CPU, I think. [Need some stats]
Krock wrote:So we need some stats about how long a file opening operation takes (Windows with path hooks, and Linux). An option would be to save them into an sqlite3 database, but there you'll have to deal with the row lookup time.
I proposed the same idea a few year ago, but can't find the log entry. However, sfan5's point was (roughly): "Nothing beats the filesystem. How should a database be faster than that?".
- Linuxdirk
- Member
- Posts: 3217
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: A good idea to improve the sub game loading time.
Why not archiving all media on server startup and send the archive to the client when it connects? This would save individual files being read and sent every time a client connects. I assume due to the overhead of reading the files and establishing a connection for each of the files sending individual files is slower than sending only one file.Krock wrote:So we need some stats about how long a file opening operation takes (Windows with path hooks, and Linux). An option would be to save them into an sqlite3 database, but there you'll have to deal with the row lookup time.
- Pyrollo
- Developer
- Posts: 385
- Joined: Mon Jan 08, 2018 15:14
- GitHub: pyrollo
- In-game: Naj
- Location: Paris
Re: A good idea to improve the sub game loading time.
Media consisting in images and sounds (PNG, OGG) are already compressed. No need to compress then once again. Archiving should be enough.Linuxdirk wrote:Yes, all media should be archived and send as one compressed single file/stream instead of loading every media individually.
[ Display Modpack ] - [ Digiterms ] - [ Crater MG ] - [ LATE ]
- Linuxdirk
- Member
- Posts: 3217
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: A good idea to improve the sub game loading time.
By the server? I've seen dozen of mods where neither the PNGs nor the OGGs were optimized.Pyrollo wrote:Media consisting in images and sounds (PNG, OGG) are already compressed.
- Pyrollo
- Developer
- Posts: 385
- Joined: Mon Jan 08, 2018 15:14
- GitHub: pyrollo
- In-game: Naj
- Location: Paris
Re: A good idea to improve the sub game loading time.
That's another problem. In my opinion re-compressing (even badly) compressed data does not make sense. I'm not even sure of the gain.Linuxdirk wrote:By the server? I've seen dozen of mods where neither the PNGs nor the OGGs were optimized.
[ Display Modpack ] - [ Digiterms ] - [ Crater MG ] - [ LATE ]
- Linuxdirk
- Member
- Posts: 3217
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: A good idea to improve the sub game loading time.
Yeah, then leave it uncompressed. Would still be better than reading and sending all media one-by-one on every client connection.
Re: A good idea to improve the sub game loading time.
Will this be support in the future? or some other way to improve the first time loading?
- Festus1965
- Member
- Posts: 4181
- Joined: Sun Jan 03, 2016 11:58
- GitHub: Festus1965
- In-game: Festus1965 Thomas Thailand Explorer
- Location: Thailand ChiangMai
- Contact:
Re: A good idea to improve the sub game loading time.
Make sure that the client itself is able to check weather this mod is local available, and load then from there local.
So gamer who want to force speed and save time, just download that mods themself into client and ... happy ?
So gamer who want to force speed and save time, just download that mods themself into client and ... happy ?
Human has no future (climate change)
If urgend, you find me in Roblox (as CNXThomas)
If urgend, you find me in Roblox (as CNXThomas)
Re: A good idea to improve the sub game loading time.
It looks like MT already manages a cache. I guess the client receives some manifest, checks the assets it already has in its cache, and then emits requests for the missing assets.
Is there any way to profile this part of the game? How much time is spent on connecting, sending/downloading, and processing the payload?
Is there any way to profile this part of the game? How much time is spent on connecting, sending/downloading, and processing the payload?
Who is online
Users browsing this forum: No registered users and 3 guests