Steam is working on adding zstd compression for game chunks (every game file is split into 1MB), it currently uses LZMA.
Wonder how much of an overall improvement it will be.
“Better” doesn’t always mean “smaller”, especially in this example. LZMA’s strength is that it compresses very small but its weakness is that it’s extremely CPU-intensive to decompress. Switching to ZSTD will actually result in larger downloads, but the massively reduced CPU load of decompressing ZSTD will mean it’s faster for most users. Instead of just counting the time it takes for the data to transfer, this is factoring in download time + decompression time. Even though ZSTD is somewhat less efficient in terms of compression ratio, it’s far more efficient computationally.
Bet that’ll save Valve on some server costs too. Storage is much cheaper than compute (though I imagine they’ll probably keep LZMA around for clients on slow connections).
Hmm true. I was thinking that steam has a lot of games and respective builds it has to compress, even if the decompression benefits are clientside only.
Each new game update would also be compressed too - I have no idea how Steam handles the update to work out what files need replacing on their end though, which might involve decompressing the files to analyse them.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]
No game suggestions, friend requests, surveys, or begging.
No Let’s Plays, streams, highlight reels/montages, random videos or shorts.
No off-topic posts/comments, within reason.
Use the original source, no clickbait titles, no duplicates.
(Submissions should be from the original source if possible, unless from paywalled or non-english sources.
If the title is clickbait or lacks context you may lightly edit the title.)
“Better” doesn’t always mean “smaller”, especially in this example. LZMA’s strength is that it compresses very small but its weakness is that it’s extremely CPU-intensive to decompress. Switching to ZSTD will actually result in larger downloads, but the massively reduced CPU load of decompressing ZSTD will mean it’s faster for most users. Instead of just counting the time it takes for the data to transfer, this is factoring in download time + decompression time. Even though ZSTD is somewhat less efficient in terms of compression ratio, it’s far more efficient computationally.
Bet that’ll save Valve on some server costs too. Storage is much cheaper than compute (though I imagine they’ll probably keep LZMA around for clients on slow connections).
Doesn’t decompression only happen client-side? I don’t imagine them compressing the files multiple times.
Hmm true. I was thinking that steam has a lot of games and respective builds it has to compress, even if the decompression benefits are clientside only.
Each new game update would also be compressed too - I have no idea how Steam handles the update to work out what files need replacing on their end though, which might involve decompressing the files to analyse them.
“most users” have gigabit pipes to the internet?