Today I discovered that there is also an excellent compression format, #Zstandard (ZSTD), is fast with an excellent compression ratio, developed by #Meta, and released as #opensource
I needed to backup my files because I have to wipe my PC and reinstall #Linux. Now I have a dilemma: choosing between #antiX and #Lubuntu. I have a fairly decent computer, so I could even install a more full-featured OS, but I prefer an OS that doesn’t use too many resources.
#zstd
https://www.youtube.com/watch?v=k5XsiuxHv_A

#LibreOffice Begins Landing #Zstd Integration

https://www.phoronix.com/news/LibreOffice-Landing-Zstd

Whoops, since when can zstdcat decompress #zstd, #gzip AND #xz files? Very useful!

Ah, apparently it can be compiled in. On #Manjaro it's linked against gzip, xz and lz4, on #NixOS (by default?) it can only decompress zstd.

Brand new PEP by @emmatyping to add Zstandard to the standard library:
https://peps.python.org/pep-0784/

Will it make it in to 3.14 before the feature freeze on 2025-05-06? It'll be close but it's possible!

The PEP also suggests namespacing the other compression libraries lzma, bz2 and zlib, with a 10-year deprecation for the old names.

Join the discussion to give your support, suggestions or feedback:

https://discuss.python.org/t/pep-784-adding-zstandard-to-the-standard-library/87377

I've just gained 30-ish GiB by compressing / & /home

btrfs filesystem defragment -r -v -czstd /home/
compress=zstd mount option in fstab

I have all of Germany in "uncompressed" #OpenStreetMap OBFs (+ the previous version in .old files).

I'm getting a compression ratio of 1.246, using the default zstd:3 level.

#Zstandard 1.5.7 has been released (#zstd / #RFC8878 / #RFC8478 / #Facebook / #Meta / #CompressionAlgorithm / #FileArchiver / #DataCompression) https://facebook.github.io/zstd/

Does anyone know of a tool which will inspect a ZStandard dictionary file? #zstd

#zstd 1.5.7 is out and it's honestly quite amazing.

Highlights for me are:

- ~10% faster at small block sizes common in databases(=filesystems?)
- Limited multi-threading by default. (You could already enable it manually; this only concerns the defaults.)
- A new --max flag that inches zstd closer to #lzma/#xz. We'll need to see more testing on how close exactly but it's impressive they managed to broaden the range this far in a single format.

https://github.com/facebook/zstd/releases/tag/v1.5.7

#Zstandard (aka #zstd) v1.5.7 is out:

https://github.com/facebook/zstd/releases/tag/v1.5.7

'"[…] a significant release […] brings enhancements across various domains, including performance, stability, and functionality […]

The compression speed for small data blocks has been notably improved at fast compression levels[…]

The --patch-from functionality of the zstd CLI […] v1.5.7 largely mitigates the speed impact of high compression levels 18+,

The compression ratio has been enhanced slightly for large data across all compression levels […]"'

Tein vähän #pakkaus-kokeiluja törmättyäni taannoin uuteen pakkaimeen, #Bzip3:een. Ainakin minun tiedostojeni #varmuuskopiointi-pakkaamisessa se hävisi pakkausteholtaan selvästi #XZ:lle, jota olen varmuuskopiointiin käyttänyt, ja pakkausnopeudeltaan selvästi #ZStd:lle, johon siirtymistä olin aprikoinut. #atkjuttuja

Rilasciato #PeaZip 10.2: Correzioni e Miglioramenti

#UnoLinux #linux #compressionedati #7zip #rar #zstd #zip #7z #brotli #zipx

https://www.laseroffice.it/blog/2025/01/02/rilasciato-peazip-10-2-correzioni-e-miglioramenti/

I love playing around with #compression

In this case, it's all text-based data in csv and xml formats.

Size:

32,696,320 202411.tar
 4,384,020 202411.tar.bz2
 4,015,912 202411.tar.zst
 3,878,583 202411.tar.bz3
 3,730,416 202411.tar.xz

zstd was invoked using zstd --ultra -22
xz was invoked using xz -9e
bzip2 was invoked using bzip2 -9
bzip3 has no compression level options

Speed:

zstd    54.31user 0.25system 0:54.60elapsed 99%CPU
xz      53.80user 0.06system 0:53.93elapsed 99%CPU
bzip2    5.33user 0.01system 0:05.35elapsed 99%CPU
bzip3    3.98user 0.02system 0:04.01elapsed 99%CPU

Maximum memory usage (RSS):

zstd    706,312
xz      300,480
bzip3    75,996
bzip2     7,680

*RSS sampled up to ten times per second during execution of the commands in question

#bzip3 is freaking amazing, yo.

#DataCompression #bzip #bz3 #zstd #zst #zstandard #xz #lzma
#CouldaBeenABlost ;)

#komprimace #gnu_linux
#Zstd je neskutečné dobrý #opensource komprimační algoritmus, měl by se víc propagovat (a používat).
Složky s webovým obsahem (HTML, CSS, obrázky atd.), celkem 220 Mb dat.
Jednovláknová komprimace do .tar.gz za 8 sekund a výsledek: 147 Mb.
Vícevláknová komprimace do .tar.gz za 1 sekundu (a taky 147 Mb).
Vícevláknová komprimace do .tar.zst za 1 sekundu a výsledek: 15 Mb (!!!)
GNU #gzip víc než 1 CPU nepodporuje, ale #pigz ano, který jsem použil pro porovnání rychlosti.

#CVE: 7-zip 任意代码执行漏洞，请更新至 24.07 或更新版本。

7zip 的 zstandard 解压组件存在漏洞，或导致任意代码执行。

CVE: CVE-2024-11477

zerodayinitiative.com/~

linksrc: https://t.me/bupt_moe/2305

#7zip #zstd

Telegram 原文

#RStats hottake

I'm pretty tired of the slow, inefficient compression utilities available by default.

gzip is fast but not great compression
xz is great compression but an absolute dog in terms of speed
bzip2 is somewhere inbetween the extremes.

Why not include #Zstd ?

Lol, I forgot how much better #Brotli is than gzip & bz2… I tried to bzip a 80GB file and it was taking forever. I'll need to do a benchmark, but on a small file it's:

gz: down to 32%, 6s
bz2: 17%, 2:20
brotli: 2% (!!), 2:40
brotli -6: 3%, 10s
(unbrotli was also like <10s)

How does #zstd compare? (I haven't tried it yet)