zstd - Algorithm Mai Saurin Cututtukan Bayanai da Facebook ke Amfani da shi


Zstandard (kuma aka sani da zstd) tushen buɗe ido ne na kyauta, shirin matsar bayanai na ainihin lokaci tare da mafi kyawun ma'aunin matsi, wanda Facebook ya haɓaka. Algorithm din matsawa mara asara ce da aka rubuta a cikin C (akwai sake aiwatarwa a Java) - don haka shirin Linux ne na asali.

Lokacin da ake buƙata, yana iya cinikin saurin matsawa don ƙimar matsawa mai ƙarfi (gudun matsawa vs matsawa rabo cinikin-kashe ana iya daidaita shi ta hanyar ƙananan haɓaka), akasin haka. Yana da yanayi na musamman don ƙananan matse bayanai, wanda aka sani da matsawa ƙamus, kuma yana iya gina ƙamus daga kowane saitin samfurin da aka bayar. Ya zo tare da mai amfani da layin umarni don ƙirƙira da yanke fayilolin .zst, .gz, .xz da .lz4.

Mahimmanci, Zstandard yana da tarin tarin APIs, yana goyan bayan kusan duk shahararrun yarukan shirye-shirye da suka haɗa da Python, Java, JavaScript, Nodejs, Perl, Ruby, C#, Go, Rust, PHP, Switft, da ƙari mai yawa.

Ana amfani da shi sosai don damfara manyan kundin bayanai a cikin tsari da yawa da amfani da lokuta a cikin Facebook; ayyuka kamar Amazon Redshift data warehousing; bayanan bayanai kamar Hadoop da Redis; cibiyar sadarwar Tor da sauran aikace-aikace da yawa ciki har da wasanni.

Ana samun sakamako masu zuwa ta hanyar yin gwaje-gwajen algorithms masu sauri da yawa akan sabar da ke aiki da Linux Debian ta amfani da lzbench, kayan aikin ma'auni mai buɗewa a cikin ƙwaƙwalwar ajiya.

Yadda ake Sanya Zstandard Compression Tool a Linux

Don shigar da Zstandard akan rarrabawar Linux, kuna buƙatar tattara shi daga tushe, amma kafin wannan farkon kuna buƙatar shigar da mahimman kayan aikin haɓakawa akan tsarin ku ta amfani da mai sarrafa fakitin rarraba ku kamar yadda aka nuna.

$ sudo apt update && sudo apt install build-essential		#Ubuntu/Debian
# yum group install "Development Tools" 			#CentOS/REHL
# dnf groupinstall "C Development Tools and Libraries"		#Fedora 22+

Da zarar an shigar da duk kayan aikin haɓaka da ake buƙata, yanzu zaku iya zazzage fakitin tushen, matsa zuwa cikin kundin adireshin gida, gina binary kuma shigar da shi kamar yadda aka nuna.

$ cd ~/Downloads
$ git clone https://github.com/facebook/zstd.git
$ cd zstd
$ make
$ sudo make install 

Da zarar an shigar da Zstandard, yanzu za mu iya ci gaba don koyon wasu ainihin amfani da misalan umarnin Zstd a cikin sashe mai zuwa.

Koyi Misalin Amfani da Umurni na 10 Zstd a cikin Linux

Tsarin layin umarni na Zstd gabaɗaya yayi kama da na gzip da kayan aikin xz, tare da ƴan bambance-bambance.

1. Don ƙirƙirar fayil ɗin matsawa .zst, kawai samar da sunan fayil don damfara shi ko amfani da alamar -z kuma yana nufin damfara, wanda shine aikin da aka saba.

$ zstd etcher-1.3.1-x86_64.AppImage 
OR
$ zstd -z etcher-1.3.1-x86_64.AppImage 

2. Don sauke fayil ɗin matsawa .zst, yi amfani da alamar -d ko uzstd mai amfani kamar yadda aka nuna.

$ zstd -d etcher-1.3.1-x86_64.AppImage.zst 
OR
$ unzstd etcher-1.3.1-x86_64.AppImage.zst 

3. Don cire tushen fayil ɗin bayan an yi aiki, ta tsohuwa, ba a goge tushen fayil ɗin ba bayan an yi nasarar matsawa ko ragewa, don share shi, yi amfani da zaɓin --rm.

$ ls etcher-1.3.1-x86_64.AppImage
$ zstd --rm  etcher-1.3.1-x86_64.AppImage
$ ls etcher-1.3.1-x86_64.AppImage

4. Don saita matakin matsawa, zstd yana da adadin masu gyara aiki, misali zaku iya saka matakin matsawa azaman -6(lamba 1-19, tsoho shine 3) kamar yadda aka nuna.

$ zstd -6 --rm etcher-1.3.1-x86_64.AppImage

5. Don saita saurin matsawa, zstd yana da ma'aunin saurin matsawa 1-10, saurin matsawa tsoho shine 1. Kuna iya cinikin matsi don saurin matsawa tare da zaɓi --sauri, mafi girma lamba da sauri da matsawa gudun.

$ zstd --fast=10 etcher-1.3.1-x86_64.AppImage

6. Don nuna bayanai game da fayil ɗin da aka matse, yi amfani da tutar -l, wanda ake amfani da shi don nuna bayanai game da fayil ɗin da aka matsa, alal misali.

$ zstd -l etcher-1.3.1-x86_64.AppImage.zst

7. Don gwada amincin fayilolin da aka matsa, yi amfani da tutar -t kamar yadda aka nuna.

$ zstd -t etcher-1.3.1-x86_64.AppImage.zst

8. Don kunna yanayin magana, yi amfani da zaɓin -v.

$ zstd -v -5 etcher-1.3.1-x86_64.AppImage

9. Don amfani da wasu nau'ikan matsi ko narkar da fayil kamar gzip, xz, lzma, da lz4, ta amfani da -format=FORMAT kamar yadda aka nuna.

$ zstd -v --format=gzip etcher-1.3.1-x86_64.AppImage
$ zstd -v --format=xz  etcher-1.3.1-x86_64.AppImage

10. Don saita fifikon tsari na zstd zuwa ainihin-lokaci, zaku iya amfani da zaɓi -priority=rt kamar yadda aka nuna.

$zstd --priority=rt etcher-1.3.1-x86_64.AppImage

Tutar -r tana ba zstd umarnin yin aiki akai-akai akan ƙamus. Kuna iya samun zaɓuɓɓuka masu amfani da yawa da ci gaba, yadda ake karantawa ko ƙirƙirar ƙamus ta hanyar tuntuɓar shafin zstd man.

$ man zstd

Wurin ajiya na Github na Zstandard: https://github.com/facebook/zstd

Zstandard mai sauri ne na ainihin-lokaci, algorithm na matsa bayanai mara asara da kayan aikin matsawa wanda ke ba da ma'aunin matsi. Gwada shi kuma raba ra'ayoyinku game da shi ko yin tambayoyi ta hanyar amsawar da ke ƙasa.