Yadda ake Shigar da Saita Apache Spark akan Ubuntu/Debian

Apache Spark sigar buɗe-tushen rarraba lissafi ce wacce aka kirkira don samar da sakamakon lissafi cikin sauri. Injin ƙididdiga ne mai ƙwaƙwalwar ajiya, ma'ana za a sarrafa bayanan a ƙwaƙwalwar.

Spark yana tallafawa API daban-daban don gudanawa, sarrafa hoto, SQL, MLLib. Hakanan yana tallafawa Java, Python, Scala, da R a matsayin yarukan da aka fi so. Spark galibi an girka shi a cikin gunguyun Hadoop amma kuma zaka iya shigarwa da saita walƙiya a cikin yanayin keɓewa.

A cikin wannan labarin, zamu ga yadda ake girka Apache Spark a cikin rarrabawar Debian da Ubuntu.

Shigar Java da Scala a cikin Ubuntu

Don shigar da Apache Spark a cikin Ubuntu, kuna buƙatar saka Java da Scala akan na'urar ku. Yawancin rarrabawar zamani suna zuwa tare da Java wanda aka girka ta tsoho kuma zaku iya tabbatar dashi ta amfani da umarni mai zuwa.

$ java -version

Idan babu fitarwa, zaku iya girka Java ta amfani da labarinmu akan yadda ake girka Java akan Ubuntu ko kuma kawai aiwatar da waɗannan umarnin don girka Java akan rabarwar Ubuntu da Debian.

$ sudo apt update
$ sudo apt install default-jre
$ java -version

Na gaba, zaku iya girka Scala daga ma'ajiyar ajiya ta hanyar kunna waɗannan umarni don bincika sikeli kuma girka shi.

$ sudo apt search scala  ⇒ Search for the package
$ sudo apt install scala ⇒ Install the package

Don tabbatar da kafuwa na Scala, gudanar da wannan umarni.

$ scala -version 

Scala code runner version 2.11.12 -- Copyright 2002-2017, LAMP/EPFL

Shigar da Apache Spark a Ubuntu

Yanzu je zuwa umarnin wget na hukuma don zazzage fayil ɗin kai tsaye a cikin tashar.

$ wget https://apachemirror.wuchna.com/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz

Yanzu buɗe tashar ka ka canza zuwa inda aka sanya fayil din da ka sauke sannan ka gudanar da wannan umarni don cire fayil ɗin tarho na Apache Spark.

$ tar -xvzf spark-3.1.1-bin-hadoop2.7.tgz

Aƙarshe, matsar da fitaccen Spark directory zuwa/opt directory.

$ sudo mv spark-3.1.1-bin-hadoop2.7 /opt/spark

Sanya Sauyin Muhalli don Walƙiya

Yanzu dole ne ka saita varian canjin canjin muhalli a cikin fayil ɗin ka .profile kafin fara walƙiya.

$ echo "export SPARK_HOME=/opt/spark" >> ~/.profile
$ echo "export PATH=$PATH:/opt/spark/bin:/opt/spark/sbin" >> ~/.profile
$ echo "export PYSPARK_PYTHON=/usr/bin/python3" >> ~/.profile

Don tabbatar da cewa waɗannan sabbin abubuwan canjin yanayin ana iya samun su a cikin kwasfa kuma akwai su ga Apache Spark, shima ya zama tilas a gudanar da wannan umarni don aiwatar da canje-canje na kwanan nan.

$ source ~/.profile

Duk binar da ke da alaƙa da walƙiya don farawa da dakatar da ayyukan suna ƙarƙashin babban fayil ɗin sbin.

$ ls -l /opt/spark

Fara Spark Spark a Ubuntu

Gudanar da umarni mai zuwa don fara aikin Spark da sabis na bawa.

$ start-master.sh
$ start-workers.sh spark://localhost:7077

Da zarar an fara sabis ɗin zuwa burauzar kuma a buga wannan shafi na samun damar URL. Daga shafin, zaka iya ganin an fara bawa da bawa na.

http://localhost:8080/
OR
http://127.0.0.1:8080

Hakanan zaka iya bincika idan walƙiya-harsashi yana aiki da kyau ta hanyar ƙaddamar da umarnin walƙiya.

$ spark-shell

Shi ke nan ga wannan labarin. Za mu kama ku tare da wani labarin mai ban sha'awa nan ba da daɗewa ba.