Yadda ake Shigar da Sanya Apache Hadoop akan Node Guda a CentOS 7


Apache Hadoop shine tushen tsarin Buɗaɗɗen ginawa don rarraba Babban Ma'ajiyar Bayanai da sarrafa bayanai a cikin gungu na kwamfuta. Aikin ya dogara ne akan abubuwa masu zuwa:

  1. Hadoop Common – yana ƙunshe da ɗakunan karatu na Java da abubuwan amfani da wasu samfuran Hadoop ke buƙata.
  2. HDFS – Tsarin Fayil Rarraba Hadoop – Tsarin fayil mai ma'auni na tushen Java wanda aka rarraba a cikin nodes da yawa.
  3. Taswirar Rage - Tsarin YARN don sarrafa manyan bayanai a layi daya.
  4. Hadoop YARN: Tsari don sarrafa albarkatun tari.

Wannan labarin zai jagorance ku kan yadda zaku iya shigar da Apache Hadoop akan gungu guda ɗaya a cikin CentOS 7 (kuma yana aiki don nau'ikan RHEL 7 da Fedora 23+). Ana kuma nusar da wannan nau'in daidaitawar azaman Yanayin Rarraba-Rarraba Hadoop.

Mataki 1: Sanya Java akan CentOS 7

1. Kafin a ci gaba da shigarwar Java, fara shiga tare da tushen mai amfani ko mai amfani da tushen gata saitin sunan mai masaukin injin ku tare da umarni mai zuwa.

# hostnamectl set-hostname master

Hakanan, ƙara sabon rikodin a cikin fayil ɗin runduna tare da injin FQDN na ku don nuna adireshin IP na tsarin ku.

# vi /etc/hosts

Ƙara layin da ke ƙasa:

192.168.1.41 master.hadoop.lan

Sauya sunan mai masaukin da ke sama da bayanan FQDN tare da saitunan ku.

2. Na gaba, je zuwa shafin saukar da Oracle Java kuma ansu rubuce-rubucen sabuwar sigar Java SE Development Kit 8 akan tsarin ku tare da taimakon umarnin curl:

# curl -LO -H "Cookie: oraclelicense=accept-securebackup-cookie" “http://download.oracle.com/otn-pub/java/jdk/8u92-b14/jdk-8u92-linux-x64.rpm”

3. Bayan Java binary downloading ya gama, shigar da kunshin ta hanyar ba da umarnin da ke ƙasa:

# rpm -Uvh jdk-8u92-linux-x64.rpm

Mataki 2: Sanya Tsarin Hadoop a cikin CentOS 7

4. Na gaba, ƙirƙirar sabon asusun mai amfani akan tsarin ku ba tare da tushen ikon da za mu yi amfani da shi don hanyar shigarwa na Hadoop da yanayin aiki ba. Sabon kundin adireshi na gida zai zauna a cikin kundin adireshi /opt/hadoop.

# useradd -d /opt/hadoop hadoop
# passwd hadoop

5. A mataki na gaba ziyarci shafin Apache Hadoop don samun hanyar haɗin yanar gizon sabuwar barga da zazzage ma'ajin a kan tsarin ku.

# curl -O http://apache.javapipe.com/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz 

6. Cire rumbun adana kwafin abun cikin directory zuwa hanyar gida ta hadoop. Hakanan, tabbatar kun canza izinin kwafin fayilolin daidai.

#  tar xfz hadoop-2.7.2.tar.gz
# cp -rf hadoop-2.7.2/* /opt/hadoop/
# chown -R hadoop:hadoop /opt/hadoop/

7. Na gaba, shiga tare da mai amfani da hadoop kuma saita Hadoop da Java Environment Variables akan tsarin ku ta hanyar gyara fayil ɗin .bash_profile.

# su - hadoop
$ vi .bash_profile

Saka layin masu zuwa a ƙarshen fayil ɗin:

## JAVA env variables
export JAVA_HOME=/usr/java/default
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar

## HADOOP env variables
export HADOOP_HOME=/opt/hadoop
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

8. Yanzu, fara canza yanayin yanayi kuma bincika matsayinsu ta hanyar ba da umarni na ƙasa:

$ source .bash_profile
$ echo $HADOOP_HOME
$ echo $JAVA_HOME

9. A ƙarshe, saita ingantaccen maɓallin ssh don asusun hadoop ta hanyar aiwatar da umarnin da ke ƙasa (maye gurbin sunan mai masauki ko FQDN akan ssh-copy-id umarni daidai).

Hakanan, barin kalmar wucewar da ba komai don shiga ta ssh ta atomatik.

$ ssh-keygen -t rsa
$ ssh-copy-id master.hadoop.lan