Yadda ake Shigo da Sanya Manajan Cloudera akan CentOS/RHEL 7 - Sashe na 3


A cikin wannan labarin, mun bayyana matakan mataki zuwa mataki don girka Manajan Cloudera kamar yadda ayyukan masana'antu suke. A Sashe na 2, mun riga mun wuce cikin abubuwan da ake buƙata na Cloudera, tabbatar cewa an shirya dukkan sabobin daidai.

  • Mafi Kyawawan Ayyuka don Sanya Hadoop Server akan CentOS/RHEL 7 - Sashe na 1
  • Kafa Hadoop Abubuwan da ake buƙata da Hardarfafa Tsaro - Sashe na 2

Anan zamu sami tarin mahaɗan 5 inda masters 2 da ma'aikata 3. Na yi amfani da lokuta 5 AWS EC2 don nuna tsarin shigarwa. Na sanya sunayen waɗannan sabobin 5 kamar yadda ke ƙasa.

master1.linux-console.net
master2.linux-console.net
worker1.linux-console.net
worker2.linux-console.net
worker3.linux-console.net

Cloudera Manager kayan aiki ne na kulawa da kulawa ga duka CDH. Yawancin lokaci muna kulawa da shi kayan aikin gudanarwa don Cloudera Hadoop. Zamu iya turawa, saka idanu, sarrafawa, da yin canjin tsari tare da amfani da wannan kayan aikin. Wannan yana da mahimmanci sosai don sarrafa duka gungu.

Da ke ƙasa akwai mahimman amfani na Cloudera Manager.

  • loaddamar da saita gungu-gungurorin Hadoop ta hanyar atomatik.
  • Kula da lafiyar gungu
  • Sanya faɗakarwa
  • Shirya matsala
  • Rahoton
  • Yin Rahoton Yin Amfani da Tattaunawa kan Albarkatu da ƙarfi

Mataki 1: Shigar da Sabar Yanar Gizon Apache akan CentOS

Za mu yi amfani da master1 azaman mai amfani da yanar gizo don wuraren ajiyar Cloudera. Hakanan, Cloudera Manager WebUI ne, saboda haka muna buƙatar shigar da Apache. Bi matakan da ke ƙasa don shigar da sabar yanar gizo ta apache.

# yum -y install httpd

Da zarar an shigar da httpd, fara shi kuma kunna yadda za'a fara shi akan taya.

# systemctl start httpd
# systemctl enable httpd

Bayan fara httpd, tabbatar da halin.

# systemctl status httpd

Bayan fara httpd, buɗe burauzar a cikin tsarin yankinku kuma liƙa adireshin IP na master1 a cikin sandar bincike, yakamata ku sami wannan shafin gwajin don tabbatar da cewa httpd yana aiki sosai.

Mataki 2: Sanya DNS na Gida don warware IP da Sunan mai masauki

Muna buƙatar samun sabar DNS ko daidaitawa/sauransu/runduna don warware IP da sunan mai masauki. Anan muna daidaitawa/sauransu/runduna, amma a ainihin lokacin, sabar DNS ɗin da aka sadaukar zata kasance don yanayin samarwa.

Bi matakan da ke ƙasa don yin shigarwa don duk sabobin ku a/etc/runduna.

# vi /etc/hosts

Wannan ya kamata a saita shi a cikin duk sabobin.

13.235.27.144   master1.linux-console.net     master1
13.235.135.170  master2.linux-console.net     master2
15.206.167.94   worker1.linux-console.net     worker1
13.232.173.158  worker2.linux-console.net     worker2
65.0.182.222    worker3.linux-console.net     worker3

Mataki na 3: Sanya Shiga kalmar wucewa ta SSH

Ana saka Manajan Cloudera akan master1 a cikin wannan zanga-zangar. Muna buƙatar daidaita ssh-kalmar sirri daga master1 zuwa duk sauran nodes. Saboda Cloudera Manager zaiyi amfani da ssh don sadarwa duk sauran nodes don shigar da fakiti.

Bi matakan da ke ƙasa don daidaita ssh kalmar sirri daga master1 zuwa duk sauran sabobin. Za mu sami mai amfani 'tecmint' don ci gaba da gaba.

Createirƙiri mai amfani 'tecmint' duk sabobin 4 ta amfani da commandradd command kamar yadda aka nuna.

# useradd -m tecmint

Don ba da dama ga tushen mai amfani 'tecmint', ƙara layin da ke ƙasa cikin fayil ɗin/sauransu/sudoers. Kuna iya ƙara wannan layin a ƙarƙashin tushen kamar yadda aka bayar a cikin sikirin.

tecmint   ALL=(ALL)    ALL

Canja zuwa mai amfani 'tecmint' kuma ƙirƙiri mabuɗin ssh a cikin master1 ta amfani da umarnin da ke ƙasa.

# sudo su tecmint
$ ssh-keygen

Yanzu kwafe maɓallin da aka kirkira zuwa duk sabobin 4 ta amfani da ssh-copy-id umarnin kamar yadda aka nuna.

$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email  
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 

Yanzu ya kamata ku sami damar ssh daga master1 zuwa sauran duk sabobin ba tare da kalmar wucewa ba kamar yadda aka nuna.

$ ssh master2
$ ssh worker1
$ ssh worker2
$ ssh worker3

Mataki na 4: Girkawa da saita Mai sarrafa Cloudera

Zamu iya amfani da ma'ajiyar mai siyarwa (Cloudera) don girka dukkan fakitin ta amfani da kayan aikin sarrafa kunshin a cikin RHEL/CentOS. A cikin ainihin lokacin, ƙirƙirar wurin ajiyarmu shine mafi kyawun aiki saboda ƙila baza mu sami damar intanet a cikin sabobin samarwa ba.

Anan zamu girka sakin Cloudera Manager 6.3.1 saki. Tunda za mu yi amfani da master1 azaman sabar repo, muna sauke fakitin a cikin hanyar da aka ambata a ƙasa.

Createirƙiri kundin adireshin da ke ƙasa a kan sabar master1 .

$ sudo mkdir -p /var/www/html/cloudera-repos/cm6

Zamu iya amfani da kayan aikin wget don zazzage fakiti akan http. Don haka, girka wget ta amfani da umarnin da ke kasa.

$ sudo yum -y install wget

Na gaba, zazzage fayil ɗin tarho na Cloudera Manager ta amfani da umarnin wget mai zuwa.

$ wget https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz

Cire fayil ɗin tar a cikin/var/www/html/Cloudera-repos/cm6, tuni mun yi master1 a matsayin webserver ta shigar da http kuma mun gwada a kan burauzar.

$ sudo tar xvfz cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/cm6 --strip-components=1

Yanzu, tabbatar cewa duk fayilolin Cloudera rpm suna nan cikin/var/www/html/cloudera-repos/cm6/RPMS/x86_64 directory.

$ cd /var/www/html/cloudera-repos/cm6
$ ll

Createirƙiri fayilolin /etc/yum.repos.d/cloudera-manager.repo a kan dukkan sabobin a cikin rundunonin masu tarin abubuwa tare da abubuwan da ke tafe, a nan master1 (65.0.101.148) shine sabar Gidan yanar gizo.

[cloudera-repo]
name=cloudera-manager
baseurl=http:///cloudera-repos/cm6/
enabled=1
gpgcheck=0

Yanzu an ƙara wurin ajiyar, gudanar da umarnin da ke ƙasa don duba wuraren da aka kunna.

$ yum repolist

Gudun umarnin da ke ƙasa don duba duk samfuran da ke da alaƙa da Cloudera a cikin maɓallin.

$ yum list available | grep cloudera*

Shigar da Cloudera-manager-server, cloudera-manager-agent, Cloud-manager-daemons Cloudera-manager-server-db-2.

$ sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server cloudera-manager-server-db-2

Gudanar da umarnin da ke ƙasa don duba duk fakitin Cloudera da aka sanya.

$ yum list installed | grep cloudera*

Gudanar da umarnin da ke ƙasa don fara girgije-scm-server-db wanda shine tushen bayanan don adana Cloudera Manager da sauran ayyukan metadata.

Ta hanyar tsoho, Cloudera tana zuwa tare da postgre-sql wanda aka saka a cikin Cloudera Manager. Muna shigar da wanda aka saka, a cikin ainihin lokacin bayanan waje wanda za'a iya amfani dashi. Zai iya zama Oracle, MySQL, ko PostgreSQL.

$ sudo systemctl start cloudera-scm-server-db

Gudun umarnin da ke ƙasa don bincika matsayin bayanan bayanan.

$ sudo systemctl status cloudera-scm-server-db

Sanya db.properties don sabar Cloudera Manager.

$ vi /etc/cloudera-scm-server/db.properties

Sanya ƙimar da ke ƙasa ita ce EMBEDDED don yin Manajan Cloudera yayi amfani da Dakin Bayanai.

com.cloudera.cmf.db.setupType=EMBEDDED

Gudun umarnin da ke ƙasa don fara sabar Cloudera Manager.

$ sudo systemctl start cloudera-scm-server

Gudun umarnin da ke ƙasa don bincika matsayin sabar Cloudera Manager.

$ sudo systemctl status cloudera-scm-server

Gudanar da umarnin da ke ƙasa don farawa da bincika matsayin wakilin Cloudera Manager.

$ sudo systemctl start cloudera-scm-agent
$ sudo systemctl status cloudera-scm-agent

Da zarar Cloud Manager Manager ya ci nasara kuma yake gudana, zaka iya duba WebUI (Shafin Shiga) a cikin mai binciken ta amfani da adireshin IP da kuma tashar tashar jiragen ruwa mai lamba 7180 wanda shine lambar tashar tashar Cloudera Manager.

https://65.0.101.148:7180

A cikin wannan labarin, mun ga tsari zuwa mataki don girka Cloudera Manager akan CentOS 7. Za mu ga CDH da sauran kayan aikin sabis a cikin labarin na gaba.