Shigar kuma Sanya Apache Oozie Jadawalin Ayyukan Aiki don CDH 4.X akan RHEL/CentOS 6/5


Oozie shine mai tsara tushen tushen tushe don Hadoop, yana sauƙaƙe tafiyar aiki da daidaitawa tsakanin ayyuka. Za mu iya ayyana dogaro tsakanin ayyuka don bayanan shigarwa don haka za mu iya sarrafa dogaron aiki ta amfani da mai tsara jadawalin ooze.

A cikin wannan koyawa, na shigar da Oozie a kan kullin maigidana (watau master a matsayin sunan mai masauki da kuma inda aka sanya namenode/JT) duk da haka a cikin tsarin samarwa ya kamata a shigar da oozie akan Hadoop daban. kumburi.

Umurnin shigarwa sun kasu kashi biyu, muna kiran shi A da B.

  1. A. Shigar Oozie.
  2. B. Kanfigareshan Oozie.

Bari mu fara tabbatar da sunan mai watsa shiri, ta amfani da bin umarnin 'hostname'.

 hostname

master

Hanyar A: Shigar Oozie akan RHEL/CentOS 6/5

Muna amfani da ma'ajin CDH na hukuma daga rukunin yanar gizon girgije don shigar da CDH4. Je zuwa sashin saukar da CDH na hukuma kuma zazzage sigar CDH4 (watau 4.6) ko kuma kuna iya amfani da bin umarnin wget don saukar da ma'ajiyar ku shigar da shi.

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/6/i386/cloudera-cdh-4-0.i386.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.i386.rpm

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.x86_64.rpm
# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/5/i386/cloudera-cdh-4-0.i386.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.i386.rpm

# wget http://archive.cloudera.com/cdh4/one-click-install/redhat/5/x86_64/cloudera-cdh-4-0.x86_64.rpm
# yum --nogpgcheck localinstall cloudera-cdh-4-0.x86_64.rpm

Da zarar, kun ƙara ma'ajiyar CDH a ƙarƙashin tsarin ku, zaku iya amfani da umarni mai zuwa don shigar da Oozie akan tsarin.

 yum install oozie

Yanzu, shigar da abokin ciniki oozie (a sama da umarnin yakamata ya rufe sashin shigarwar abokin ciniki duk da haka idan ba haka ba to gwada umarnin ƙasa).

 yum install oozie-client

Lura: Shigarwa na sama kuma yana saita sabis na oozie don aiki a farawa tsarin. Kyakkyawan aiki! Mun gama da kashi na farko na shigarwa yanzu bari mu matsa zuwa kashi na biyu don saita oozie.

Hanyar B: Kanfigareshan Oozie akan RHEL/CentOS 6/5

Kamar yadda oozie baya hulɗa kai tsaye tare da Hadoop, ba ma buƙatar kowane tsari na taswira anan.

Tsanaki: Da fatan za a saita duk saitunan yayin da oozie baya gudana, wannan yana nufin dole ne ku bi matakan ƙasa yayin da sabis na oozie baya gudana.

Oozie yana da 'Derby' azaman tsoho da aka gina a cikin DB duk da haka, zan ba da shawarar ku yi amfani da Mysql DB. Don haka, bari mu shigar da bayanan MySQL ta amfani da labarin mai zuwa.

  1. Saka MySQL Database a cikin RHEL/CentOS 6/5

Da zarar kun gama tare da sashin shigarwa, matsa gaba gaba don ƙirƙirar oozie DB da ba da gata kamar yadda aka nuna a ƙasa.

 mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.38 MySQL Community Server (GPL) by Remi

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database oozie;
Query OK, 1 row affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

Na gaba, saita kaddarorin Oozie don MySQL. Bude fayil 'oozie-site.xml' kuma shirya kaddarorin masu zuwa kamar yadda aka nuna.

 cd /etc/oozie/conf
 vi oozie-site.xml

Shigar da kaddarorin masu biyowa (kawai maye gurbin maigida [sunan mai masaukina] da sunan mai masaukinku).

<property>
        <name>oozie.service.JPAService.jdbc.driver</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.url</name>
        <value>jdbc:mysql://master:3306/oozie</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.username</name>
        <value>oozie</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.password</name>
        <value>oozie</value>
    </property>

Zazzage kuma ƙara direban haɗin haɗin JAR na MySQL JDBC zuwa Oozie lib directory. Don yin haka, gudanar da umarni mai mahimmanci akan tashar tashar.

 cd /tmp/
 wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.31.tar.gz
 tar -zxf mysql-connector-java-5.1.31.tar.gz	
 cd mysql-connector-java-5.1.31
 cp mysql-connector-java-5.1.31-bin.jar /var/lib/oozie/

Ƙirƙirar tsarin bayanai na oozie ta aiwatar da umarni a ƙasa kuma a lura cewa ya kamata a gudanar da wannan azaman mai amfani da oozie.

 sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -run
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_LOG=/var/log/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/usr/lib/oozie/oozie-server-0.20
setting CATALINA_OPTS=-Xmx1024m
setting OOZIE_HTTPS_PORT=11443
...
DONE
Oozie DB has been created for Oozie version '3.3.2-cdh4.7.0'
The SQL commands have been written to: /tmp/ooziedb-8250405588513665350.sql

Kuna buƙatar zazzage ExtJS lib daga intanit don kunna na'urar wasan bidiyo na oozie. Jeka shafin CDH ExtJS na hukuma, kuma zazzage dakunan karatu na ExtJS 2.2 ko kuna iya zazzage fakitin ta amfani da umarni mai zuwa.

 cd /tmp/
 wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
 unzip ext-2.2.zip
 mv ext-2.2 /var/lib/oozie/

A ƙarshe, fara uwar garken oozie, ta hanyar bin umarni.

 service oozie status
not running.

 service oozie start

 service oozie status
running

 oozie admin -oozie http://localhost:11000/oozie -status
System mode: NORMAL

Bude oozie UI ta amfani da burauzar da kuka fi so, kuma ku nuna adireshin IP ɗin ku. A wannan yanayin, IP na shine 192.168.1.129.

http://192.168.1.129:11000

Yanzu idan kun ga wannan UI. Taya murna!! Kun yi nasarar daidaita oozie.

Anyi nasarar gwada wannan hanya akan RHEL/CentOS 6/5. A cikin labarai na masu zuwa, zan raba yadda ake daidaitawa da tsara ayyukan hadoop ta hanyar oozie. Kasance da haɗin kai don ƙarin kuma kar a manta da yin tsokaci game da ra'ayoyin ku.