Yadda Ake Sanya Elasticsearch, Logstash, da Kibana (ELK Stack) akan CentOS/RHEL 7
Idan kai mutum ne wanda ke, ko ya kasance a baya, mai kula da dubawa da nazarin rajistan ayyukan a cikin Linux, kun san abin da mafarkin wannan aikin zai iya zama idan ana sa ido kan ayyuka da yawa a lokaci guda.
A kwanakin da suka gabata, dole ne a yi wannan aikin da hannu, tare da sarrafa kowane nau'in log ɗin daban. Abin farin ciki, haɗin Elasticsearch, Logstash, da Kibana a gefen uwar garken, tare da Filebeat a gefen abokin ciniki, ya sa wannan aiki mai wahala ya zama kamar tafiya a cikin wurin shakatawa a yau.
Abubuwan farko guda uku suna samar da abin da ake kira ELK stack, wanda babban manufarsa shine tattara rajistan ayyukan daga sabar da yawa a lokaci guda (wanda kuma aka sani da shiga tsakani).
Ginshikan gidan yanar gizo na tushen java yana ba ku damar bincika rajistan ayyukan cikin sauri a kallo don sauƙin kwatantawa da gyara matsala. Ana aika waɗannan rajistan ayyukan abokin ciniki zuwa uwar garken tsakiya ta Filebeat, wanda za'a iya kwatanta shi azaman wakilin jigilar kaya.
Bari mu ga yadda duk waɗannan sassan suka dace tare. Yanayin gwajin mu zai ƙunshi injuna masu zuwa:
Central Server: CentOS 7 (IP address: 192.168.0.29). 2 GB of RAM. Client #1: CentOS 7 (IP address: 192.168.0.100). 1 GB of RAM. Client #2: Debian 8 (IP address: 192.168.0.101). 1 GB of RAM.
Lura cewa ƙimar RAM ɗin da aka bayar anan ba ƙaƙƙarfan sharuɗɗa ba ne, amma ƙimar da aka ba da shawarar don nasarar aiwatar da tari na ELK akan sabar ta tsakiya. Karancin RAM akan abokan ciniki ba zai haifar da bambanci ba, idan akwai, kwata-kwata.
Sanya ELK Stack akan Sabar
Bari mu fara da shigar da tarin ELK akan uwar garken, tare da taƙaitaccen bayani akan abin da kowane ɓangaren ke yi:
- Elasticsearch yana adana rajistan ayyukan da abokan ciniki suka aiko.
- Logstash yana aiwatar da waɗannan rajistan ayyukan.
- Kibana yana ba da hanyar haɗin yanar gizon da za ta taimaka mana mu bincika da kuma bincika rajistan ayyukan.
Shigar da fakiti masu zuwa akan uwar garken tsakiya. Da farko, za mu shigar da nau'in Java JDK 8 (sabuntawa 102, na baya-bayan nan a lokacin wannan rubutun), wanda ya dogara da abubuwan ELK.
Kuna iya so a fara dubawa a cikin shafin zazzagewar Java a nan don ganin ko akwai sabon sabuntawa da ake samu.
# yum update # cd /opt # wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u102-b14/jre-8u102-linux-x64.rpm" # rpm -Uvh jre-8u102-linux-x64.rpm
Lokaci don bincika ko an kammala shigarwa cikin nasara:
# java -version
Don shigar da sabbin nau'ikan Elasticsearch, Logstash, da Kibana, dole ne mu ƙirƙiri ma'ajiyar yum da hannu kamar haka:
1. Shigo maɓallin Elasticsearch na jama'a GPG zuwa mai sarrafa fakitin rpm:
# rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch
2. Saka layin masu zuwa zuwa fayil ɗin daidaitawa elasticsearch.repo:
[elasticsearch] name=Elasticsearch repository baseurl=http://packages.elastic.co/elasticsearch/2.x/centos gpgcheck=1 gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch enabled=1
3. Sanya kunshin Elasticsearch.
# yum install elasticsearch
Lokacin da shigarwa ya cika, za a sa ka fara da kunna elasticsearch:
4. Fara kuma kunna sabis.
# systemctl daemon-reload # systemctl enable elasticsearch # systemctl start elasticsearch
5. Bada izinin zirga-zirga ta tashar tashar TCP 9200 a cikin Tacewar zaɓinku:
# firewall-cmd --add-port=9200/tcp # firewall-cmd --add-port=9200/tcp --permanent
6. Bincika idan Elasticsearch ya amsa buƙatu masu sauƙi akan HTTP:
# curl -X GET http://localhost:9200
Fitowar umarnin da ke sama yakamata yayi kama da:
Tabbatar kun kammala matakan da ke sama sannan ku ci gaba da Logstash. Tun da Logstash da Kibana suna raba maɓallin Elasticsearch GPG, babu buƙatar sake shigo da shi kafin shigar da fakitin.
7. Saka layin masu zuwa zuwa fayil ɗin daidaitawa logstash.repo:
[logstash] name=Logstash baseurl=http://packages.elasticsearch.org/logstash/2.2/centos gpgcheck=1 gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch enabled=1
8. Shigar da kunshin Logstash:
# yum install logstash
9. Ƙara takardar shaidar SSL dangane da adireshin IP na uwar garken ELK a layin da ke ƙasa da sashin [ v3_ca ] a cikin /etc/pki/tls/openssl.cnf >:
[ v3_ca ] subjectAltName = IP: 192.168.0.29
10. Ƙirƙirar takardar shedar sa hannu ta kai mai aiki na kwanaki 365:
# cd /etc/pki/tls # openssl req -config /etc/pki/tls/openssl.cnf -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt
11. Sanya shigarwar Logstash, fitarwa, da tace fayiloli:
Shigarwa: Ƙirƙiri /etc/logstash/conf.d/input.conf kuma saka layin masu zuwa a ciki. Wannan ya zama dole don Logstash ya koyi yadda ake sarrafa bugun daga abokan ciniki. Tabbatar cewa hanyar zuwa takaddun shaida da maɓalli sun dace da madaidaitan hanyoyin kamar yadda aka bayyana a mataki na baya:
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
Fitowa (/etc/logstash/conf.d/output.conf) fayil:
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
Tace (/etc/logstash/conf.d/filter.conf) fayil. Za mu shigar da saƙonnin syslog don sauƙi:
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
12. Tabbatar da fayilolin sanyi na Logstash.
# service logstash configtest
13. Fara kuma kunna logstash:
# systemctl daemon-reload # systemctl start logstash # systemctl enable logstash
14. Sanya Tacewar zaɓi don ba da damar Logstash don samun rajistan ayyukan daga abokan ciniki (TCP tashar jiragen ruwa 5044):
# firewall-cmd --add-port=5044/tcp # firewall-cmd --add-port=5044/tcp --permanent
14. Saka layin masu zuwa zuwa fayil ɗin daidaitawa kibana.repo:
[kibana] name=Kibana repository baseurl=http://packages.elastic.co/kibana/4.4/centos gpgcheck=1 gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch enabled=1
15. Sanya kunshin Kibana:
# yum install kibana
16. Fara da kunna Kibana.
# systemctl daemon-reload # systemctl start kibana # systemctl enable kibana
17. Tabbatar cewa za ku iya samun dama ga hanyar haɗin yanar gizon Kibana daga wata kwamfuta (ba da damar zirga-zirga akan tashar TCP 5601):
# firewall-cmd --add-port=5601/tcp # firewall-cmd --add-port=5601/tcp --permanent
18. Kaddamar da Kibana (http://192.168.0.29:5601) don tabbatar da cewa za ku iya samun damar haɗin yanar gizon:
Za mu dawo nan bayan mun shigar da kuma saita Filebeat akan abokan ciniki.
Sanya Filebeat akan Sabar Abokin Ciniki
Za mu nuna muku yadda ake yin wannan don Abokin ciniki #1 (maimaita don Abokin ciniki #2 daga baya, canza hanyoyi idan an zartar da rarrabawar ku).
1. Kwafi takardar shaidar SSL daga uwar garken zuwa abokan ciniki:
# scp /etc/pki/tls/certs/logstash-forwarder.crt [email :/etc/pki/tls/certs/
2. Shigo maɓallin Elasticsearch na jama'a GPG zuwa mai sarrafa fakitin rpm:
# rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch
3. Ƙirƙiri ma'ajiya don Filebeat (/etc/yum.repos.d/filebeat.repo) a cikin tushen rarrabawar CentOS:
[filebeat] name=Filebeat for ELK clients baseurl=https://packages.elastic.co/beats/yum/el/$basearch enabled=1 gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch gpgcheck=1
4. Sanya tushen don shigar da Filebeat akan Debian da abubuwan da suka samo asali:
# aptitude install apt-transport-https # echo "deb https://packages.elastic.co/beats/apt stable main" > /etc/apt/sources.list.d/filebeat.list # aptitude update
5. Sanya fakitin Filebeat:
# yum install filebeat [On CentOS and based Distros] # aptitude install filebeat [On Debian and its derivatives]
6. Fara kuma kunna Filebeat:
# systemctl start filebeat # systemctl enable filebeat
Maganar taka tsantsan anan. Ana adana saitin filebeat a cikin fayil ɗin YAML, wanda ke buƙatar tsayayyen shigarwa. Yi hankali da wannan yayin da kuke gyara /etc/filebeat/filebeat.yml kamar haka:
- A ƙarƙashin hanyoyi, nuna waɗanne fayilolin log ɗin ya kamata a aika' zuwa sabar ELK.
- Karkashin masu sa ido:
input_type: log document_type: syslog
- Karƙashin fitarwa:
- Rashin amsa layin da ke farawa da logstash.
- Nuna adireshin IP na uwar garken ELK ɗinku da tashar jiragen ruwa inda Logstash ke sauraren runduna.
- Tabbatar hanyar zuwa takardar shaidar tana nuna ainihin fayil ɗin da kuka ƙirƙira a Mataki na I (Sashen Logstash) a sama.
Ana kwatanta matakan da ke sama a cikin hoto mai zuwa:
Ajiye canje-canje, sannan a sake kunna Filebeat akan abokan ciniki:
# systemctl restart filebeat
Da zarar mun kammala matakan da ke sama akan abokan ciniki, jin daɗin ci gaba.
Don tabbatar da cewa za a iya aika rajistan ayyukan daga abokan ciniki da kuma karɓa cikin nasara, gudanar da umarni mai zuwa akan sabar ELK:
# curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty'
Fitowar yakamata tayi kama da (lura yadda ake karɓar saƙonni daga /var/log/saƙonni da /var/log/amintacce daga abokin ciniki1 da abokin ciniki2):
In ba haka ba, duba fayil ɗin daidaitawar Filebeat don kurakurai.
# journalctl -xe
bayan yunƙurin sake kunnawa Filebeat zai nuna maka layin (s) masu laifi.
Bayan mun tabbatar da cewa abokan ciniki suna jigilar rajistan ayyukan kuma sun sami nasara akan sabar. Abu na farko da za mu yi a cikin Kibana shine saita tsarin ƙididdiga kuma saita shi azaman tsoho.
Kuna iya siffanta fihirisar a matsayin cikakken ma'ajin bayanai a cikin mahallin bayanai mai alaƙa. Za mu tafi tare da
filebeat-*(ko kuma kuna iya amfani da madaidaicin ma'aunin bincike kamar yadda aka bayyana a cikin takaddun hukuma).Shigar da
filebeat-*a cikin sunan Fihirisar ko filin tsari sannan danna Ƙirƙiri:
Da fatan za a lura cewa za a ba ku damar shigar da ƙarin ƙa'idodin bincike mai kyau daga baya. Na gaba, danna tauraro a cikin koren rectangle don saita shi azaman tsohuwar ƙirar ƙira:
A ƙarshe, a cikin menu na Gano za ku sami filayen da yawa don ƙara zuwa rahoton gani na log. Kawai kaɗa su kuma danna Ƙara:
Za a nuna sakamakon a tsakiyar yankin allo kamar yadda aka nuna a sama. Jin daɗin yin wasa (ƙara da cire filayen daga rahoton log) don sanin Kibana.
Ta hanyar tsoho, Kibana zai nuna bayanan da aka sarrafa a cikin mintuna 15 na ƙarshe (duba kusurwar dama ta sama) amma kuna iya canza wannan ɗabi'ar ta zaɓi wani tsarin lokaci:
Takaitawa
A cikin wannan labarin mun bayyana yadda ake saita tarin ELK don tattara bayanan tsarin da abokan ciniki biyu suka aiko, CentOS 7 da injin Debian 8.
Yanzu zaku iya komawa zuwa takaddun Elasticsearch na hukuma kuma ku sami ƙarin cikakkun bayanai kan yadda ake amfani da wannan saitin don dubawa da bincika rajistan ayyukanku da inganci.
Idan kuna da wasu tambayoyi, kar a yi jinkirin yin tambaya. Muna jiran ji daga gare ku.