elk-logstash-kibana（三）

时间 2019-11-24 标签 elk logstash kibana 三

1、修改logstash.ymljava

 unzip logstash-6.3.2

vim config/logstash.yml

#添加：检查全部ip

http.host: "0.0.0.0"

2、在logstash下的config下建立（根据本身log日志地址设置：安装nginx等）node

[elk@xiaoyuer config]$ cat syslog.conf 
input {
  syslog {
    type => "system-syslog"
    port => 10514
  }
}
output {
  elasticsearch {
    hosts => ["139.199.124.136:9200"]  # 定义es服务器的ip
    index => "system-syslog-%{+YYYY.MM.dd}" # 定义索引
  }
}
input {
  file {  # 指定一个文件做为输入源
    path => "/home/elk/nginx/logs/elk_access.log"  # 指定文件的路径
    start_position => "beginning"  # 指定什么时候开始收集
    type => "nginx"  # 定义日志类型，可自定义
  }
}
filter {  # 配置过滤器
    grok {
        match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:http_verb} %{NOTSPACE:http_request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_http_request})\" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"}  # 定义日志的输出格式
    }
    geoip {
        source => "clientip"
    }
}
output {
    stdout { codec => rubydebug }
    elasticsearch {
        hosts => ["139.199.124.136:9200"]
        index => "nginx-test-%{+YYYY.MM.dd}"
  }
}

nginx日志编译二进制：python

nginx添加：

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

3、vim kibana.ymllinux

tar xf kibana-6.3.2-linux-x86_64.tar.gz

#vim kibana-6.3.2-linux-x86_64/config/kibana.yml

elasticsearch.url: "http://139.199.124.136:9200"    #ela地址
server.host: 0.0.0.0
server.port: 5601
logging.dest: /elk/logs/kibana.log  #日志存放目录，自定义

4、配置kibana服务器的ip以及配置的监听端口：nginx

vim /etc/rsyslog.confvim

*.* @@192.168.77.130:10514

重启rsyslogruby

systemctl restart rsyslog服务器

-----------启动服务查看elasticsearch

[root@xiaoyuer ~]# netstat -lntup|egrep '9600|5601'tcp

tcp        0      0 0.0.0.0:9600                0.0.0.0:*                   LISTEN      5076/java           
tcp        0      0 0.0.0.0:5601                0.0.0.0:*                   LISTEN      5073/node

5、启动服务汇总

#cat /etc/rc.local
/home/elk/nginx/sbin/nginx
su -c '/home/elk/elasticsearch-6.3.2/bin/elasticsearch -d >>/dev/null 2>&1 &' elk
#su -c 'cd /home/elk/elasticsearch-head-master;grunt server >>/dev/null 2>&1 &' elk
su -c '/home/elk/kibana-6.3.2-linux-x86_64//bin/kibana  >>/dev/null 2>&1 &' elk
su -c '/home/elk/logstash-6.3.2/bin/logstash -f /home/elk/logstash-6.3.2/config/syslog.conf  >>/dev/null 2>&1 &' elk
#su -c '/home/elk/filebeat-6.3.2-linux-x86_64/filebeat &' elk