组件描述

组件	功能描述	核心作用	主要用途
Elasticsearch	搜索和分析引擎，负责处理大规模数据的搜索和分析。	Elastic Stack的核心，提供数据索引和查询能力。	存储和快速检索海量数据
Kibana	数据可视化平台，支持创建图表和仪表盘，监控Elastic Stack状态，集成多种应用。	展示和理解Elasticsearch数据，增强用户体验。	可视化数据分析和监控
Logstash	数据收集引擎，实时采集、解析和存储数据。	收集、过滤和解析数据至目标数据库。	日志和事件数据的集中处理
Filebeat	轻量级日志收集器，部署在服务器上收集并转发日志文件。	收集本地或远程服务器的日志数据。	高效、轻量的日志数据收集和传输

filebeat

下载安装包

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.12.2-linux-x86_64.tar.gz

解压到安装目录

tar xf filebeat-8.12.2-linux-x86_64.tar.gz -C /data/
mv /data/filebeat-8.12.2-linux-x86_64/ /data/filebeat-8.12.2

编辑配置文件/data/filebeat-8.12.2/filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /apps/logs/*.log
  tags: ["applogs"]
  fields_under_root: true

output.logstash:
  hosts: ["x.x.x.x:5044"]

创建systemd配置文件

cat > /etc/systemd/system/filebeat.service << EOF
[Unit]
Description=Filebeat - Lightweight Shipper for Logs and Metrics
Documentation=https://www.elastic.co/guide/en/beats/filebeat/current/index.html
Wants=network-online.target
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/data/filebeat-8.12.2/filebeat -c /data/filebeat-8.12.2/filebeat.yml -e
Restart=on-failure
LimitNOFILE=65536
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

[Install]
WantedBy=multi-user.target
EOF

启动服务

systemctl enable filebeat
systemctl start filebeat

logstash

下载安装包

wget https://artifacts.elastic.co/downloads/logstash/logstash-8.12.2-linux-x86_64.tar.gz

解压到安装目录

tar -xf logstash-8.12.2-linux-x86_64.tar.gz -C /data/

清空配置文件

> logstash.yml

编辑配置文件/data/logstash-8.12.2/config/logstash.conf

input {
	beats {
		port => 5044
	}

	tcp {
		port => 50000
	}
}

filter {
  json {
    source => "original_message"
  }

  date {
    match => [ "timestamp", "ISO8601" ]
    locale => en
  }

  mutate {
    remove_field => ["event", "log", "host", "@version", "timestamp", "original_message"]
  }

}

output {
  elasticsearch {
    hosts => ["http://x.x.x.x:9200","http://x.x.x.x:9200","http://x.x.x.x:9200"]
    index => "<index>-%{+YYYY.MM.dd}"
    user  => "elastic"
    password => "<password>"
  }
}

创建用户和组

sudo useradd -r logstash
sudo groupadd logstash
sudo usermod -a -G logstash logstash
sudo chown -R logstash:logstash /data/logstash-8.12.2
sudo chmod -R 755 /data/logstash-8.12.2

创建systemd配置文件

cat > /etc/systemd/system/logstash.service << EOF
[Unit]
Description=Logstash
Documentation=https://www.elastic.co/guide/en/logstash/current/index.html
ConditionPathExists=/data/logstash-8.12.2/bin/logstash
After=network.target

[Service]
#User=logstash
#Group=logstash
#Environment="LS_JAVA_OPTS=-Xmx2g -Xms2g"
ExecStart=/data/logstash-8.12.2/bin/logstash -f /data/logstash-8.12.2/config/logstash.conf
Restart=always
RestartSec=5
TimeoutSec=300
LimitNOFILE=65536
LimitMEMLOCK=infinity
Nice=19

[Install]
WantedBy=multi-user.target
EOF

启动服务

sudo systemctl start logstash
sudo systemctl enable logstash

kibana

设置es密码

elasticsearch-reset-password -u kibana_system -i

下载安装包

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.12.2-linux-x86_64.tar.gz

解压安装包

tar xf kibana-8.12.2-linux-x86_64.tar.gz -C /data/

编辑配置文件/data/kibana-8.12.2/config/kibana.yml

server.port: 5601
server.host: "x.x.x.x"
server.ssl.enabled: false
elasticsearch.hosts: ["http://x.x.x.x:9200","http://x.x.x.x:9200","http://x.x.x.x:9200","http://x.x.x.x:9200","http://x.x.x.x:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "xxxxxxx"
elasticsearch.ssl.verificationMode: none
logging.appenders.default:
  type: file
  fileName: /var/logs/kibana.log
  layout:
    type: json
i18n.locale: "zh-CN"
monitoring.ui.ccs.enabled: false
xpack.monitoring.ui.container.elasticsearch.enabled: true

创建kibana systemd配置文件

cat > /etc/systemd/system/kibana.service << EOF
[Unit]
Description=Kibana
Documentation=https://www.elastic.co/guide/en/kibana/current/index.html
Wants=network-online.target
After=network-online.target

[Service]
#User=kibana
#Group=kibana
#Environment="NODE_OPTIONS=-Xms512m -Xmx512m"
ExecStart=/data/kibana-8.12.2/bin/kibana --allow-root
Restart=always
RestartSec=5
TimeoutSec=300
LimitNOFILE=65536
LimitMEMLOCK=infinity
Nice=19
KillMode=mixed

[Install]
WantedBy=multi-user.target
EOF

启动服务

systemctl start kibana
systemctl enable kibana

访问账户：elastic/xxxxxxx

创建视图：Stack Management》Kibana》数据视图》创建视图