Setup A Monitoring and Alerting System
Collect system logs
- small: Filebeat -> ElasticSearch -> Kibana
- scale to large: Filebeat or vector.dev agent -> Apache Kafka -> Logstash -> ElasticSearch -> Kibana
Collect base infrastructure logs: the same as Collect system logs.
Collect business logs: the same as Collect system logs.
| Feature | Component | Recommended Version | Required |
|---|---|---|---|
| Collect logs agent | Filebeat | 7.19.x | Y |
| Collect logs agent | vector.dev agent | - | N |
| Log store and full-text search | ElasticSearch | 7.19.x | Y |
| Query and view logs | Kibana | 7.19.x | Y |
| Dashboard for everything | Grafana | 11.6.x | Y |
| Store monitoring metrics | Prometheus | 3.6.x | Y |
| Alerting | Alertmanager | 0.29.x | Y |
| Increase peak log throughput | Logstash | 7.19.x | N |
| Increase peak log throughput | Apache Kafka | 4.x | N |
Why should we stick with ELK 7.x instead of the latest generally available version, 9.x (as of 2025)? Because the features in 7.x are sufficient for most use cases, and it still supports Windows 7 as a development environment.