Observability Driven Development


Why Observability ?

Three Pillers of Observability

Understand the system without getting into the system's console Provides the productive development environment and know your end user issues on UI console.

Logs

Centralized Logging

Real time logs from networks, infrastructure, and applications for search

Metrics

Metrics

Right Sizing and performce of the system

Traces

Traces

APM, RUM => Latency Check for App centric view
Observability Design #1

Kibana

Elasticsearch

distributed search engine

Kibana

Kibana

Dashboards and metrics UI for elasticsearch

Logstash

Logstash

Data Transfer Pipeline

Beats

Logstash

Heartbeat, Packetbeat, Filebeat, Metricbeat, Functionbeat, Auditbeat
Observability Design #2

Prometheus

Prometheus

distributed search engine

Grafana

Grafana

Dashboards and metrics UI for multiple datasources

Loki

Loki

Log aggregation system Tail your logs in realtime

Tempo

Tempo

high-scale distributed tracing backend Identify errors coming from multiple services