Prometheus

1
What Is Apache Prometheus ?

• A system for event monitoring and alerting
• Open source / Apache 2.0 license
• Records real-time metrics in a time series
  database
• Released by Cloud Native Computing Foundation
• Storage in memory / on disk / various
  integrations
• Has multiple visualisation options
• Written in Go

2
What Is Apache Prometheus ?

• Provides PromQL query language
• Has a highly dimensional data model
• Has many third party integration options
• Many client libraries available
• Alerting via PromQL and alertManager
• Has wide community support

3
Prometheus Client Libraries

• Official client libraries include
• Go, Java, Scala, Python, Ruby
• Unofficial client libraries include
• Bash, C, C, Common Lisp, Dart
• Elixir, Erlang, Haskell, Lua for Nginx
• Lua for Tarantool, .NET / C, Node.js
• Perl, PHP, R, Rust
• Also use text base exposition formats to
• Expose metrics to Prometheus

4
Prometheus Storage

• By default Prometheus stores to local file system
• But this storage is not clustered or replicated
• Does not scale and is not durable, alternatives
  include

• Google Cloud Spanner (R/W)
• Graphite (W)
• InfluxDB (R/W)
• IRONdb (R/W)
• Kafka (W)
• M3DB (R/W)
• OpenTSDB (W)
• PostgreSQL/TimescaleDB (R/W)

• QuasarDB (R/W)
• SignalFx (W)
• Splunk (R/W)
• TiKV (R/W)
• Thanos (R/W)
• VictoriaMetrics (W)
• Wavefront (W)

• AppOptics (W)
• Azure Data Explorer (R/W)
• Azure Event Hubs (W)
• Chronix (W)
• Cortex (R/W)
• CrateDB (R/W)
• Elasticsearch (W)
• Gnocchi (W)

5
Architecture

• Prometheus server scrapes and stores time series
  data
• Client libraries for instrumenting application
  code
• Push gateway for supporting short-lived jobs
• Special-purpose exporters for services like
• HAProxy, StatsD, Graphite, etc.
• Alertmanager to handle alerts
• Various support tools

6
Architecture
7
Prometheus Usage

• Works well for
• Recording any purely numeric time series data
• Machine-centric monitoring
• Monitoring dynamic service-oriented architectures
• Does not works for
• Situations where you need 100 accuracy
• Such as for per-request billing

8
Prometheus Users
9
Visualisation

• Uses Grafana data source for visualisation
• Can create Grafana graphs and dashboards
• Can import pre-built dashboards from Grafana.com
• Can have dashboards update in real time
• See next slide for an example

10
Visualisation
11
Prometheus PromQL

• Select and aggregate time series data in real
  time
• Display data as graph, table or send to HTTP API
• PromQL expressions evaluate to
• Instant vector
• A set of time series containing a single sample
  for each time series, all sharing the same
  timestamp
• Range vector
• A set of time series containing a range of data
  points over time for each time series
• Scalar, String
• Instant vector is the only type that can be
  directly graphed

12
Prometheus PromQL Examples

• Return all time series with the metric
  http_requests_total
• http_requests_total
• Return all time series with the metric
• http_requests_total and
• the given job and handler labels
• http_requests_totaljob"apiserver",
  handler"/api/comments"
• Return a whole range of time
• (in this case 5 minutes) for the same vector
• making it a range vector
• http_requests_totaljob"apiserver",
  handler"/api/comments"5m

13
Available Books

• See Big Data Made Easy
• Apress Jan 2015
• See Mastering Apache Spark
• Packt Oct 2015
• See Complete Guide to Open Source Big Data
  Stack
• Apress Jan 2018
• Find the author on Amazon
• www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
• Connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020

14
Connect

• Feel free to connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020
• See my open source blog at
• open-source-systems.blogspot.com/
• I am always interested in
• New technology
• Opportunities
• Technology based issues
• Big data integration