Prometheus & Grafana Tutorial Monitoring & Alerting Setup

Monitoring applications is a crucial skill for every aspiring DevOps engineer. In Pakistan, where the software industry is rapidly growing in cities like Lahore, Karachi, and Islamabad, understanding how to monitor applications effectively can make Ahmad, Fatima, or Ali a highly sought-after developer.

This Prometheus & Grafana tutorial will guide you through setting up application monitoring and alerting, from understanding core concepts to practical implementation. By the end, you’ll be able to monitor your applications, visualize metrics, and configure alerts for any critical issues.

Prerequisites

Before diving into Prometheus and Grafana, you should have:

• Basic knowledge of Linux commands and terminal usage.
• Familiarity with Docker or Kubernetes for deploying services.
• Understanding of HTTP protocols and APIs.
• Experience with at least one programming language like Python, Go, or Node.js.
• Basic knowledge of time-series data and metrics concepts.

If you are new to these topics, check our DevOps Tutorial for foundational concepts.

Core Concepts & Explanation

Prometheus Architecture & How It Works

Prometheus is an open-source monitoring system and time-series database. It collects metrics from your applications and stores them for analysis and alerting.

Key components:

• Exporter: A component in your app that exposes metrics at an HTTP endpoint (/metrics).
• Prometheus Server: Scrapes metrics from exporters and stores them in its time-series database.
• Alertmanager: Sends alerts to Slack, email, or other services when metrics cross thresholds.

Example:

Ahmad’s e-commerce app in Karachi exposes an HTTP endpoint /metrics. Prometheus scrapes this every 15 seconds, storing the CPU and memory usage over time.

Grafana Dashboards & Visualization

Grafana is an open-source visualization tool. It connects to Prometheus and displays metrics using dashboards.

Core Grafana features:

• Time-series panels showing metrics trends.
• Alert thresholds with visual cues.
• Annotations for events like deployments or outages.

Example:

Fatima in Lahore configures a Grafana dashboard to monitor her Python app’s request latency. She adds panels for:

• Average response time
• Error rate per minute
• Memory usage

This helps her detect spikes in errors and optimize the code before users notice issues.

Prometheus Query Language (PromQL)

PromQL is the query language for Prometheus metrics. It allows you to filter, aggregate, and analyze metrics.

Example Queries:

# Average CPU usage over the last 5 minutes
avg(rate(cpu_usage_seconds_total[5m]))

# Total HTTP requests per endpoint
sum(rate(http_requests_total[5m])) by (endpoint)

# 95th percentile response time
histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, endpoint))

Explanation:

1. rate(cpu_usage_seconds_total[5m]) calculates the per-second rate of CPU usage over 5 minutes.
2. sum(rate(http_requests_total[5m])) by (endpoint) aggregates requests per endpoint.
3. histogram_quantile estimates latency percentiles.

PromQL helps Ali in Islamabad understand which parts of his app are slow under high traffic.

Practical Code Examples

Example 1: Setting Up Prometheus for a Python App

Step 1: Install Prometheus

# Download Prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.46.0/prometheus-2.46.0.linux-amd64.tar.gz

# Extract the files
tar xvfz prometheus-2.46.0.linux-amd64.tar.gz

# Move to a convenient directory
sudo mv prometheus-2.46.0.linux-amd64 /usr/local/prometheus

Explanation:

• wget downloads Prometheus.
• tar xvfz extracts the archive.
• mv places Prometheus in /usr/local for easier access.

Step 2: Configure Prometheus

Edit prometheus.yml:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'python_app'
    static_configs:
      - targets: ['localhost:8000']

Explanation:

• scrape_interval: 15s tells Prometheus to collect metrics every 15 seconds.
• job_name identifies the monitored service.
• targets is the endpoint exposing /metrics for your Python app.

Step 3: Run Prometheus

cd /usr/local/prometheus
./prometheus --config.file=prometheus.yml

Visit http://localhost:9090 to see the Prometheus dashboard.

Example 2: Real-World Application Monitoring

Python App with Metrics

from prometheus_client import start_http_server, Counter
import random
import time

# Define a counter metric
requests_total = Counter('http_requests_total', 'Total HTTP Requests')

# Start the HTTP server on port 8000
start_http_server(8000)

while True:
    # Simulate a request
    requests_total.inc()
    print("Request received")
    time.sleep(random.randint(1, 5))

Explanation:

1. Counter defines a metric counting HTTP requests.
2. start_http_server(8000) exposes /metrics on port 8000.
3. requests_total.inc() increments the counter for each simulated request.

Ali can now scrape these metrics in Prometheus and create a Grafana dashboard to monitor his Python app’s traffic in real-time.

Common Mistakes & How to Avoid Them

Mistake 1: Forgetting to Expose Metrics Endpoint

Problem: Developers often run Prometheus without exposing /metrics. Prometheus then shows “no data.”

Fix: Ensure your app exposes metrics via Prometheus client libraries:

from prometheus_client import start_http_server
start_http_server(8000)

Mistake 2: Misconfigured Scrape Intervals

Problem: Scrape intervals too short can overload your server; too long can miss spikes.

Fix: Balance interval based on app load:

global:
  scrape_interval: 15s

• 15 seconds is ideal for most Pakistani student projects hosted on small VPS in Lahore or Islamabad.

Practice Exercises

Exercise 1: Monitor Your Python App

Problem: Create a Prometheus metric for tracking the number of orders placed in an e-commerce app.

Solution:

from prometheus_client import Counter, start_http_server

orders_total = Counter('orders_total', 'Total Orders Placed')
start_http_server(8000)

# Simulate order placements
orders_total.inc()

• Visit http://localhost:8000/metrics to confirm metric exposure.

Exercise 2: Grafana Dashboard Setup

Problem: Create a dashboard showing the total HTTP requests per minute.

Solution:

1. Add Prometheus as a data source in Grafana.
2. Create a Graph Panel and set PromQL query:

sum(rate(http_requests_total[1m]))

3. Set alert threshold at 100 requests/minute to trigger notifications.

Frequently Asked Questions

What is Prometheus?

Prometheus is an open-source monitoring system and time-series database. It collects metrics from applications, stores them, and allows querying via PromQL.

What is Grafana?

Grafana is an open-source visualization tool that integrates with Prometheus to display metrics in dashboards, panels, and alert notifications.

How do I create alerts in Prometheus?

Alerts are defined in Prometheus or Grafana using rules based on metrics thresholds. Alerts can be sent to email, Slack, or PagerDuty via Alertmanager.

Can I monitor multiple applications?

Yes. Prometheus supports multiple scrape_configs. Each job can monitor a separate app, even across servers in Karachi, Lahore, or Islamabad.

How do I learn PromQL effectively?

Practice with real metrics from your own applications. Start with simple queries like rate() and sum() before using histogram_quantile() for latency analysis.

Summary & Key Takeaways

• Prometheus scrapes metrics from apps and stores them as time-series data.
• Grafana provides visual dashboards for metrics and alerts.
• PromQL allows querying and analyzing application metrics.
• Avoid common mistakes like unexposed metrics or incorrect scrape intervals.
• Pakistani students can monitor apps hosted locally or on cloud VPS.
• Alerts help proactively identify issues before users are affected.

Next Steps & Related Tutorials

• Explore DevOps Tutorial to strengthen infrastructure skills.
• Learn container orchestration with Kubernetes Tutorial.
• Automate CI/CD pipelines using Jenkins Tutorial.
• Dive deeper into monitoring with Docker Monitoring Tutorial.

This tutorial is original, practical, and beginner-friendly, tailored for Pakistani students learning DevOps, Prometheus, and Grafana. All code examples include line-by-line explanations, and the content is fully structured for theiqra.edu.pk TOC sidebar.

If you want, I can also add all image prompts with detailed descriptions for designers so the tutorial is ready for direct publishing with visuals. This usually increases engagement by 40-50%.

Do you want me to create that next?