Monitoring & Grafana Dashboards

How to start the monitoring stack, what dashboards ship with the app, the file formats involved, and how to create or import your own.

What's In the Stack

Service	Port	Job
Prometheus	`9090`	Scrapes metrics from services (hyperswitch, killbill, itself)
Grafana	`3000`	Dashboards + alerts, reads from Prometheus & Loki
Loki	`3100`	Stores aggregated logs
Promtail	`9080`	Ships Docker container logs to Loki via the Docker socket

Step 1 — Start the Monitoring Stack

terminal

docker compose --profile monitoring up -d

# Verify all four are healthy
docker compose ps prometheus grafana loki promtail

Log in to Grafana at http://localhost:3000 with admin / admin (or the GRAFANA_ADMIN_PASSWORD from your root .env). You'll be asked to set a new password on first login.

Step 2 — Dashboards That Ship With the App

The repo provisions dashboards automatically from monitoring/grafana/dashboards/dashboards/. The Prometheus data source and the OpenPay Overview dashboard are loaded at container start (no manual import needed):

OpenPay Overview — service up/down status for Hyperswitch, Kill Bill, and the total up count.

Config file	Purpose
`monitoring/prometheus/prometheus.yml`	What Prometheus scrapes and how often (15s)
`monitoring/grafana/datasources/prometheus.yml`	Registers Prometheus as Grafana's data source
`monitoring/grafana/dashboards/dashboards.yml`	Tells Grafana to auto-load dashboards from a folder
`monitoring/grafana/dashboards/dashboards/*.json`	The actual dashboards (one JSON file per dashboard)
`monitoring/loki/loki.yml`	Loki storage/retention config
`monitoring/promtail/config.yml`	Log shipper config (Docker socket → Loki)

Step 3 — The File Format (What a Dashboard Actually Is)

A Grafana dashboard is a single JSON file — nothing more. The main blocks are:

title & uid — dashboard name and unique ID
panels[] — each panel is one visualization (stat, time series, bar gauge…)
panels[].targets[] — the PromQL queries powering each panel
templating — optional dropdown variables (e.g. pick a service)
time — the default time range

minimal dashboard JSON (shape)

{
  "title": "My Dashboard",
  "uid": "my-dashboard",
  "panels": [
    {
      "type": "stat",
      "title": "Hyperswitch Up?",
      "targets": [
        { "expr": "up{job="hyperswitch"}", "legendFormat": "Hyperswitch" }
      ]
    }
  ],
  "time": { "from": "now-6h", "to": "now" }
}

Step 4 — Create a Dashboard (Three Ways)

A. In the UI (easiest)

In Grafana: Dashboards → New → New dashboard → Add visualization. Pick the Prometheus data source, write a PromQL query, save. Then export via Share → Export → Save JSON to file if you want it in the repo.

B. Import a template

Grafana hosts community templates at grafana.com/grafana/dashboards (filter by data source = Prometheus). Copy the ID, then in Grafana: Dashboards → New → Import → paste ID → Load. This is the fastest way to get battle-tested dashboards.

C. Add to the repo

Drop the JSON into monitoring/grafana/dashboards/dashboards/ and restart Grafana — the provider auto-loads it. This is how the bundled OpenPay Overview dashboard is shipped.

add a dashboard to the repo

# 1. Place your exported JSON here:
#    monitoring/grafana/dashboards/dashboards/my-dashboard.json

# 2. Reload (picks up within 30s, or force it):
docker compose restart grafana

Step 5 — Explore Logs

Promtail ships every Docker container's logs to Loki. In Grafana open Explore, switch the data source to Loki, and query e.g. {service="hyperswitch"} or use the Log labels dropdown to pick a container.

example Loki queries

{service="hyperswitch"}          # all Hyperswitch logs
{service="killbill"} |~ "error"  # Kill Bill errors only
{container="core-hyperswitch"}

Troubleshooting

Dashboards don't appear: ensure the four monitoring containers are healthy (docker compose ps), then restart Grafana.
Panels show “No data”: the target service may not expose Prometheus metrics, or the scrape target is unreachable. Check http://localhost:9090/targets for scrape errors.
No logs in Loki: confirm the Docker socket is mounted to promtail (already in docker-compose.yml) and check docker compose logs promtail.