Grafana
Grafana unified alerting delivers to BatonDeck through a Webhook contact point.
- Auth:
api_key— Grafana cannot compute the BatonDeck HMAC, but it can send a custom header. - Endpoint:
POST https://<your-oncall-host>/i/<integrationId> - Header:
x-batondeck-key: <your key>
1. Create the integration
create_integration {
projectId, name: "prod-grafana", source: "grafana", boardId: "B-…",
auth: { mode: "api_key", apiKey: "<a long random value you generate>" },
routing: { mode: "oncall", onCallAgent: "claude-oncall" }
}You generate the key (minimum 8 characters — generate far more, e.g.
openssl rand -base64 32). BatonDeck stores only its SHA-256 hash, so there is no read-back: keep
your copy. The response's endpointPath is what you paste into Grafana.
2. Add the contact point
Alerting → Contact points → Add contact point, integration type Webhook.
| Field | Value |
|---|---|
| URL | https://<your-oncall-host>/i/<integrationId> |
| HTTP Method | POST |
| Custom / extra HTTP headers | name x-batondeck-key, value your key |
Then point a notification policy at the contact point so your alert rules route to it.
The one thing to get right: the key goes in a custom header named
x-batondeck-key. Grafana's webhook contact point also offers Basic Auth and Authorization Header Scheme / Credentials, and both are tempting. Neither works: inapi_keymode the function reads onlyx-batondeck-key, and theAuthorizationheader is read only injwtmode. Putting the key inAuthorizationproducesapi_key_mismatchin the delivery log against a config that looks completely correct. If your Grafana version has no custom-header field, front it with a proxy that adds the header.
3. What lands on the ticket
Grafana's webhook body is fixed — you do not template it. This is how each entry in its alerts[]
array becomes an incident:
| Ticket field | Comes from |
|---|---|
| Correlation key | alerts[].fingerprint (falls back to a hash of the alert's label set) |
| Firing / resolved | alerts[].status — "resolved" resolves, anything else fires |
| Severity | alerts[].labels.severity, defaulting to warning |
| Title | alerts[].annotations.summary, else alerts[].labels.alertname, else "Grafana alert" |
| Description | alerts[].annotations.description |
| Labels / annotations | the alert's own maps, verbatim |
| Links | generatorURL, silenceURL, dashboardURL, panelURL — whichever are present |
| Occurred at | startsAt; for a resolved alert, endsAt when it is a real timestamp |
Two consequences worth designing your alert rules around:
- Give every rule a
summaryannotation. Without one, tickets are titled by rawalertname— readable to you, much less useful to an agent picking the ticket up cold. - Set a
severitylabel. Everything without one iswarning→ priority normal. Usecritical/error/warning/info, or map your own vocabulary with the integration'smapping.severityMap.
Grouped deliveries: one ticket per alert fingerprint
Grafana groups alerts by notification policy before it sends, so a single webhook call can carry
many alerts. BatonDeck fans that out: each entry in alerts[] becomes its own delivery and its own
incident ticket, keyed on its own fingerprint. Twelve alerts in one group is twelve tickets, twelve
correlation keys, twelve independent lifecycles — not one ticket listing twelve problems.
This is deliberate. An agent works one problem at a time, and a grouped ticket cannot be claimed, resolved or closed per-problem. If you want fewer tickets, group less aggressively at the rule level — a single rule producing one alert per instance really is N problems.
The envelope's top-level status is ignored entirely; each alert's own status decides whether it
fires or resolves. A mixed group therefore opens some tickets and resolves others in one call, which
is correct.
When nothing arrives
| Delivery log reason | What it means |
|---|---|
| No rows at all | Grafana never reached the function — check the URL, and check the contact point is actually referenced by a notification policy. Grafana's Test button on the contact point is the quickest probe. |
api_key_mismatch | The header name or value differs — almost always the Authorization trap above. |
404 from the endpoint | Wrong integration id, or the integration is disabled. Both answer identically on purpose. |
