Prometheus Alertmanager

Alertmanager delivers to BatonDeck through its native webhook_config — no bridge, no exporter.

• Auth: api_key — Alertmanager cannot compute the BatonDeck HMAC, but it can send a custom header.
• Endpoint: POST https://<your-oncall-host>/i/<integrationId>
• Header: x-batondeck-key: <your key>

1. Create the integration

create_integration {
  projectId, name: "prod-alertmanager", source: "alertmanager", boardId: "B-…",
  auth: { mode: "api_key", apiKey: "<a long random value you generate>" },
  routing: { mode: "broadcast" }
}

You generate the key (openssl rand -base64 32 is a reasonable source). Only its SHA-256 hash is stored — there is no read-back, so keep your copy.

2. Configure the receiver

receivers:
  - name: batondeck
    webhook_configs:
      - url: https://<your-oncall-host>/i/ing_0123456789abcdef0123456789abcdef
        send_resolved: true
        max_alerts: 0
        http_config:
          http_headers:
            x-batondeck-key:
              secrets: [ "<your key>" ]

route:
  receiver: batondeck
  group_by: [alertname, cluster, service]

Validate before reloading: amtool check-config alertmanager.yml.

The one thing to get right: use http_headers, not authorization. http_config.authorization and http_config.basic_auth both produce an Authorization header, and in api_key mode the function never reads it — Authorization is consulted only in jwt mode. The key must arrive as x-batondeck-key or the delivery is refused with api_key_mismatch, which looks like a wrong key rather than a wrong header. If amtool rejects http_headers, your Alertmanager predates the field: put a proxy in front that adds the header, or use jwt mode with a real IdP-issued token.

send_resolved: true matters. Without it Alertmanager never tells BatonDeck an alert cleared, so every incident ticket stays open until a human closes it.

max_alerts: 0 matters too. A non-zero max_alerts makes Alertmanager truncate the batch and report the count in truncatedAlerts — a field BatonDeck does not read. Truncated alerts simply never become tickets, silently.

3. What lands on the ticket

Alertmanager's body is fixed. Each entry in its alerts[] array maps like this:

So the two rule-authoring habits that pay off here are the standard Prometheus ones: a summary annotation (it becomes the ticket title) and a severity label (it becomes the priority). Without severity everything is warning → normal; map your own vocabulary with mapping.severityMap.

Grouped alerts: one ticket per alert fingerprint

group_by means one webhook call routinely carries many alerts. BatonDeck fans that out: every entry in alerts[] becomes its own delivery and its own incident ticket, keyed on its own fingerprint. A group of twenty is twenty tickets with twenty independent lifecycles — because an agent claims, works and closes one problem at a time, and a grouped ticket cannot be closed per-problem.

The envelope's top-level status is ignored; each alert's own status decides. A group containing both firing and resolved alerts therefore does both in one call.

Repeat notifications are deduplicated

Alertmanager is the only source that supplies its own delivery identifier: the envelope's groupKey. Each event is then keyed <groupKey>#<fingerprint> — and groupKey is stable across repeat_interval re-notifications of the same group.

The practical effect: re-notifying an unchanged alert is free and cannot duplicate a ticket, but it also does not register as a fresh occurrence. The open incident ticket is already the running record of that alert; a repeat notification does not add to it.

When nothing arrives

Next: On-call ingestion overview · Grafana