PagerDuty agents

A PagerDuty agent wakes when an incident triggers, investigates it, and writes what it found back to the incident. pagerdutyChannel() verifies PagerDuty's v3 webhooks and sends every later event for that incident to the session that started the investigation.

Investigate a new incident

When an incident triggers, the agent starts a turn with the incident's title, service, urgency, and link.

// bot/channels/pagerduty.ts
import { defaultPagerDutyAuth, pagerdutyChannel } from "@grok/agent-sdk/channels/pagerduty";

export default pagerdutyChannel({
  deliverReplies: true,
  onIncident: (ctx, event) => {
    if (event.eventType !== "incident.triggered") {
      return null;
    }
    return {
      // `auth` starts a model turn running as the event's PagerDuty actor.
      auth: defaultPagerDutyAuth(ctx),
      message: `Investigate PagerDuty incident #${event.incident.number}: ${event.incident.title}`,
      context: [`Service: ${event.incident.service?.summary ?? "unknown"}`],
    };
  },
});

With deliverReplies: true, the channel posts each reply as a note on the incident. It doesn't post anything until it knows the agent's own PagerDuty user, because each note comes back as a webhook event and would otherwise start another turn. Handlers you write in events replace the built-in ones.

Hooks also get ctx.pagerduty, a REST client with the channel's credentials. Use the incident from the event or channel state when you write, so the model can't choose the target.

onIncident: async (ctx, event) => {
  const incident = await ctx.pagerduty.getIncident(event.incident.id);
  // …
},

On Cursor hosting, declare PagerDuty's API host and the token:

// bot/agent.ts
import { defineAgent } from "@grok/agent-sdk";

export default defineAgent({
  hosting: {
    egressDomains: ["api.pagerduty.com"],
    secretNames: ["PAGERDUTY_TOKEN"],
  },
});

Follow the incident in one session

People acknowledge, annotate, reassign, and resolve an incident while the agent investigates. Return a dispatch for those events too, and each one reaches the session already working on the incident.

onIncident: (ctx, event) => {
  const followUps = ["incident.acknowledged", "incident.reassigned", "incident.resolved"];
  if (event.eventType !== "incident.triggered" && !followUps.includes(event.eventType)) {
    return null;
  }
  return {
    auth: defaultPagerDutyAuth(ctx),
    message: `${event.eventType} by ${ctx.agent?.summary ?? "PagerDuty"}`,
  };
},
onNote: (ctx, note) => {
  return {
    auth: defaultPagerDutyAuth(ctx),
    message: `Note on the incident: ${note.content}`,
  };
},

Add both to the same pagerdutyChannel() call. A re-trigger after a resolve is a new PagerDuty incident, so it starts a new session. ctx.incidentKey carries the alert's dedup key, so the agent can recall what it found the last time the same alert fired.

Ignore your own actions

PagerDuty sends the agent's own changes back as webhook events, such as a note it posted or an incident it resolved. The channel acknowledges those events without starting a turn, and the same action by a person still arrives.

export default pagerdutyChannel({
  // Events caused by the agent's own PagerDuty user are skipped.
  // The user comes from, in order: selfUserIds, PAGERDUTY_USER_ID,
  // or the user PAGERDUTY_TOKEN acts as (GET /users/me).
  selfUserIds: ["PBOT001"],
  onIncident: (ctx) => ({ auth: defaultPagerDutyAuth(ctx) }),
});

A user-level API key or an OAuth user token acts as one user. An account-level key or an app token acts as no user, so the channel uses the user whose email matches PAGERDUTY_FROM_EMAIL. A read-only agent with no token sets PAGERDUTY_USER_ID instead. When none of these names a user, the channel skips nothing.

Events

Each hook receives one family of v3 events:

Hook	Events
onIncident	incident.triggered, .acknowledged, .unacknowledged, .escalated, .reassigned, .delegated, .resolved, .reopened, .priority_updated, .incident_type.changed
onNote	incident.annotated
onResponder	incident.responder.added, incident.responder.replied
onEvent	Everything else, such as status updates and workflow events

Without onIncident, a triggered or reopened incident starts a turn. The channel acknowledges every other event until you add its hook. Each hook returns { auth }, { task }, or null, as Handler results describes.

Connect

Create a v3 webhook subscription in PagerDuty for the services the agent should respond to, and point it at https://<your-host>/<slug>/v1/channels/pagerduty. Set the channel's secrets on the deployment, or as environment variables locally.

Variable	Used for	Override
PAGERDUTY_WEBHOOK_SECRET	Verifying X-PagerDuty-Signature. On Cursor hosting, a delivery with a bad signature gets a 401.	webhookSecret: { secretName }
PAGERDUTY_USER_ID	The agent's own PagerDuty user ID. Optional when PAGERDUTY_TOKEN names a user.	selfUserIds: [ids] or { secretName }
PAGERDUTY_TOKEN	Reply notes, ctx.pagerduty, and finding the agent's user. Use a REST API key, or an OAuth token as Bearer <token>.	token: { secretName }
PAGERDUTY_FROM_EMAIL	The From user for writes with an account-level key or app token.	from: { secretName }
PAGERDUTY_API_BASE_URL	The REST host, such as https://api.eu.pagerduty.com. Defaults to https://api.pagerduty.com.	apiBaseUrl

createPagerDutyClient() takes the same options for code outside the channel.

agent-sdk deploy --dir . --slug incident-responder
agent-sdk secret set incident-responder PAGERDUTY_WEBHOOK_SECRET PAGERDUTY_TOKEN

The next delivery picks up a new value without another deploy. Until the webhook secret is set, the route accepts only loopback deliveries under agent-sdk dev.

Several subscriptions

PagerDuty gives each webhook subscription its own signing secret. Put one secret on each line of PAGERDUTY_WEBHOOK_SECRET:

<signing secret of subscription 1>
<signing secret of subscription 2>

A delivery passes when its signature matches any line. Set the value from that file:

agent-sdk secret set incident-responder PAGERDUTY_WEBHOOK_SECRET < pagerduty-webhook-secrets.txt

To rotate a secret, add the new one before PagerDuty signs with it, and remove the old one once nothing signs with it.

Test locally

agent-sdk pagerduty replay posts synthesized v3 deliveries for one incident to the running agent, in order. Without PAGERDUTY_WEBHOOK_SECRET, agent-sdk dev admits them unsigned from loopback. With it set, replay signs them the way PagerDuty does.

agent-sdk dev &
agent-sdk pagerduty replay --events triggered,acknowledged,annotated,resolved \
  --comment "Rolled back the 09:30 deploy."

Each event joins the session the trigger started. Pass an incident ID to start another session, --dry-run to print the bodies, or --fixtures-dir fixtures/pagerduty to save them as fixtures for evals.

To post a delivery you saved from the webhook log, sign it:

export PAGERDUTY_WEBHOOK_SECRET=local-test-secret
agent-sdk dev &

body=fixtures/pagerduty/incident-triggered.json
signature=$(openssl dgst -sha256 -hmac "$PAGERDUTY_WEBHOOK_SECRET" -hex < "$body" | awk '{print $NF}')
curl -X POST http://127.0.0.1:3000/<slug>/v1/channels/pagerduty \
  -H 'content-type: application/json' \
  -H "x-pagerduty-signature: v1=$signature" \
  --data-binary @"$body"

Related

• Channels reference
• Webhooks and custom channels
• Evals