Intercom Observability

Overview

Comprehensive observability for Intercom integrations covering Prometheus metrics,

OpenTelemetry traces, structured logging, and alert rules for error rates, latency,

and rate-limit usage. Read this page for the workflow and shape of each layer, then

drill into references/implementation.md for the full,

copy-pasteable code and references/examples.md for

end-to-end worked scenarios.

Prerequisites

• Prometheus or compatible metrics backend
• OpenTelemetry SDK (optional, for tracing)
• Pino or similar structured logger
• Grafana or alerting system

Instructions

Build the six observability layers in order. Each step below is the summary and the

essential skeleton — the complete implementation for every step lives in

references/implementation.md.

Step 1: Prometheus metrics

Define five instruments on a shared Registry: a request counter, a duration

histogram, an error counter, a rate-limit gauge, and a webhook counter. Label by

endpoint/method/status (never by unbounded IDs — see Error Handling).

import { Registry, Counter, Histogram, Gauge } from "prom-client";
const registry = new Registry();
const intercomRequests = new Counter({
  name: "intercom_api_requests_total",
  help: "Total Intercom API requests",
  labelNames: ["endpoint", "method", "status"] as const,
  registers: [registry],
});
// + duration Histogram, error Counter, rate-limit Gauge, webhook Counter

Full metric set → references/implementation.md, Step 1.

Step 2: Instrumented client wrapper

Wrap IntercomClient in a Proxy that times every service method, increments the

success/error counters, records error/status codes on IntercomError, and zeros the

rate-limit gauge on a 429 — so instrumentation is automatic for all endpoints.

Full proxy → references/implementation.md, Step 2.

Step 3: Structured logging

Configure Pino with a contact serializer that emits only id/role and never

logs email, name, or phone. Add logIntercomOp and logWebhook helpers for

consistent operation/webhook log lines.

Full logger → references/implementation.md, Step 3.

Step 4: OpenTelemetry tracing

Wrap calls in tracedIntercomCall, which opens a per-operation intercom.* span, sets

OK/ERROR status, records exceptions, and attaches statuscode/errorcode/request_id

attributes on Intercom errors.

Full tracer → references/implementation.md, Step 4.

Step 5: Alert rules

Ship the Prometheus rule group with five alerts: high error rate (>5%), high P95

latency (>3s), low rate limit (<1000), auth failures (401s), and webhook failures.

Full YAML → references/implementation.md, Step 5.

Step 6: Metrics endpoint

Expose the registry on GET /metrics for Prometheus to scrape.

Full route → references/implementation.md, Step 6.

Output

Applying this skill produces:

• Instrumented client — an IntercomClient proxy that emits metrics on every call, with zero per-call changes to existing code.
• Metrics — intercomapirequeststotal, intercomapirequestdurationseconds, intercomapierrorstotal, intercomratelimitremaining, intercomwebhooksprocessedtotal, scraped at GET /metrics.
• Traces — one per-operation intercom.* span per call, with Intercom error attributes on failures.
• Structured logs — PII-redacted JSON operation and webhook log lines.
• Alerts — a Prometheus rule group covering error rate, latency, rate limit, auth, and webhooks.

Key metrics summary

Error Handling

Examples

The following scenarios are covered in full in

references/examples.md:

• Contact lookup end-to-end — one contacts.find call producing a counter increment, a histogram sample, a span, and a PII-redacted log line.
• Rate-limit (429) event — how the proxy zeros the rate-limit gauge and which alerts fire.
• Webhook success/failure accounting — counting processed vs. failed webhooks per topic.
• Scraping /metrics — the raw Prometheus exposition and how it feeds Grafana and the alert rules.

Minimal end-to-end skeleton:

const client = instrumentedClient(new IntercomClient({ token: process.env.INTERCOM_ACCESS_TOKEN! }));
const contact = await tracedIntercomCall(
  "contacts.find",
  { "intercom.contact_id": contactId },
  () => client.contacts.find({ contactId })
);

Resources

• full implementation walkthrough — every step's complete code
• worked examples — end-to-end scenarios and observed output
• Prometheus Best Practices
• OpenTelemetry Node.js
• Pino Logger

Next Steps

For incident response once these signals are firing, see the intercom-incident-runbook

skill, which turns these alerts into a triage-and-mitigation procedure.