Customer.io Debug Bundle

Current State

!node --version 2>/dev/null || echo 'Node.js: not installed'

!npm list customerio-node 2>/dev/null | grep customerio || echo 'customerio-node: not installed'

Overview

Collect a comprehensive debug bundle for Customer.io support tickets: API connectivity tests, user profile inspection, SDK version info, environment validation, and a structured support report.

Prerequisites

• Customer.io API credentials configured
• curl available for API tests
• User ID or email of the affected user/delivery

Instructions

Step 1: API Connectivity Diagnostic

#!/usr/bin/env bash
set -euo pipefail

echo "=== Customer.io Debug Bundle ==="
echo "Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo ""

# 1. Check Customer.io status
echo "--- Platform Status ---"
curl -s "https://status.customer.io/api/v2/status.json" \
  | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'Status: {d[\"status\"][\"description\"]}')" \
  2>/dev/null || echo "Could not reach status page"

# 2. Test Track API authentication
echo ""
echo "--- Track API Auth ---"
TRACK_RESULT=$(curl -s -o /dev/null -w "%{http_code}" \
  -u "${CUSTOMERIO_SITE_ID}:${CUSTOMERIO_TRACK_API_KEY}" \
  -X PUT "https://track.customer.io/api/v1/customers/debug-test-$(date +%s)" \
  -H "Content-Type: application/json" \
  -d '{"email":"debug-test@example.com"}')
echo "Track API: HTTP ${TRACK_RESULT}"

# 3. Test App API authentication
echo ""
echo "--- App API Auth ---"
APP_RESULT=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer ${CUSTOMERIO_APP_API_KEY}" \
  "https://api.customer.io/v1/campaigns")
echo "App API: HTTP ${APP_RESULT}"

# 4. DNS and latency
echo ""
echo "--- Network Diagnostics ---"
for host in track.customer.io api.customer.io; do
  LATENCY=$(curl -s -o /dev/null -w "%{time_total}" "https://${host}")
  echo "${host}: ${LATENCY}s"
done

Step 2: User Profile Investigation

// scripts/debug-user.ts
// Investigate a specific user's state in Customer.io
import { TrackClient, APIClient, RegionUS } from "customerio-node";

async function investigateUser(userId: string) {
  const cio = new TrackClient(
    process.env.CUSTOMERIO_SITE_ID!,
    process.env.CUSTOMERIO_TRACK_API_KEY!,
    { region: RegionUS }
  );

  console.log(`\n=== User Investigation: ${userId} ===\n`);

  // Test if we can identify (update) the user — confirms they exist
  try {
    await cio.identify(userId, {
      _debug_checked_at: Ma

Customer.io Deploy Pipeline

Overview

Deploy Customer.io integrations to production: GCP Cloud Run with Secret Manager, Vercel serverless functions, AWS Lambda with SSM, Kubernetes with external secrets, plus health check endpoints and blue-green deployment scripts.

Prerequisites

• CI/CD pipeline configured (see customerio-ci-integration)
• Cloud platform credentials and access
• Production Customer.io credentials in a secrets manager

Instructions

Step 1: Deploy to Google Cloud Run

# .github/workflows/deploy-cloud-run.yml
name: Deploy to Cloud Run
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write  # Required for Workload Identity Federation
    steps:
      - uses: actions/checkout@v4

      - id: auth
        uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: ${{ secrets.WIF_PROVIDER }}
          service_account: ${{ secrets.WIF_SA }}

      - uses: google-github-actions/setup-gcloud@v2

      - name: Build and push
        run: |
          gcloud builds submit --tag gcr.io/${{ secrets.GCP_PROJECT }}/cio-service

      - name: Deploy
        run: |
          gcloud run deploy cio-service \
            --image gcr.io/${{ secrets.GCP_PROJECT }}/cio-service \
            --region us-central1 \
            --set-secrets "CUSTOMERIO_SITE_ID=cio-site-id:latest,\
              CUSTOMERIO_TRACK_API_KEY=cio-track-key:latest,\
              CUSTOMERIO_APP_API_KEY=cio-app-key:latest" \
            --set-env-vars "CUSTOMERIO_REGION=us,NODE_ENV=production" \
            --min-instances 1 \
            --max-instances 10 \
            --memory 512Mi \
            --cpu 1 \
            --allow-unauthenticated

Step 2: Health Check Endpoint

// routes/health.ts
import { TrackClient, RegionUS } from "customerio-node";
import { Router } from "express";

const router = Router();

router.get("/health", async (_req, res) => {
  const checks: Record<string, { status: string; latency_ms?: number }> = {};

  // Check Track API
  const cio = new TrackClient(
    process.env.CUSTOMERIO_SITE_ID!,
    process.env.CUSTOMERIO_TRACK_API_KEY!,
    { region: RegionUS }
  );

  const start = Date.now();
  try {
    await cio.identify("health-check", {
      email: "health@internal.example.com",
      _health_check: true,
    });
    checks.track_api = { status: "ok", latency_ms: Date.now() - start };
  } catch (err: any) {
    checks.track_api = { status: `error: ${err.statusCode}` };
  }

  const allOk = Object.values(checks).every((c) => c.status === "ok");

  res.status(allOk ? 200 : 503).json({
    status: allOk ? "healthy" : "degraded",
    checks,
    ve

Customer.io Hello World

Overview

Create a minimal working Customer.io integration: identify a user (create/update their profile), track an event, and send a transactional email. This covers the three fundamental Customer.io operations.

Prerequisites

• customerio-node installed (npm install customerio-node)
• CUSTOMERIOSITEID and CUSTOMERIOTRACKAPI_KEY configured
• CUSTOMERIOAPPAPI_KEY configured (for transactional email example)

Instructions

Step 1: Identify a User (Create/Update Profile)

// hello-customerio.ts
import { TrackClient, RegionUS } from "customerio-node";

const cio = new TrackClient(
  process.env.CUSTOMERIO_SITE_ID!,
  process.env.CUSTOMERIO_TRACK_API_KEY!,
  { region: RegionUS }
);

// identify() creates the user if they don't exist, or updates if they do.
// The first argument is your internal user ID (immutable — use DB primary key).
await cio.identify("user-123", {
  email: "hello@example.com",          // Required for email campaigns
  first_name: "Jane",
  last_name: "Doe",
  plan: "pro",
  created_at: Math.floor(Date.now() / 1000),  // Unix seconds, NOT milliseconds
});

console.log("User identified in Customer.io");

Key rules:

• id (first arg) should be your immutable database ID — never use email as ID
• email attribute is required if you want to send email campaigns
• created_at must be Unix timestamp in seconds (not ms) — Math.floor(Date.now() / 1000)
• All custom attributes are stored on the user profile and usable in segments + Liquid templates

Step 2: Track an Event

// Track a custom event on the user's activity timeline.
// Events trigger campaigns — the event name must match exactly in the dashboard.
await cio.track("user-123", {
  name: "signed_up",                   // snake_case, matches campaign trigger
  data: {
    signup_method: "google_oauth",
    referral_source: "product_hunt",
    timestamp: Math.floor(Date.now() / 1000),
  },
});

console.log("Event tracked in Customer.io");

Key rules:

• User must be identified before tracking events (call identify() first)
• Event name is case-sensitive and must match your campaign trigger exactly
• Use snakecase for event names — signedup, not Signed Up or signedUp
• data properties are accessible in Liquid templates as {{ event.property_name }}

Step 3: Track a

Customer.io Known Pitfalls

Overview

The 12 most common Customer.io integration mistakes, with the wrong pattern, the correct pattern, and why it matters. Use this as a code review checklist and developer onboarding reference.

The Pitfall Catalog

Pitfall 1: Wrong API Key Type

// WRONG — using Track API key for transactional messages
const api = new APIClient(process.env.CUSTOMERIO_TRACK_API_KEY!);
// Gets 401 because App API uses a DIFFERENT bearer token

// CORRECT — use the App API key
const api = new APIClient(process.env.CUSTOMERIO_APP_API_KEY!);

Why: Customer.io has two separate authentication systems. Track API uses Basic Auth (Site ID + Track Key). App API uses Bearer Auth (App Key). They are not interchangeable.

Pitfall 2: Millisecond Timestamps

// WRONG — JavaScript Date.now() returns milliseconds
await cio.identify("user-1", {
  created_at: Date.now(),           // 1704067200000 → year 55976
});

// CORRECT — Customer.io expects Unix seconds
await cio.identify("user-1", {
  created_at: Math.floor(Date.now() / 1000),  // 1704067200
});

Why: Customer.io accepts millisecond values without error but interprets them as seconds, resulting in dates thousands of years in the future. Segments using date comparisons silently break.

Pitfall 3: Track Before Identify

// WRONG — tracking before identifying creates orphaned events
await cio.track("new-user", { name: "signed_up", data: {} });
// User profile doesn't exist yet — event may be lost

// CORRECT — always identify first
await cio.identify("new-user", { email: "user@example.com" });
await cio.track("new-user", { name: "signed_up", data: {} });

Why: Track calls on non-existent users may be silently dropped. Always identify() before track().

Pitfall 4: Using Email as User ID

// WRONG — email can change, creating duplicate profiles
await cio.identify("user@example.com", { email: "user@example.com" });
// When user changes email, old profile orphaned, new one created

// CORRECT — use immutable database ID
await cio.identify("usr_abc123", {
  email: "user@example.com",    // Email as attribute, not ID
});

Why: The first argument to identify() is the permanent user ID. If you use email and the user changes it, you get two profiles. Use your database primary key instead.

Pitfall 5: Missing Email Attribute

// WRONG — user can't receive email campaigns
await cio.identify("user-1", {
  first_name: "Jane",
  plan: "p

Customer.io Load & Scale

Overview

Load testing and scaling strategies for high-volume Customer.io integrations: k6 load test scripts, scaling architecture selection based on volume tier, Kubernetes HPA autoscaling, message queue buffering, and rate-limit-aware batch processing.

Scaling Architecture by Volume

Customer.io rate limit is ~100 req/sec per workspace. Plan your architecture around this.

Instructions

Step 1: k6 Load Test Script

// load-tests/customerio.js
// Run: k6 run --vus 10 --duration 60s load-tests/customerio.js
import http from "k6/http";
import { check, sleep } from "k6";
import { Counter, Trend } from "k6/metrics";

const SITE_ID = __ENV.CUSTOMERIO_SITE_ID;
const API_KEY = __ENV.CUSTOMERIO_TRACK_API_KEY;
const BASE_URL = "https://track.customer.io/api/v1";
const AUTH = `${SITE_ID}:${API_KEY}`;

const identifyLatency = new Trend("cio_identify_latency");
const trackLatency = new Trend("cio_track_latency");
const errors = new Counter("cio_errors");

export const options = {
  scenarios: {
    identify_load: {
      executor: "ramping-arrival-rate",
      startRate: 10,
      timeUnit: "1s",
      preAllocatedVUs: 20,
      maxVUs: 50,
      stages: [
        { duration: "30s", target: 50 },   // Ramp to 50/sec
        { duration: "60s", target: 80 },   // Hold at 80/sec (near limit)
        { duration: "30s", target: 10 },   // Cool down
      ],
    },
  },
  thresholds: {
    cio_identify_latency: ["p(95)<500", "p(99)<2000"],
    cio_track_latency: ["p(95)<500", "p(99)<2000"],
    cio_errors: ["count<50"],
  },
};

export default function () {
  const userId = `k6-load-${__VU}-${__ITER}`;
  const headers = {
    "Content-Type": "application/json",
    Authorization: `Basic ${encoding.b64encode(AUTH)}`,
  };

  // Identify
  const identifyRes = http.put(
    `${BASE_URL}/customers/${userId}`,
    JSON.stringify({
      email: `${userId}@loadtest.example.com`,
      _load_test: true,
      created_at: Math.floor(Date.now() / 1000),
    }),
    { headers }
  );

  identifyLatency.add(identifyRes.timings.duration);
  check(identifyRes, { "identify 200": (r) => r.status === 200 }) || errors.add(1

Customer.io Local Dev Loop

Overview

Set up an efficient local development workflow for Customer.io: environment isolation via separate workspaces, a dry-run client for safe development, test mocks for unit tests, and prefixed events that never pollute production data.

Prerequisites

• customerio-node installed
• Separate Customer.io workspace for development (recommended — free workspaces available)
• dotenv or similar for environment variable loading

Instructions

Step 1: Environment Configuration

# .env.development
CUSTOMERIO_SITE_ID=dev-site-id
CUSTOMERIO_TRACK_API_KEY=dev-track-key
CUSTOMERIO_APP_API_KEY=dev-app-key
CUSTOMERIO_REGION=us
CUSTOMERIO_DRY_RUN=false
CUSTOMERIO_EVENT_PREFIX=dev_

# .env.test
CUSTOMERIO_SITE_ID=not-needed
CUSTOMERIO_TRACK_API_KEY=not-needed
CUSTOMERIO_APP_API_KEY=not-needed
CUSTOMERIO_DRY_RUN=true
CUSTOMERIO_EVENT_PREFIX=test_

Step 2: Environment-Aware Client

// lib/customerio-dev.ts
import { TrackClient, APIClient, RegionUS, RegionEU } from "customerio-node";

interface CioConfig {
  siteId: string;
  trackApiKey: string;
  appApiKey: string;
  region: typeof RegionUS | typeof RegionEU;
  dryRun: boolean;
  eventPrefix: string;
}

function loadConfig(): CioConfig {
  return {
    siteId: process.env.CUSTOMERIO_SITE_ID ?? "",
    trackApiKey: process.env.CUSTOMERIO_TRACK_API_KEY ?? "",
    appApiKey: process.env.CUSTOMERIO_APP_API_KEY ?? "",
    region: process.env.CUSTOMERIO_REGION === "eu" ? RegionEU : RegionUS,
    dryRun: process.env.CUSTOMERIO_DRY_RUN === "true",
    eventPrefix: process.env.CUSTOMERIO_EVENT_PREFIX ?? "",
  };
}

export class DevTrackClient {
  private client: TrackClient | null = null;
  private config: CioConfig;
  private log: typeof console.log;

  constructor() {
    this.config = loadConfig();
    this.log = console.log.bind(console);
    if (!this.config.dryRun) {
      this.client = new TrackClient(
        this.config.siteId,
        this.config.trackApiKey,
        { region: this.config.region }
      );
    }
  }

  async identify(userId: string, attributes: Record<string, any>) {
    const prefixedId = `${this.config.eventPrefix}${userId}`;
    if (this.config.dryRun) {
      this.log("[DRY RUN] identify:", prefixedId, attributes);
      return;
    }
    return this.client!.identify(prefixedId, attributes);
  }

  async track(userId: string, eventName: string, data?: Record<string, any>) {
    const prefixedId = `${this.config.eventPrefix}${userId}`;
    const prefixedEvent = `${this.config.eventPrefix}${eventName}`;
    if (this.config.dryRun) {
      this.log("[DRY RUN] track:", prefixedId, prefixedEvent, data);
      return;
    }
    return this.client!.track(prefixedId, {
      name: p

Customer.io Multi-Environment Setup

Overview

Configure isolated Customer.io environments for dev, staging, and production: separate workspaces per environment, typed configuration with validation, environment-aware client wrappers, Kubernetes ConfigMap overlays, and data isolation verification.

Prerequisites

• Customer.io account with multiple workspaces (create at fly.customer.io)
• Environment variable management (dotenv, secrets manager)
• CI/CD pipeline for per-environment deployment

Workspace Strategy

Each workspace has its own Site ID, Track API Key, and App API Key. Create workspaces at Settings > Workspace Settings.

Instructions

Step 1: Typed Environment Configuration

// config/customerio.ts
import { RegionUS, RegionEU } from "customerio-node";

type CioEnvironment = "development" | "ci" | "staging" | "production";

interface CioEnvConfig {
  siteId: string;
  trackApiKey: string;
  appApiKey: string;
  region: typeof RegionUS | typeof RegionEU;
  dryRun: boolean;
  logLevel: "debug" | "info" | "warn" | "error";
  eventPrefix: string;     // Prefix events in non-prod to prevent confusion
}

function validateConfig(config: CioEnvConfig, env: CioEnvironment): void {
  if (!config.siteId) throw new Error(`Missing CUSTOMERIO_SITE_ID for ${env}`);
  if (!config.trackApiKey) throw new Error(`Missing CUSTOMERIO_TRACK_API_KEY for ${env}`);
  if (env === "production" && config.dryRun) {
    throw new Error("Production cannot be in dry-run mode");
  }
  if (env === "production" && config.eventPrefix) {
    throw new Error("Production must not use event prefix");
  }
}

export function loadCioConfig(): CioEnvConfig {
  const env = (process.env.NODE_ENV ?? "development") as CioEnvironment;
  const region = process.env.CUSTOMERIO_REGION === "eu" ? RegionEU : RegionUS;

  const config: CioEnvConfig = {
    siteId: process.env.CUSTOMERIO_SITE_ID ?? "",
    trackApiKey: process.env.CUSTOMERIO_TRACK_API_KEY ?? "",
    appApiKey: process.env.CUSTOMERIO_APP_API_KEY ?? "&q

Customer.io Observability

Overview

Implement comprehensive observability for Customer.io integrations: Prometheus metrics (latency, error rates, delivery funnel), structured JSON logging with PII redaction, OpenTelemetry tracing, and Grafana dashboard definitions.

Prerequisites

• Customer.io integration deployed
• Prometheus + Grafana (or compatible metrics stack)
• Structured logging system (pino recommended)

Key Metrics to Track

Instructions

Step 1: Prometheus Metrics

// lib/customerio-metrics.ts
import { Counter, Histogram, Gauge, Registry } from "prom-client";

const registry = new Registry();

export const cioMetrics = {
  apiDuration: new Histogram({
    name: "cio_api_duration_ms",
    help: "Customer.io API call duration in milliseconds",
    labelNames: ["operation", "status"] as const,
    buckets: [10, 25, 50, 100, 250, 500, 1000, 2500, 5000],
    registers: [registry],
  }),

  apiRequests: new Counter({
    name: "cio_api_requests_total",
    help: "Total Customer.io API requests",
    labelNames: ["operation"] as const,
    registers: [registry],
  }),

  apiErrors: new Counter({
    name: "cio_api_errors_total",
    help: "Customer.io API errors",
    labelNames: ["operation", "status_code"] as const,
    registers: [registry],
  }),

  emailSent: new Counter({
    name: "cio_email_sent_total",
    help: "Emails sent via Customer.io",
    labelNames: ["type"] as const,  // "transactional" or "campaign"
    registers: [registry],
  }),

  emailBounced: new Counter({
    na

Customer.io Performance Tuning

Overview

Optimize Customer.io API performance for high-volume integrations: HTTP connection pooling, identify deduplication caching, event batching with flush control, fire-and-forget async tracking, and regional routing.

Prerequisites

• Working Customer.io integration
• Understanding of your traffic patterns and volume
• Monitoring to measure improvement (see customerio-observability)

Performance Targets

Instructions

Step 1: HTTP Connection Pooling

// lib/customerio-pooled.ts
import { TrackClient, RegionUS } from "customerio-node";
import https from "https";

// The customerio-node SDK creates new connections by default.
// Reuse connections with a keep-alive agent.
const agent = new https.Agent({
  keepAlive: true,
  maxSockets: 25,        // Max concurrent connections
  maxFreeSockets: 10,    // Keep idle connections open
  timeout: 30000,        // 30s socket timeout
  keepAliveMsecs: 15000, // TCP keep-alive probe interval
});

// Apply to the SDK by creating a singleton with the agent
// Note: customerio-node doesn't directly accept an agent,
// but we configure Node.js global agent for HTTPS
https.globalAgent = agent;

// Singleton client — one instance = one connection pool
const cio = new TrackClient(
  process.env.CUSTOMERIO_SITE_ID!,
  process.env.CUSTOMERIO_TRACK_API_KEY!,
  { region: RegionUS }
);

export { cio };

Step 2: Identify Deduplication Cache

// lib/customerio-dedup.ts
// Skip duplicate identify() calls within a time window

class LRUCache<K, V> {
  private map = new Map<K, V>();
  constructor(private maxSize: number) {}

  get(key: K): V | undefined {
    const val = this.map.get(key);
    if (val !== undefined) {
      // Move to end (most recent)
      this.map.delete(key);
      this.map.set(key, val);
    }
    return val;
  }

  set(key: K, val: V): void {
    this.map.delete(key);
    this.map.set(key, val);
    if (this.map.size > this.maxSize) {
      const oldest = this.map.keys().next().value;
      this.map.delete(oldest!);
    }
  }
}

import { createHash } from "crypto";
import { TrackClient, RegionUS } from "customerio-node";

Customer.io Primary Workflow

Overview

Implement Customer.io's core messaging workflow: identify users with segment-ready attributes, track lifecycle events that trigger campaigns, and set up the data layer for automated onboarding, nurture, and re-engagement sequences.

Prerequisites

• customerio-node configured with Track API credentials
• Campaigns created in Customer.io dashboard (triggered by events you define)
• Understanding of your user lifecycle stages

How Campaigns Work

Your App (SDK)           Customer.io Dashboard          User
─────────────           ────────────────────          ────
cio.identify(user)  →   Profile created/updated
cio.track("signed_up")  →   Campaign trigger fires
                             Wait 1 day             →   Welcome email
                             Check: verified?
                             ├─ No                  →   Verification reminder
                             └─ Yes → Wait 3 days   →   Feature tips email

Events tracked via the SDK trigger campaigns you build in the dashboard. The SDK sends the data; the dashboard defines the workflow logic.

Instructions

Step 1: Define Your Event Taxonomy

// lib/customerio-events.ts
import { TrackClient, RegionUS } from "customerio-node";

// Central event definitions — every event your app tracks
export const CIO_EVENTS = {
  // Onboarding
  SIGNED_UP: "signed_up",
  EMAIL_VERIFIED: "email_verified",
  PROFILE_COMPLETED: "profile_completed",
  FIRST_PROJECT_CREATED: "first_project_created",

  // Engagement
  FEATURE_USED: "feature_used",
  INVITED_TEAMMATE: "invited_teammate",
  UPGRADE_STARTED: "upgrade_started",
  UPGRADE_COMPLETED: "upgrade_completed",

  // Lifecycle
  SUBSCRIPTION_RENEWED: "subscription_renewed",
  SUBSCRIPTION_CANCELLED: "subscription_cancelled",
  TRIAL_EXPIRING: "trial_expiring",

  // Commerce
  CHECKOUT_STARTED: "checkout_started",
  CHECKOUT_COMPLETED: "checkout_completed",
  REFUND_REQUESTED: "refund_requested",
} as const;

type EventName = (typeof CIO_EVENTS)[keyof typeof CIO_EVENTS];

Step 2: Build the Messaging Service

// services/customerio-messaging.ts
import { TrackClient, RegionUS } from "customerio-node";
import { CIO_EVENTS } from "../lib/customerio-events";

const cio = new TrackClient(
  process.env.CUSTOMERIO_SITE_ID!,
  process.env.CUSTOMERIO_TRACK_API_KEY!,
  { region: RegionUS }
);

interface UserProfile {
  id: string;
  email: string;
  firstName: string;
  lastName?: string;
  plan: string;
  companyName?: string;
}

export class MessagingService {
  /** Call on user signup — 

Customer.io Production Checklist

Overview

Comprehensive go-live checklist for Customer.io integrations: credentials audit, integration quality review, email deliverability setup, monitoring configuration, smoke tests, and staged rollout plan.

Prerequisites

• Customer.io integration complete and tested in staging
• Production workspace credentials ready
• Sending domain configured and verified

Production Checklist

1. Credentials & Configuration

2. Integration Quality

// scripts/prod-audit.ts
import { TrackClient, RegionUS } from "customerio-node";

async function auditIntegration() {
  const checks: { name: string; pass: boolean; detail: string }[] = [];

  // Check 1: Credentials exist
  const siteId = process.env.CUSTOMERIO_SITE_ID;
  const trackKey = process.env.CUSTOMERIO_TRACK_API_KEY;
  const appKey = process.env.CUSTOMERIO_APP_API_KEY;
  checks.push({
    name: "Track credentials",
    pass: !!(siteId && trackKey),
    detail: siteId ? `Site ID: ${siteId.substring(0, 4)}...` : "MISSING",
  });
  checks.push({
    name: "App API credential",
    pass: !!appKey,
    detail: appKey ? "Set" : "MISSING (needed for transactional)",
  });

  // Check 2: API connectivity
  if (siteId && trackKey) {
    const cio = new TrackClient(siteId, trackKey, { region: RegionUS });
    try {
      await cio.identify("prod-audit-test", {
        email: "prod-audit@example.com",
      });
      await cio.suppress("prod-audit-test");
      checks.push({ name: "Track API", pass: true, detail: "Connected" });
    } catch (err: any) {
      checks.push({
        name: "Track API",
        pass: false,
        detail: `${err.statusCode}: ${err.message}`,
      });
    }
  }

  // Report
  console.log("\n=== Customer.io Production Audit ===\n");
  for (const check of checks) {
    const icon = check.pass ? "PASS" : "FAIL";
    console.log(`[${icon}] ${check.name}: ${check

Customer.io Rate Limits

Overview

Understand Customer.io's API rate limits and implement proper throttling: token bucket limiters, exponential backoff with jitter, queue-based processing, and 429 response handling.

Rate Limit Reference

These are approximate. Customer.io uses sliding window rate limiting. When exceeded, you get a 429 Too Many Requests response.

Instructions

Step 1: Token Bucket Rate Limiter

// lib/rate-limiter.ts
export class TokenBucket {
  private tokens: number;
  private lastRefill: number;

  constructor(
    private readonly maxTokens: number = 80,  // Stay under 100/sec limit
    private readonly refillRate: number = 80   // Tokens per second
  ) {
    this.tokens = maxTokens;
    this.lastRefill = Date.now();
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.maxTokens, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens >= 1) {
      this.tokens -= 1;
      return;
    }
    // Wait until a token is available
    const waitMs = ((1 - this.tokens) / this.refillRate) * 1000;
    await new Promise((r) => setTimeout(r, Math.ceil(waitMs)));
    this.tokens = 0;
    this.lastRefill = Date.now();
  }
}

Step 2: Exponential Backoff with Jitter

// lib/backoff.ts
interface BackoffOptions {
  maxRetries: number;
  baseDelayMs: number;
  maxDelayMs: number;
  jitter: number;         // 0 to 1
}

const DEFAULTS: BackoffOptions = {
  maxRetries: 4,
  baseDelayMs: 1000,
  maxDelayMs: 60000,
  jitter: 0.25,
};

export async function withBackoff<T>(
  fn: () => Promise<T>,
  opts: Partial<BackoffOptions> = {}
): Promise<T> {
  const { maxRetries, baseDelayMs, maxDelayMs, jitter } = { ...DEFAULTS, ...opts };
  let lastErr: Error | undefined;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err: any) {
      lastErr = err;
      const status = err.statusCode ?? err.status;

      // Don't retry 4xx errors (except 429)
      if (status >= 400 && status < 500 && status !== 429) throw err;

      if (a

Customer.io Reference Architecture

Overview

Enterprise-grade reference architecture for Customer.io: a service layer separating Track and App API concerns, event-driven processing with message queues, repository pattern for user-to-CIO sync, webhook event bus, and infrastructure as code.

Architecture Principles

1. Two Clients, Two Concerns — TrackClient for behavioral data in, APIClient for messages out
2. Event-Driven — Message queues decouple your app from Customer.io API availability
3. Idempotent Operations — All writes safely retryable via content hashing
4. Service Layer — Business logic never calls Customer.io SDK directly
5. Observability — Every operation emits timing and error metrics

Architecture Diagram

┌─────────────┐    ┌───────────────────┐    ┌──────────────┐
│ Application │───>│ MessagingService  │───>│ Track API    │
│ Routes      │    │ (service layer)   │    │ identify()   │
└─────────────┘    │                   │    │ track()      │
                   │ - identify users  │    └──────────────┘
                   │ - track events    │
                   │ - send txn emails │    ┌──────────────┐
                   │                   │───>│ App API      │
                   └───────────────────┘    │ sendEmail()  │
                          │                 │ broadcast()  │
                          │                 └──────────────┘
                          v
                   ┌───────────────────┐
                   │ Event Queue       │    ┌──────────────┐
                   │ (Redis/Kafka)     │───>│ DLQ          │
                   │ for reliability   │    │ (failures)   │
                   └───────────────────┘    └──────────────┘

┌─────────────┐    ┌───────────────────┐    ┌──────────────┐
│ Customer.io │───>│ Webhook Handler   │───>│ BigQuery     │
│ Webhooks    │    │ HMAC verification │    │ (analytics)  │
└─────────────┘    │ Event routing     │    └──────────────┘

Instructions

Step 1: Core Service Layer

// services/messaging-service.ts
import { EventEmitter } from "events";
import { TrackClient, APIClient, SendEmailRequest, RegionUS, RegionEU } from "customerio-node";

interface MessagingConfig {
  siteId: string;
  trackApiKey: string;
  appApiKey: string;
  region: "us" | "eu";
}

export class MessagingService extends EventEmitter {
  private track: TrackClient;
  private app: APIClient;

  constructor(config: MessagingConfig) {
    super();
    const region = config.region === "eu" ? RegionEU : RegionUS;
    this.track = new TrackClient(config.siteId, config.trackApiKey, { region });
    this.app = new APIClient(config.appApiKey, { region });
  }

  async identifyUser(use

Customer.io Reliability Patterns

Overview

Implement fault-tolerant Customer.io integrations: circuit breaker (stop cascading failures), retry with jitter (handle transient errors), fallback queue (survive outages), idempotency guard (prevent duplicates), and graceful degradation (never crash your app for analytics).

Prerequisites

• Working Customer.io integration
• Understanding of failure modes (429, 5xx, timeouts, DNS failures)
• Redis (recommended for queue-based patterns)

Instructions

Pattern 1: Circuit Breaker

// lib/circuit-breaker.ts
type CircuitState = "CLOSED" | "OPEN" | "HALF_OPEN";

export class CircuitBreaker {
  private state: CircuitState = "CLOSED";
  private failureCount = 0;
  private successCount = 0;
  private lastFailureTime = 0;

  constructor(
    private readonly failureThreshold: number = 5,
    private readonly successThreshold: number = 3,
    private readonly resetTimeoutMs: number = 30000
  ) {}

  get currentState(): CircuitState {
    if (this.state === "OPEN") {
      // Check if enough time has passed to try again
      if (Date.now() - this.lastFailureTime > this.resetTimeoutMs) {
        this.state = "HALF_OPEN";
        this.successCount = 0;
      }
    }
    return this.state;
  }

  async execute<T>(fn: () => Promise<T>): Promise<T> {
    if (this.currentState === "OPEN") {
      throw new Error("Circuit breaker is OPEN — Customer.io calls blocked");
    }

    try {
      const result = await fn();
      this.onSuccess();
      return result;
    } catch (err) {
      this.onFailure();
      throw err;
    }
  }

  private onSuccess(): void {
    this.failureCount = 0;
    if (this.state === "HALF_OPEN") {
      this.successCount++;
      if (this.successCount >= this.successThreshold) {
        this.state = "CLOSED";
        console.log("Circuit breaker: CLOSED (recovered)");
      }
    }
  }

  private onFailure(): void {
    this.failureCount++;
    this.lastFailureTime = Date.now();
    if (this.failureCount >= this.failureThreshold) {
      this.state = "OPEN";
      console.warn(
        `Circuit breaker: OPEN (${this.failureCount} failures). ` +
        `Will retry in ${this.resetTimeoutMs / 1000}s`
      );
    }
  }

  getStatus(): { state: CircuitState; failures: number; lastFailure: Date | null } {
    return {
      state: this.currentState,
      failures: this.failureCount,
      lastFailure: this.lastFailureTime ? new Date(this.lastFailureTime) : null,
    };
  }
}

Pattern 2: Retry with Jitter

// lib/retry.ts
export async function retryWithJitter<T>(
  fn: () => Promise<T>,
  maxRetries = 3,
  baseDelayMs = 1000
): Promise<T> {
  for (let attempt = 0;

Customer.io SDK Patterns

Overview

Production-ready patterns for customerio-node: type-safe wrappers with enum-constrained events, retry with exponential backoff, event batching for high-volume scenarios, and singleton lifecycle management.

Prerequisites

• customerio-node installed
• TypeScript project (recommended for type-safe patterns)
• Understanding of your event taxonomy

Instructions

Pattern 1: Type-Safe Client Wrapper

// lib/customerio-typed.ts
import { TrackClient, RegionUS, RegionEU } from "customerio-node";

// Define your event taxonomy as a union type
type CioEvent =
  | { name: "signed_up"; data: { method: string; source?: string } }
  | { name: "plan_changed"; data: { from: string; to: string; mrr: number } }
  | { name: "feature_used"; data: { feature: string; duration_ms?: number } }
  | { name: "checkout_completed"; data: { order_id: string; total: number; items: number } }
  | { name: "subscription_cancelled"; data: { reason: string; feedback?: string } };

// Define user attributes with strict types
interface CioUserAttributes {
  email: string;
  first_name?: string;
  last_name?: string;
  plan?: "free" | "starter" | "pro" | "enterprise";
  company?: string;
  created_at?: number;       // Unix seconds
  last_seen_at?: number;     // Unix seconds
  [key: string]: unknown;    // Allow additional attributes
}

export class TypedCioClient {
  private client: TrackClient;

  constructor(siteId: string, apiKey: string, region: "us" | "eu" = "us") {
    this.client = new TrackClient(siteId, apiKey, {
      region: region === "eu" ? RegionEU : RegionUS,
    });
  }

  async identify(userId: string, attributes: CioUserAttributes): Promise<void> {
    await this.client.identify(userId, {
      ...attributes,
      last_seen_at: Math.floor(Date.now() / 1000),
    });
  }

  async track(userId: string, event: CioEvent): Promise<void> {
    await this.client.track(userId, {
      name: event.name,
      data: { ...event.data, tracked_at: Math.floor(Date.now() / 1000) },
    });
  }

  async suppress(userId: string): Promise<void> {
    await this.client.suppress(userId);
  }

  async destroy(userId: string): Promise<void> {
    await this.client.destroy(userId);
  }
}

Pattern 2: Retry with Exponential Backoff

// lib/customerio-retry.ts
interface RetryOptions {
  maxRetries: number;
  baseDelayMs: number;
  maxDelayMs: number;
  jitterFactor: number;  // 0 to 1
}

const DEFAULT_RETRY: RetryOptions = {
  maxRetries: 3,
  baseDelayMs: 1000,
  maxDelayMs: 30000,
  jitterFactor: 0.3,
};

async function withRetry<T>(
  fn: () => Promise<T>,
  opts: RetryOptions = DEFAUL

Customer.io Security Basics

Overview

Implement security best practices for Customer.io: secrets management for API credentials, PII sanitization before sending data, webhook signature verification (HMAC-SHA256), API key rotation, and GDPR/CCPA data deletion compliance.

Prerequisites

• Customer.io account with admin access
• Understanding of your data classification (what is PII)
• Secrets management system (recommended for production)

Instructions

Step 1: Secure Credential Storage

// lib/customerio-secrets.ts
// NEVER hardcode credentials — use environment variables or a secrets manager

// Option A: Environment variables (acceptable for most apps)
const siteId = process.env.CUSTOMERIO_SITE_ID;
const trackKey = process.env.CUSTOMERIO_TRACK_API_KEY;

// Option B: GCP Secret Manager (recommended for production)
import { SecretManagerServiceClient } from "@google-cloud/secret-manager";

const secretClient = new SecretManagerServiceClient();

async function getSecret(name: string): Promise<string> {
  const [version] = await secretClient.accessSecretVersion({
    name: `projects/my-project/secrets/${name}/versions/latest`,
  });
  return version.payload?.data?.toString() ?? "";
}

async function createCioClient() {
  const [siteId, trackKey] = await Promise.all([
    getSecret("customerio-site-id"),
    getSecret("customerio-track-api-key"),
  ]);

  return new TrackClient(siteId, trackKey, { region: RegionUS });
}

Step 2: PII Sanitization

// lib/customerio-sanitize.ts
// Sanitize user data BEFORE sending to Customer.io

const NEVER_SEND = new Set([
  "ssn", "social_security", "tax_id",
  "credit_card", "card_number", "cvv",
  "password", "password_hash",
  "bank_account", "routing_number",
]);

const HASH_FIELDS = new Set([
  "phone", "phone_number",
  "ip_address", "ip",
  "address", "street_address",
]);

import { createHash } from "crypto";

function hashValue(value: string): string {
  return createHash("sha256").update(value).digest("hex").substring(0, 16);
}

export function sanitizeAttributes(
  attrs: Record<string, any>
): Record<string, any> {
  const clean: Record<string, any> = {};

  for (const [key, value] of Object.entries(attrs)) {
    const lowerKey = key.toLowerCase();

    // Strip highly sensitive fields entirely
    if (NEVER_SEND.has(lowerKey)) continue;

    // Hash PII fields
    if (HASH_FIELDS.has(lowerKey) && typeof value === "string") {
      clean[`${key}_hash`] = hashValue(value);
      continue;
    }

    clean[key] = value;
  }

  return clean;
}

// Usage
import { TrackClient

Customer.io Upgrade & Migration

Current State

!npm list customerio-node 2>/dev/null | grep customerio || echo 'customerio-node: not installed'

!npm view customerio-node version 2>/dev/null || echo 'Cannot check latest version'

Overview

Plan and execute customerio-node SDK upgrades safely: assess current version, review breaking changes, apply code migrations, and validate with staged rollout.

Prerequisites

• Current SDK version identified (npm list customerio-node)
• Test environment available
• Version control for rollback

Major Version Migration Reference

Legacy CustomerIO to Modern TrackClient + APIClient

Older versions of customerio-node used a single CustomerIO class. Modern versions split into TrackClient (tracking) and APIClient (transactional/broadcasts).

// BEFORE — Legacy pattern (customerio-node < 2.x)
const CustomerIO = require("customerio-node");
const cio = new CustomerIO(siteId, apiKey);
cio.identify("user-1", { email: "user@example.com" });
cio.track("user-1", { name: "event_name" });

// AFTER — Modern pattern (customerio-node >= 2.x)
import { TrackClient, APIClient, RegionUS } from "customerio-node";

const cio = new TrackClient(siteId, apiKey, { region: RegionUS });
await cio.identify("user-1", { email: "user@example.com" });
await cio.track("user-1", { name: "event_name", data: {} });

const api = new APIClient(appApiKey, { region: RegionUS });
await api.sendEmail(request);

Key changes:

• TrackClient replaces CustomerIO for identify/track
• APIClient is new — handles transactional + broadcasts
• Region is now explicit (RegionUS or RegionEU)
• Methods return Promises (must await)
• Event tracking uses { name, data } object instead of positional args

Instructions

Step 1: Assess Current Version

// scripts/cio-version-check.ts
import { readFileSync, existsSync } from "fs";

function assessVersion() {
  // Check installed version
  const lockPath = "package-lock.json";
  if (existsSync(lockPath)) {
    const lock = JSON.parse(readFileSync(lockPath, "utf-8"));
    const installed =
      lock.packages?.["node_modules/customerio-node"]?.version ??
      lock.dependencies?.["customerio-node"]?.version ??
      "not found in lockfile";
    console.log(`Installed: customerio-node@${installed}`);
  }

  // Check package.json declared version
  const pkg = JSON.pars

Customer.io Webhooks & Events

Overview

Implement Customer.io reporting webhook handling: receive real-time delivery events (sent, delivered, opened, clicked, bounced, complained, unsubscribed), verify HMAC-SHA256 signatures, process events reliably with queuing, and stream to a data warehouse.

How Reporting Webhooks Work

Customer.io                    Your Server                   Data Warehouse
──────────                    ───────────                   ──────────────
Email sent   →  POST /webhooks/cio  →  Verify signature
Email opened →  POST /webhooks/cio  →  Parse event type
Link clicked →  POST /webhooks/cio  →  Route to handler  →  INSERT INTO events
Email bounced → POST /webhooks/cio  →  Suppress user

Configure at: Data & Integrations > Integrations > Reporting Webhooks

Prerequisites

• Public HTTPS endpoint for webhook receiver
• Webhook signing key from Customer.io dashboard
• Express or similar HTTP framework

Instructions

Step 1: Define Webhook Event Types

// types/customerio-webhooks.ts

// Customer.io reporting webhook event metrics
type CioMetric =
  | "sent"          // Message sent to delivery provider
  | "delivered"     // Delivery provider confirmed receipt
  | "opened"        // Recipient opened the email
  | "clicked"       // Recipient clicked a link
  | "converted"     // Recipient completed a conversion goal
  | "bounced"       // Email bounced (hard or soft)
  | "spammed"       // Recipient marked as spam
  | "unsubscribed"  // Recipient unsubscribed
  | "dropped"       // Message dropped (suppressed, invalid)
  | "deferred"      // Delivery temporarily deferred
  | "failed";       // Delivery failed

interface CioWebhookEvent {
  // Event metadata
  event_id: string;
  metric: CioMetric;
  timestamp: number;          // Unix seconds

  // Recipient info
  customer_id: string;
  email_address?: string;

  // Message info
  subject?: string;
  template_id?: number;
  campaign_id?: number;
  broadcast_id?: number;
  action_id?: number;

  // Delivery details
  delivery_id?: string;

  // Link tracking (for "clicked" events)
  href?: string;
  link_id?: number;
}

Step 2: Webhook Handler with Signature Verification

// routes/webhooks/customerio.ts
import { createHmac, timingSafeEqual } from "crypto";
import { Router, Request, Response } from "express";

const WEBHOOK_SECRET = process.env.CUSTOMERIO_WEBHOOK_SECRET!;

function verifySignature(rawBody: Buffer, signature: string): boolean {
  const expected = createHmac("sha256", WEBHOOK_SECRET)
    .update(rawBody)
    .digest("hex");

  try {
    return timingSafeEqual(
      Buffer.from(signature, "utf