Fly.io Events & Monitoring

Overview

Fly.io does not have traditional webhooks. Instead, monitor machine state changes via the Machines API, process structured logs via fly logs, and use health check endpoints for automated responses.

Prerequisites

• A scoped monitoring identity, approved app/region policy, event ledger, and an incident owner.
• Redaction controls for logs and synthetic fixtures for state-change, health, and duplicate-event testing.

Output

Return an event-processing receipt with opaque machine/event ID, monitor version, idempotency result, aggregate health state, notification destination, and redacted error category. Do not include tokens, log bodies, or user data.

Error Handling

• Reject unknown app/region events and quarantine unexpected schemas or notification destinations.
• Bound polling/retry, deduplicate state transitions, and pause automated actions on access or health anomalies.
• Preserve only redacted incident evidence and use the rollback path before replaying actions.

Examples

Send a synthetic machine state transition twice. The monitor records the first once, marks the second duplicate, and rejects an event from an unapproved app without sending its body to any notification channel.

Instructions

Step 1: Poll Machine State Changes

// Monitor machine state transitions via Machines API
async function watchMachines(appName: string, callback: (event: MachineEvent) => void) {
  const client = new FlyClient(appName, process.env.FLY_API_TOKEN!);
  const stateCache = new Map<string, string>();

  setInterval(async () => {
    const machines = await client.listMachines();
    for (const m of machines) {
      const prev = stateCache.get(m.id);
      if (prev && prev !== m.state) {
        callback({
          machineId: m.id,
          region: m.region,
          previousState: prev,
          currentState: m.state,
          timestamp: new Date(),
        });
      }
      stateCache.set(m.id, m.state);
    }
  }, 10_000);  // Check every 10 seconds
}

interface MachineEvent {
  machineId: string;
  region: string;
  previousState: string;
  currentState: string;
  timestamp: Date;
}

Step 2: Health Check Event Handler

// Implement health check that reports machine health
// Fly.io uses this to auto-restart unhealthy machines

import express from 'express';
const app = express();

app.get('/health', async (req, res) => {
  const checks = {
    database: await checkPostgres(),
    redis: await checkRedis(),
    memory: process.memoryUsage().heapUsed < 500 * 1024 * 1024,  // < 500MB
  };

  const healthy = Object.values(checks).every(Boolean);
  res.status(healthy ? 200 : 503).json({
    status: healthy ? 'healthy' : 'unhealthy',
    region: process.env.FLY_REGION,
    machine: process.env.FLY_MACHINE_ID,
    checks,
  });
});

Step 3: Structured Log Processing

# Stream logs and process with jq
fly logs -a my-app --json | jq -c 'select(.level == "error")' | while read -r line; do
  echo "$line" >> errors.jsonl
  # Send to Slack, PagerDuty, etc.
done

# Search recent logs for specific patterns
fly logs -a my-app --no-tail | grep -i "error\|crash\|oom"

Step 4: Deployment Event Notifications

# Post-deploy notification in CI
fly deploy -a my-app && \
curl -X POST "$SLACK_WEBHOOK_URL" \
  -H "Content-Type: application/json" \
  -d "{\"text\": \"Deployed my-app to Fly.io. Status: $(fly status -a my-app --json | jq -r '.Status')\"}"

Resources

• Machines API
• Health Checks
• Fly Logs

Next Steps

For performance optimization, see flyio-performance-tuning.