Apify Webhooks & Events

Overview

Securely handle Apify webhooks with signature validation and replay protection.

Prerequisites

• Apify webhook secret configured
• HTTPS endpoint accessible from internet
• Understanding of cryptographic signatures
• Redis or database for idempotency (optional)

Webhook Endpoint Setup

Express.js

import express from 'express';
import crypto from 'crypto';

const app = express();

// IMPORTANT: Raw body needed for signature verification
app.post('/webhooks/apify',
  express.raw({ type: 'application/json' }),
  async (req, res) => {
    const signature = req.headers['x-apify-signature'] as string;
    const timestamp = req.headers['x-apify-timestamp'] as string;

    if (!verifyApifySignature(req.body, signature, timestamp)) {
      return res.status(401).json({ error: 'Invalid signature' });
    }

    const event = JSON.parse(req.body.toString());
    await handleApifyEvent(event);

    res.status(200).json({ received: true });
  }
);

Signature Verification

function verifyApifySignature(
  payload: Buffer,
  signature: string,
  timestamp: string
): boolean {
  const secret = process.env.APIFY_WEBHOOK_SECRET!;

  // Reject old timestamps (replay attack protection)
  const timestampAge = Date.now() - parseInt(timestamp) * 1000;
  if (timestampAge > 300000) { // 5 minutes
    console.error('Webhook timestamp too old');
    return false;
  }

  // Compute expected signature
  const signedPayload = `${timestamp}.${payload.toString()}`;
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');

  // Timing-safe comparison
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expectedSignature)
  );
}

Event Handler Pattern

type ApifyEventType = 'resource.created' | 'resource.updated' | 'resource.deleted';

interface ApifyEvent {
  id: string;
  type: ApifyEventType;
  data: Record<string, any>;
  created: string;
}

const eventHandlers: Record<ApifyEventType, (data: any) => Promise<void>> = {
  'resource.created': async (data) => { /* handle */ },
  'resource.updated': async (data) => { /* handle */ },
  'resource.deleted': async (data) => { /* handle */ }
};

async function handleApifyEvent(event: ApifyEvent): Promise<void> {
  const handler = eventHandlers[event.type];

  if (!handler) {
    console.log(`Unhandled event type: ${event.type}`);
    return;
  }

  try {
    await handler(event.data);
    console.log(`Processed ${event.type}: ${event.id}`);
  } catch (error) {
    console.error(`Failed to process ${event.type}: ${event.id}`, error);
    throw error; // Rethrow to trigger retry
  }
}

Idempotency Handling

import { Redis } from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

async function isEventProcessed(eventId: string): Promise<boolean> {
  const key = `apify:event:${eventId}`;
  const exists = await redis.exists(key);
  return exists === 1;
}

async function markEventProcessed(eventId: string): Promise<void> {
  const key = `apify:event:${eventId}`;
  await redis.set(key, '1', 'EX', 86400 * 7); // 7 days TTL
}

Webhook Testing

# Use Apify CLI to send test events
apify webhooks trigger resource.created --url http://localhost:3000/webhooks/apify

# Or use webhook.site for debugging
curl -X POST https://webhook.site/your-uuid \
  -H "Content-Type: application/json" \
  -d '{"type": "resource.created", "data": {}}'

Instructions

Step 1: Register Webhook Endpoint

Configure your webhook URL in the Apify dashboard.

Step 2: Implement Signature Verification

Use the signature verification code to validate incoming webhooks.

Step 3: Handle Events

Implement handlers for each event type your application needs.

Step 4: Add Idempotency

Prevent duplicate processing with event ID tracking.

Output

• Secure webhook endpoint
• Signature validation enabled
• Event handlers implemented
• Replay attack protection active

Error Handling

Examples

Testing Webhooks Locally

# Use ngrok to expose local server
ngrok http 3000

# Send test webhook
curl -X POST https://your-ngrok-url/webhooks/apify \
  -H "Content-Type: application/json" \
  -d '{"type": "test", "data": {}}'

Resources

• Apify Webhooks Guide
• Webhook Security Best Practices

Next Steps

For performance optimization, see apify-performance-tuning.