apify-ci-integration SKILL.md View full skill →

Configure Apify CI/CD integration with GitHub Actions and testing.

ReadWriteEditBash(gh:*)

Apify CI Integration

Overview

Set up CI/CD pipelines for Apify integrations with automated testing.

Prerequisites

GitHub repository with Actions enabled
Apify test API key
npm/pnpm project configured

Instructions

Step 1: Create GitHub Actions Workflow

Create .github/workflows/apify-integration.yml:


name: Apify Integration Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  APIFY_API_KEY: ${{ secrets.APIFY_API_KEY }}

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      APIFY_API_KEY: ${{ secrets.APIFY_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test -- --coverage
      - run: npm run test:integration

Step 2: Configure Secrets


gh secret set APIFY_API_KEY --body "sk_test_***"

Step 3: Add Integration Tests


describe('Apify Integration', () => {
  it.skipIf(!process.env.APIFY_API_KEY)('should connect', async () => {
    const client = getApifyClient();
    const result = await client.healthCheck();
    expect(result.status).toBe('ok');
  });
});

Output

Automated test pipeline
PR checks configured
Coverage reports uploaded
Release workflow ready

Error Handling

Issue	Cause	Solution
Secret not found	Missing configuration	Add secret via `gh secret set`
Tests timeout	Network issues	Increase timeout or mock
Auth failures	Invalid key	Check secret value

Examples

Release Workflow


on:
  push:
    tags: ['v*']

jobs:
  release:
    runs-on: ubuntu-latest
    env:
      APIFY_API_KEY: ${{ secrets.APIFY_API_KEY_PROD }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - name: Verify Apify production readiness
        run: npm run test:integration
      - run: npm run build
      - run: npm publish

Branch Protection


required_status_checks:
  - "test"
  - "apify-integration"

Resources

Next Steps

For deployment patterns, see apify-deploy-integration.

apify-common-errors SKILL.md View full skill →

Diagnose and fix Apify common errors and exceptions.

ReadGrepBash(curl:*)

apify-core-workflow-a SKILL.md View full skill →

Execute Apify primary workflow: Core Workflow A.

ReadWriteEditBash(npm:*)Grep

apify-core-workflow-b SKILL.md View full skill →

Execute Apify secondary workflow: Core Workflow B.

ReadWriteEditBash(npm:*)Grep

apify-cost-tuning SKILL.md View full skill →

Optimize Apify costs through tier selection, sampling, and usage monitoring.

ReadGrep

Apify Cost Tuning

Overview

Optimize Apify costs through smart tier selection, sampling, and usage monitoring.

Prerequisites

Access to Apify billing dashboard
Understanding of current usage patterns
Database for usage tracking (optional)
Alerting system configured (optional)

Pricing Tiers

Tier	Monthly Cost	Included	Overage
Free	$0	1,000 requests	N/A
Pro	$99	100,000 requests	$0.001/request
Enterprise	Custom	Unlimited	Volume discounts

Cost Estimation


interface UsageEstimate {
  requestsPerMonth: number;
  tier: string;
  estimatedCost: number;
  recommendation?: string;
}

function estimateApifyCost(requestsPerMonth: number): UsageEstimate {
  if (requestsPerMonth <= 1000) {
    return { requestsPerMonth, tier: 'Free', estimatedCost: 0 };
  }

  if (requestsPerMonth <= 100000) {
    return { requestsPerMonth, tier: 'Pro', estimatedCost: 99 };
  }

  const proOverage = (requestsPerMonth - 100000) * 0.001;
  const proCost = 99 + proOverage;

  return {
    requestsPerMonth,
    tier: 'Pro (with overage)',
    estimatedCost: proCost,
    recommendation: proCost > 500
      ? 'Consider Enterprise tier for volume discounts'
      : undefined,
  };
}

Usage Monitoring


class ApifyUsageMonitor {
  private requestCount = 0;
  private bytesTransferred = 0;
  private alertThreshold: number;

  constructor(monthlyBudget: number) {
    this.alertThreshold = monthlyBudget * 0.8; // 80% warning
  }

  track(request: { bytes: number }) {
    this.requestCount++;
    this.bytesTransferred += request.bytes;

    if (this.estimatedCost() > this.alertThreshold) {
      this.sendAlert('Approaching Apify budget limit');
    }
  }

  estimatedCost(): number {
    return estimateApifyCost(this.requestCount).estimatedCost;
  }

  private sendAlert(message: string) {
    // Send to Slack, email, PagerDuty, etc.
  }
}

Cost Reduction Strategies

Step 1: Request Sampling


function shouldSample(samplingRate = 0.1): boolean {
  return Math.random() < samplingRate;
}

// Use for non-critical telemetry
if (shouldSample(0.1)) { // 10% sample
  await apifyClient.trackEvent(event);
}

Step 2: Batching Requests


// Instead of N individual calls
await Promise.all(ids.map(id => apifyClient.get(id)));

// Use batch endpoint (1 call)
await apifyClient.batchGet(ids);

Step 3: Caching (from P16)

Cache frequently accessed data
Use cache invalidation webhooks

apify-debug-bundle SKILL.md View full skill →

Collect Apify debug evidence for support tickets and troubleshooting.

ReadBash(grep:*)Bash(curl:*)Bash(tar:*)Grep

Apify Debug Bundle

Overview

Collect all necessary diagnostic information for Apify support tickets.

Prerequisites

Apify SDK installed
Access to application logs
Permission to collect environment info

Instructions

Step 1: Create Debug Bundle Script


#!/bin/bash
# apify-debug-bundle.sh

BUNDLE_DIR="apify-debug-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BUNDLE_DIR"

echo "=== Apify Debug Bundle ===" > "$BUNDLE_DIR/summary.txt"
echo "Generated: $(date)" >> "$BUNDLE_DIR/summary.txt"

Step 2: Collect Environment Info


# Environment info
echo "--- Environment ---" >> "$BUNDLE_DIR/summary.txt"
node --version >> "$BUNDLE_DIR/summary.txt" 2>&1
npm --version >> "$BUNDLE_DIR/summary.txt" 2>&1
echo "APIFY_API_KEY: ${APIFY_API_KEY:+[SET]}" >> "$BUNDLE_DIR/summary.txt"

Step 3: Gather SDK and Logs


# SDK version
npm list @apify/sdk 2>/dev/null >> "$BUNDLE_DIR/summary.txt"

# Recent logs (redacted)
grep -i "apify" ~/.npm/_logs/*.log 2>/dev/null | tail -50 >> "$BUNDLE_DIR/logs.txt"

# Configuration (redacted - secrets masked)
echo "--- Config (redacted) ---" >> "$BUNDLE_DIR/summary.txt"
cat .env 2>/dev/null | sed 's/=.*/=***REDACTED***/' >> "$BUNDLE_DIR/config-redacted.txt"

# Network connectivity test
echo "--- Network Test ---" >> "$BUNDLE_DIR/summary.txt"
echo -n "API Health: " >> "$BUNDLE_DIR/summary.txt"
curl -s -o /dev/null -w "%{http_code}" https://api.apify.com/health >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

Step 4: Package Bundle


tar -czf "$BUNDLE_DIR.tar.gz" "$BUNDLE_DIR"
echo "Bundle created: $BUNDLE_DIR.tar.gz"

Output

apify-debug-YYYYMMDD-HHMMSS.tar.gz archive containing:
summary.txt - Environment and SDK info
logs.txt - Recent redacted logs
config-redacted.txt - Configuration (secrets removed)

Error Handling

Item Purpose Included

Environment versions Compatibility check ✓

SDK version Version-specific bugs ✓

Error logs (redacted) Root cause analysis ✓

Config (redacted) Configuration issues ✓

Network test

Connectivi

apify-deploy-integration SKILL.md View full skill →

Deploy Apify integrations to Vercel, Fly.

ReadWriteEditBash(vercel:*)Bash(fly:*)Bash(gcloud:*)

Apify Deploy Integration

Overview

Deploy Apify-powered applications to popular platforms with proper secrets management.

Prerequisites

Apify API keys for production environment
Platform CLI installed (vercel, fly, or gcloud)
Application code ready for deployment
Environment variables documented

Vercel Deployment

Environment Setup


# Add Apify secrets to Vercel
vercel secrets add apify_api_key sk_live_***
vercel secrets add apify_webhook_secret whsec_***

# Link to project
vercel link

# Deploy preview
vercel

# Deploy production
vercel --prod

vercel.json Configuration


{
  "env": {
    "APIFY_API_KEY": "@apify_api_key"
  },
  "functions": {
    "api/**/*.ts": {
      "maxDuration": 30
    }
  }
}

Fly.io Deployment

fly.toml


app = "my-apify-app"
primary_region = "iad"

[env]
  NODE_ENV = "production"

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true

Secrets


# Set Apify secrets
fly secrets set APIFY_API_KEY=sk_live_***
fly secrets set APIFY_WEBHOOK_SECRET=whsec_***

# Deploy
fly deploy

Google Cloud Run

Dockerfile


FROM node:20-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
CMD ["npm", "start"]

Deploy Script


#!/bin/bash
# deploy-cloud-run.sh

PROJECT_ID="${GOOGLE_CLOUD_PROJECT}"
SERVICE_NAME="apify-service"
REGION="us-central1"

# Build and push image
gcloud builds submit --tag gcr.io/$PROJECT_ID/$SERVICE_NAME

# Deploy to Cloud Run
gcloud run deploy $SERVICE_NAME \
  --image gcr.io/$PROJECT_ID/$SERVICE_NAME \
  --region $REGION \
  --platform managed \
  --allow-unauthenticated \
  --set-secrets=APIFY_API_KEY=apify-api-key:latest

Environment Configuration Pattern


// config/apify.ts
interface ApifyConfig {
  apiKey: string;
  environment: 'development' | 'staging' | 'production';
  webhookSecret?: string;
}

export function getApifyConfig(): ApifyConfig {
  const env = process.env.NODE_ENV || 'development';

  return {
    apiKey: process.env.APIFY_API_KEY!,
    environment: env as ApifyConfig['environment'],
    webhookSecret: process.env.APIFY_WEBHOOK_SECRET,
  };
}

Health Check Endpoint


// api/health.ts
export async function GET() {
  const apifyStatus = await checkApifyConnection();

  return Response.json({
    status: apifyStatus ? 'healthy' : '


                
                  
                  
                  apify-hello-world
                  SKILL.md
                  View full skill →
                
                
                  Create a minimal working Apify example.
                  
                      ReadWriteEdit
                    
                
                
                  Apify Hello World
Overview
Minimal working example demonstrating core Apify functionality.
Prerequisites

Completed apify-install-auth setup
Valid API credentials configured
Development environment ready

Instructions
Step 1: Create Entry File
Create a new file for your hello world example.
Step 2: Import and Initialize Client

import { ApifyClient } from '@apify/sdk';

const client = new ApifyClient({
  apiKey: process.env.APIFY_API_KEY,
});

Step 3: Make Your First API Call

async function main() {
  // Your first API call here
}

main().catch(console.error);

Output

Working code file with Apify client initialization
Successful API response confirming connection
Console output showing:


Success! Your Apify connection is working.

Error Handling

Error
Cause
Solution


Import Error
SDK not installed
Verify with npm list or pip show


Auth Error
Invalid credentials
Check environment variable is set


Timeout
Network issues
Increase timeout or check connectivity


Rate Limit
Too many requests
Wait and retry with exponential backoff


Examples
TypeScript Example

import { ApifyClient } from '@apify/sdk';

const client = new ApifyClient({
  apiKey: process.env.APIFY_API_KEY,
});

async function main() {
  // Your first API call here
}

main().catch(console.error);

Python Example

from apify import ApifyClient

client = ApifyClient()

# Your first API call here

Resources

Apify Getting Started
Apify API Reference
Apify Examples

Next Steps
Proceed to apify-local-dev-loop for development workflow setup.


                
                  
                  
                  apify-install-auth
                  SKILL.md
                  View full skill →
                
                
                  Install and configure Apify SDK/CLI authentication.
                  
                      ReadWriteEditBash(npm:*)Bash(pip:*)Grep
                    
                
                
                  Apify Install & Auth
Overview
Set up Apify SDK/CLI and configure authentication credentials.
Prerequisites

Node.js 18+ or Python 3.10+
Package manager (npm, pnpm, or pip)
Apify account with API access
API key from Apify dashboard

Instructions
Step 1: Install SDK

# Node.js
npm install @apify/sdk

# Python
pip install apify

Step 2: Configure Authentication

# Set environment variable
export APIFY_API_KEY="your-api-key"

# Or create .env file
echo 'APIFY_API_KEY=your-api-key' >> .env

Step 3: Verify Connection

// Test connection code here

Output

Installed SDK package in node_modules or site-packages
Environment variable or .env file with API key
Successful connection verification output

Error Handling

Error
Cause
Solution


Invalid API Key
Incorrect or expired key
Verify key in Apify dashboard


Rate Limited
Exceeded quota
Check quota at https://docs.apify.com


Network Error
Firewall blocking
Ensure outbound HTTPS allowed


Module Not Found
Installation failed
Run npm install or pip install again


Examples
TypeScript Setup

import { ApifyClient } from '@apify/sdk';

const client = new ApifyClient({
  apiKey: process.env.APIFY_API_KEY,
});

Python Setup

from apify import ApifyClient

client = ApifyClient(
    api_key=os.environ.get('APIFY_API_KEY')
)

Resources

Apify Documentation
Apify Dashboard
Apify Status

Next Steps
After successful auth, proceed to apify-hello-world for your first API call.


                
                  
                  
                  apify-local-dev-loop
                  SKILL.md
                  View full skill →
                
                
                  Configure Apify local development with hot reload and testing.
                  
                      ReadWriteEditBash(npm:*)Bash(pnpm:*)Grep
                    
                
                
                  Apify Local Dev Loop
Overview
Set up a fast, reproducible local development workflow for Apify.
Prerequisites

Completed apify-install-auth setup
Node.js 18+ with npm/pnpm
Code editor with TypeScript support
Git for version control

Instructions
Step 1: Create Project Structure

my-apify-project/
├── src/
│   ├── apify/
│   │   ├── client.ts       # Apify client wrapper
│   │   ├── config.ts       # Configuration management
│   │   └── utils.ts        # Helper functions
│   └── index.ts
├── tests/
│   └── apify.test.ts
├── .env.local              # Local secrets (git-ignored)
├── .env.example            # Template for team
└── package.json

Step 2: Configure Environment

# Copy environment template
cp .env.example .env.local

# Install dependencies
npm install

# Start development server
npm run dev

Step 3: Setup Hot Reload

{
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "test": "vitest",
    "test:watch": "vitest --watch"
  }
}

Step 4: Configure Testing

import { describe, it, expect, vi } from 'vitest';
import { ApifyClient } from '../src/apify/client';

describe('Apify Client', () => {
  it('should initialize with API key', () => {
    const client = new ApifyClient({ apiKey: 'test-key' });
    expect(client).toBeDefined();
  });
});

Output

Working development environment with hot reload
Configured test suite with mocking
Environment variable management
Fast iteration cycle for Apify development

Error Handling

Error
Cause
Solution


Module not found
Missing dependency
Run npm install


Port in use
Another process
Kill process or change port


Env not loaded
Missing .env.local
Copy from .env.example


Test timeout
Slow network
Increase test timeout


Examples
Mock Apify Responses

vi.mock('@apify/sdk', () => ({
  ApifyClient: vi.fn().mockImplementation(() => ({
    // Mock methods here
  })),
}));

Debug Mode

# Enable verbose logging
DEBUG=APIFY=* npm run dev

Resources

Apify SDK Reference
Vitest Documentation
tsx Documentation

Next St


                
                  
                  
                  apify-performance-tuning
                  SKILL.md
                  View full skill →
                
                
                  Optimize Apify API performance with caching, batching, and connection pooling.
                  
                      ReadWriteEdit
                    
                
                
                  Apify Performance Tuning
Overview
Optimize Apify API performance with caching, batching, and connection pooling.
Prerequisites

Apify SDK installed
Understanding of async patterns
Redis or in-memory cache available (optional)
Performance monitoring in place

Latency Benchmarks

Operation
P50
P95
P99


Read
50ms
150ms
300ms


Write
100ms
250ms
500ms


List
75ms
200ms
400ms


Caching Strategy
Response Caching

import { LRUCache } from 'lru-cache';

const cache = new LRUCache<string, any>({
  max: 1000,
  ttl: 60000, // 1 minute
  updateAgeOnGet: true,
});

async function cachedApifyRequest<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttl?: number
): Promise<T> {
  const cached = cache.get(key);
  if (cached) return cached as T;

  const result = await fetcher();
  cache.set(key, result, { ttl });
  return result;
}

Redis Caching (Distributed)

import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

async function cachedWithRedis<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttlSeconds = 60
): Promise<T> {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await fetcher();
  await redis.setex(key, ttlSeconds, JSON.stringify(result));
  return result;
}

Request Batching

import DataLoader from 'dataloader';

const apifyLoader = new DataLoader<string, any>(
  async (ids) => {
    // Batch fetch from Apify
    const results = await apifyClient.batchGet(ids);
    return ids.map(id => results.find(r => r.id === id) || null);
  },
  {
    maxBatchSize: 100,
    batchScheduleFn: callback => setTimeout(callback, 10),
  }
);

// Usage - automatically batched
const [item1, item2, item3] = await Promise.all([
  apifyLoader.load('id-1'),
  apifyLoader.load('id-2'),
  apifyLoader.load('id-3'),
]);

Connection Optimization

import { Agent } from 'https';

// Keep-alive connection pooling
const agent = new Agent({
  keepAlive: true,
  maxSockets: 10,
  maxFreeSockets: 5,
  timeout: 30000,
});

const client = new ApifyClient({
  apiKey: process.env.APIFY_API_KEY!,
  httpAgent: agent,
});

Pagination Optimization

async function* paginatedApifyList<T>(
  fetcher: (cursor?: string) => Promise<{ data: T[]; nextCursor?: string }>
): AsyncGenerator<T> {
  let cursor: strin


                
                  
                  
                  apify-prod-checklist
                  SKILL.md
                  View full skill →
                
                
                  Execute Apify production deployment checklist and rollback procedures.
                  
                      ReadBash(kubectl:*)Bash(curl:*)Grep
                    
                
                
                  Apify Production Checklist
Overview
Complete checklist for deploying Apify integrations to production.
Prerequisites

Staging environment tested and verified
Production API keys available
Deployment pipeline configured
Monitoring and alerting ready

Instructions
Step 1: Pre-Deployment Configuration

[ ] Production API keys in secure vault
[ ] Environment variables set in deployment platform
[ ] API key scopes are minimal (least privilege)
[ ] Webhook endpoints configured with HTTPS
[ ] Webhook secrets stored securely

Step 2: Code Quality Verification

[ ] All tests passing (npm test)
[ ] No hardcoded credentials
[ ] Error handling covers all Apify error types
[ ] Rate limiting/backoff implemented
[ ] Logging is production-appropriate

Step 3: Infrastructure Setup

[ ] Health check endpoint includes Apify connectivity
[ ] Monitoring/alerting configured
[ ] Circuit breaker pattern implemented
[ ] Graceful degradation configured

Step 4: Documentation Requirements

[ ] Incident runbook created
[ ] Key rotation procedure documented
[ ] Rollback procedure documented
[ ] On-call escalation path defined

Step 5: Deploy with Gradual Rollout

# Pre-flight checks
curl -f https://staging.example.com/health
curl -s https://status.apify.com

# Gradual rollout - start with canary (10%)
kubectl apply -f k8s/production.yaml
kubectl set image deployment/apify-integration app=image:new --record
kubectl rollout pause deployment/apify-integration

# Monitor canary traffic for 10 minutes
sleep 600
# Check error rates and latency before continuing

# If healthy, continue rollout to 50%
kubectl rollout resume deployment/apify-integration
kubectl rollout pause deployment/apify-integration
sleep 300

# Complete rollout to 100%
kubectl rollout resume deployment/apify-integration
kubectl rollout status deployment/apify-integration

Output

Deployed Apify integration
Health checks passing
Monitoring active
Rollback procedure documented

Error Handling

Alert
Condition
Severity


API Down
5xx errors > 10/min
P1


High Latency
p99 > 5000ms
P2


Rate Limited
429 errors > 5/min
P2


Auth Failures
401/403 errors > 0
P1


Examples
Health Check Implementation

async function healthCheck(): Promise<{ status: string; apify: any }> {
  const start = Date.now();
  tr


                
                  
                  
                  apify-rate-limits
                  SKILL.md
                  View full skill →
                
                
                  Implement Apify rate limiting, backoff, and idempotency patterns.
                  
                      ReadWriteEdit
                    
                
                
                  Apify Rate Limits
Overview
Handle Apify rate limits gracefully with exponential backoff and idempotency.
Prerequisites

Apify SDK installed
Understanding of async/await patterns
Access to rate limit headers

Instructions
Step 1: Understand Rate Limit Tiers

Tier
Requests/min
Requests/day
Burst


Free
60
1,000
10


Pro
300
10,000
50


Enterprise
1,000
100,000
200


Step 2: Implement Exponential Backoff with Jitter

async function withExponentialBackoff<T>(
  operation: () => Promise<T>,
  config = { maxRetries: 5, baseDelayMs: 1000, maxDelayMs: 32000, jitterMs: 500 }
): Promise<T> {
  for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error: any) {
      if (attempt === config.maxRetries) throw error;
      const status = error.status || error.response?.status;
      if (status !== 429 && (status < 500 || status >= 600)) throw error;

      // Exponential delay with jitter to prevent thundering herd
      const exponentialDelay = config.baseDelayMs * Math.pow(2, attempt);
      const jitter = Math.random() * config.jitterMs;
      const delay = Math.min(exponentialDelay + jitter, config.maxDelayMs);

      console.log(`Rate limited. Retrying in ${delay.toFixed(0)}ms...`);
      await new Promise(r => setTimeout(r, delay));
    }
  }
  throw new Error('Unreachable');
}

Step 3: Add Idempotency Keys

import { v4 as uuidv4 } from 'uuid';
import crypto from 'crypto';

// Generate deterministic key from operation params (for safe retries)
function generateIdempotencyKey(operation: string, params: Record<string, any>): string {
  const data = JSON.stringify({ operation, params });
  return crypto.createHash('sha256').update(data).digest('hex');
}

async function idempotentRequest<T>(
  client: ApifyClient,
  params: Record<string, any>,
  idempotencyKey?: string  // Pass existing key for retries
): Promise<T> {
  // Use provided key (for retries) or generate deterministic key from params
  const key = idempotencyKey || generateIdempotencyKey(params.method || 'POST', params);
  return client.request({
    ...params,
    headers: { 'Idempotency-Key': key, ...params.headers },
  });
}

Output

Reliable API calls with automatic retry
Idempotent requests preventing duplicates
Rate limit headers properly handled

Error Handling

                
                  
                  
                  apify-reference-architecture
                  SKILL.md
                  View full skill →
                
                
                  Implement Apify reference architecture with best-practice project layout.
                  
                      ReadGrep
                    
                
                
                  Apify Reference Architecture
Overview
Production-ready architecture patterns for Apify integrations.
Prerequisites

Understanding of layered architecture
Apify SDK knowledge
TypeScript project setup
Testing framework configured

Project Structure

my-apify-project/
├── src/
│   ├── apify/
│   │   ├── client.ts           # Singleton client wrapper
│   │   ├── config.ts           # Environment configuration
│   │   ├── types.ts            # TypeScript types
│   │   ├── errors.ts           # Custom error classes
│   │   └── handlers/
│   │       ├── webhooks.ts     # Webhook handlers
│   │       └── events.ts       # Event processing
│   ├── services/
│   │   └── apify/
│   │       ├── index.ts        # Service facade
│   │       ├── sync.ts         # Data synchronization
│   │       └── cache.ts        # Caching layer
│   ├── api/
│   │   └── apify/
│   │       └── webhook.ts      # Webhook endpoint
│   └── jobs/
│       └── apify/
│           └── sync.ts         # Background sync job
├── tests/
│   ├── unit/
│   │   └── apify/
│   └── integration/
│       └── apify/
├── config/
│   ├── apify.development.json
│   ├── apify.staging.json
│   └── apify.production.json
└── docs/
    └── apify/
        ├── SETUP.md
        └── RUNBOOK.md

Layer Architecture

┌─────────────────────────────────────────┐
│             API Layer                    │
│   (Controllers, Routes, Webhooks)        │
├─────────────────────────────────────────┤
│           Service Layer                  │
│  (Business Logic, Orchestration)         │
├─────────────────────────────────────────┤
│          Apify Layer        │
│   (Client, Types, Error Handling)        │
├─────────────────────────────────────────┤
│         Infrastructure Layer             │
│    (Cache, Queue, Monitoring)            │
└─────────────────────────────────────────┘

Key Components
Step 1: Client Wrapper

// src/apify/client.ts
export class ApifyService {
  private client: ApifyClient;
  private cache: Cache;
  private monitor: Monitor;

  constructor(config: ApifyConfig) {
    this.client = new ApifyClient(config);
    this.cache = new Cache(config.cacheOptions);
    this.monitor = new Monitor('apify');
  }

  async get(id: string): Promise<Resource> {
    return this.cache.getOrFetch(id, () =>
      this.monitor.track('get', () => this.client.get(id))
    );
  }
}

Step 2: Error Boundary

// src/apify/errors.ts
export class ApifyServiceError extends Error {
  constructor(
    message: string,
    public readonly code: string,
    public readonly retryable: boolean,
    public readonly originalError?: Error
  ) {
    super(message);
    this.name = 'ApifyServiceError';
  }
}

export function wrapApifyError(error: unknown

                

              

                
                  
                  
                  apify-sdk-patterns
                  SKILL.md
                  View full skill →
                
                
                  Apply production-ready Apify SDK patterns for TypeScript and Python.
                  
                      ReadWriteEdit
                    
                
                
                  Apify SDK Patterns
Overview
Production-ready patterns for Apify SDK usage in TypeScript and Python.
Prerequisites

Completed apify-install-auth setup
Familiarity with async/await patterns
Understanding of error handling best practices

Instructions
Step 1: Implement Singleton Pattern (Recommended)

// src/apify/client.ts
import { ApifyClient } from '@apify/sdk';

let instance: ApifyClient | null = null;

export function getApifyClient(): ApifyClient {
  if (!instance) {
    instance = new ApifyClient({
      apiKey: process.env.APIFY_API_KEY!,
      // Additional options
    });
  }
  return instance;
}

Step 2: Add Error Handling Wrapper

import { ApifyError } from '@apify/sdk';

async function safeApifyCall<T>(
  operation: () => Promise<T>
): Promise<{ data: T | null; error: Error | null }> {
  try {
    const data = await operation();
    return { data, error: null };
  } catch (err) {
    if (err instanceof ApifyError) {
      console.error({
        code: err.code,
        message: err.message,
      });
    }
    return { data: null, error: err as Error };
  }
}

Step 3: Implement Retry Logic

async function withRetry<T>(
  operation: () => Promise<T>,
  maxRetries = 3,
  backoffMs = 1000
): Promise<T> {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await operation();
    } catch (err) {
      if (attempt === maxRetries) throw err;
      const delay = backoffMs * Math.pow(2, attempt - 1);
      await new Promise(r => setTimeout(r, delay));
    }
  }
  throw new Error('Unreachable');
}

Output

Type-safe client singleton
Robust error handling with structured logging
Automatic retry with exponential backoff
Runtime validation for API responses

Error Handling

Header
Description
Action

                
              

Pattern
Use Case
Benefit


Safe wrapper
All API calls
Prevents uncaught exceptions


Retry logic
Transient failures
Improves reliability


Type guards
Response validation
Catches API changes


Logging
All operations
Debugging and monitoring


Examples
Factory Pattern (Multi-tenant)

const clients = new Map<string, ApifyClient>();

export function getClientForTenant(tenantId: string): ApifyClient {
  if (!clients.has(tenantId)) {
    const apiKey = getTenantApiKey(tenantId);
    clients.set(tenantId, new ApifyClient({ apiKey }));
  }
  return clients.get(tenantId)!;
}


                

              

                
                  
                  
                  apify-security-basics
                  SKILL.md
                  View full skill →
                
                
                  Apply Apify security best practices for secrets and access control.
                  
                      ReadWriteGrep
                    
                
                
                  Apify Security Basics
Overview
Security best practices for Apify API keys, tokens, and access control.
Prerequisites

Apify SDK installed
Understanding of environment variables
Access to Apify dashboard

Instructions
Step 1: Configure Environment Variables

# .env (NEVER commit to git)
APIFY_API_KEY=sk_live_***
APIFY_SECRET=***

# .gitignore
.env
.env.local
.env.*.local

Step 2: Implement Secret Rotation

# 1. Generate new key in Apify dashboard
# 2. Update environment variable
export APIFY_API_KEY="new_key_here"

# 3. Verify new key works
curl -H "Authorization: Bearer ${APIFY_API_KEY}" \
  https://api.apify.com/health

# 4. Revoke old key in dashboard

Step 3: Apply Least Privilege

Environment
Recommended Scopes


Development
read:*


Staging
read:*, write:limited


Production
Only required scopes


Output

Secure API key storage
Environment-specific access controls
Audit logging enabled

Error Handling

Security Issue
Detection
Mitigation


Exposed API key
Git scanning
Rotate immediately


Excessive scopes
Audit logs
Reduce permissions


Missing rotation
Key age check
Schedule rotation


Examples
Service Account Pattern

const clients = {
  reader: new ApifyClient({
    apiKey: process.env.APIFY_READ_KEY,
  }),
  writer: new ApifyClient({
    apiKey: process.env.APIFY_WRITE_KEY,
  }),
};

Webhook Signature Verification

import crypto from 'crypto';

function verifyWebhookSignature(
  payload: string, signature: string, secret: string
): boolean {
  const expected = crypto.createHmac('sha256', secret).update(payload).digest('hex');
  return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected));
}

Security Checklist

[ ] API keys in environment variables
[ ] .env files in .gitignore
[ ] Different keys for dev/staging/prod
[ ] Minimal scopes per environment
[ ] Webhook signatures validated
[ ] Audit logging enabled

Audit Logging

interface AuditEntry {
  timestamp: Date;
  action: string;
  userId: string;
  resource: string;
  result: 'success' | 'failure';
  metadata?: Record<string, any>;
}


                

              

                
                  
                  
                  apify-upgrade-migration
                  SKILL.md
                  View full skill →
                
                
                  Analyze, plan, and execute Apify SDK upgrades with breaking change detection.
                  
                      ReadWriteEditBash(npm:*)Bash(git:*)
                    
                
                
                  Apify Upgrade & Migration
Overview
Guide for upgrading Apify SDK versions and handling breaking changes.
Prerequisites

Current Apify SDK installed
Git for version control
Test suite available
Staging environment

Instructions
Step 1: Check Current Version

npm list @apify/sdk
npm view @apify/sdk version

Step 2: Review Changelog

open https://github.com/apify/sdk/releases

Step 3: Create Upgrade Branch

git checkout -b upgrade/apify-sdk-vX.Y.Z
npm install @apify/sdk@latest
npm test

Step 4: Handle Breaking Changes
Update import statements, configuration, and method signatures as needed.
Output

Updated SDK version
Fixed breaking changes
Passing test suite
Documented rollback procedure

Error Handling

SDK Version
API Version
Node.js
Breaking Changes


3.x
2024-01
18+
Major refactor


2.x
2023-06
16+
Auth changes


1.x
2022-01
14+
Initial release


Examples
Import Changes

// Before (v1.x)
import { Client } from '@apify/sdk';

// After (v2.x)
import { ApifyClient } from '@apify/sdk';

Configuration Changes

// Before (v1.x)
const client = new Client({ key: 'xxx' });

// After (v2.x)
const client = new ApifyClient({
  apiKey: 'xxx',
});

Rollback Procedure

npm install @apify/sdk@1.x.x --save-exact

Deprecation Handling

// Monitor for deprecation warnings in development
if (process.env.NODE_ENV === 'development') {
  process.on('warning', (warning) => {
    if (warning.name === 'DeprecationWarning') {
      console.warn('[Apify]', warning.message);
      // Log to tracking system for proactive updates
    }
  });
}

// Common deprecation patterns to watch for:
// - Renamed methods: client.oldMethod() -> client.newMethod()
// - Changed parameters: { key: 'x' } -> { apiKey: 'x' }
// - Removed features: Check release notes before upgrading

Resources

Apify Changelog
Apify Migration Guide

Next Steps
For CI integration during upgrades, see apify-ci-integration.
                
              

                
                  
                  
                  apify-webhooks-events
                  SKILL.md
                  View full skill →
                
                
                  Implement Apify webhook signature validation and event handling.
                  
                      ReadWriteEditBash(curl:*)
                    
                
                
                  Apify Webhooks & Events
Overview
Securely handle Apify webhooks with signature validation and replay protection.
Prerequisites

Apify webhook secret configured
HTTPS endpoint accessible from internet
Understanding of cryptographic signatures
Redis or database for idempotency (optional)

Webhook Endpoint Setup
Express.js

import express from 'express';
import crypto from 'crypto';

const app = express();

// IMPORTANT: Raw body needed for signature verification
app.post('/webhooks/apify',
  express.raw({ type: 'application/json' }),
  async (req, res) => {
    const signature = req.headers['x-apify-signature'] as string;
    const timestamp = req.headers['x-apify-timestamp'] as string;

    if (!verifyApifySignature(req.body, signature, timestamp)) {
      return res.status(401).json({ error: 'Invalid signature' });
    }

    const event = JSON.parse(req.body.toString());
    await handleApifyEvent(event);

    res.status(200).json({ received: true });
  }
);

Signature Verification

function verifyApifySignature(
  payload: Buffer,
  signature: string,
  timestamp: string
): boolean {
  const secret = process.env.APIFY_WEBHOOK_SECRET!;

  // Reject old timestamps (replay attack protection)
  const timestampAge = Date.now() - parseInt(timestamp) * 1000;
  if (timestampAge > 300000) { // 5 minutes
    console.error('Webhook timestamp too old');
    return false;
  }

  // Compute expected signature
  const signedPayload = `${timestamp}.${payload.toString()}`;
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');

  // Timing-safe comparison
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expectedSignature)
  );
}

Event Handler Pattern

type ApifyEventType = 'resource.created' | 'resource.updated' | 'resource.deleted';

interface ApifyEvent {
  id: string;
  type: ApifyEventType;
  data: Record<string, any>;
  created: string;
}

const eventHandlers: Record<ApifyEventType, (data: any) => Promise<void>> = {
  'resource.created': async (data) => { /* handle */ },
  'resource.updated': async (data) => { /* handle */ },
  'resource.deleted': async (data) => { /* handle */ }
};

async function handleApifyEvent(event: ApifyEvent): Promise<void> {
  const handler = eventHandlers[event.type];

  if (!handler) {
    console.log(`Unhandled event type: ${event.type}`);
    return;
  }

  try {
    await handler(event.data);
    console.log(`Processed ${event.type}: ${event.id}`);
  } catch (error) {
    console.error(`Failed to process ${event.type}: ${event.id}`,


        

      
      

      
      

      
      

      
      
  Ready to use apify-pack?
  
    
    
  




      
      
          Related Plugins
          
            
                ai-ethics-validator
                AI ethics and fairness validation
              
                ai-experiment-logger
                Track and analyze AI experiments with a web dashboard and MCP tools
              
                ai-ml-engineering-pack
                Professional AI/ML Engineering toolkit: Prompt engineering, LLM integration, RAG systems, AI safety with 12 expert plugins
              
                ai-sdk-agents
                Multi-agent orchestration with AI SDK v5 - handoffs, routing, and coordination for any AI provider (OpenAI, Anthropic, Google)
              
                anomaly-detection-system
                Detect anomalies and outliers in data
              
                automl-pipeline-builder
                Build AutoML pipelines
              
          
        

      
      
          Tags
          
            apifysaassdkintegration
          
        
    
  

  

    

    
    
        
            
                Stay in the Loop
                
                    
                    
                    
                
                No spam. Unsubscribe anytime.
            

            
                
                    Product
                    
                        Explore
                        Skills
                        Cowork
                        Compare
                        Tools
                    
                
                
                    Resources
                    
                        Changelog
                        Collections
                        Playbooks
                        Research
                        Learning
                    
                
                
                    Company
                    
                        Community
                        Spotlight
                        GitHub
                    
                
                
                    Legal
                    
                        Privacy
                        Terms
                        Acceptable Use
                    
                
            

            
                Tons of Skills by Intent Solutions. Marine. Citadel Grad. 20 years ops → self-taught dev → AI architect.
                © 2026 Tons of Skills | Intent Solutions

Aspect	Workflow A	Workflow B
Use Case	Primary	Secondary
Complexity	Medium	Lower
Performance	Standard	Optimized

apify-pack

Installation

Skills (18)

Apify CI Integration

Overview

Prerequisites

Instructions

Step 1: Create GitHub Actions Workflow

Step 2: Configure Secrets

Step 3: Add Integration Tests

Output

Error Handling

Examples

Release Workflow

Branch Protection

Resources

Next Steps

Apify Common Errors

Overview

Prerequisites

Instructions

Step 1: Identify the Error

Step 2: Find Matching Error Below

Step 3: Apply Solution

Output

Error Handling

Authentication Failed

Rate Limit Exceeded

Network Timeout

Examples

Quick Diagnostic Commands

Escalation Path

Resources

Next Steps

Apify Core Workflow A

Overview

Prerequisites

Instructions

Step 1: Initialize

Step 2: Execute

Step 3: Finalize

Output

Error Handling

Examples

Complete Workflow

Common Variations

Resources

Next Steps

Apify Core Workflow B

Overview

Prerequisites

Instructions

Step 1: Setup

Step 2: Process

Step 3: Complete

Output

Error Handling

Examples

Complete Workflow

Error Recovery

Resources

Next Steps

Apify Cost Tuning

Overview

Prerequisites

Pricing Tiers

Cost Estimation

Usage Monitoring

Cost Reduction Strategies

Step 1: Request Sampling

Step 2: Batching Requests

Step 3: Caching (from P16)

Apify Debug Bundle

Overview

Prerequisites

Instructions

Step 1: Create Debug Bundle Script

Step 2: Collect Environment Info

Step 3: Gather SDK and Logs

Step 4: Package Bundle