AssemblyAI CI Integration

Overview

Set up CI/CD pipelines for AssemblyAI transcription projects with unit tests (mocked), integration tests (live API), and cost-controlled test strategies.

Prerequisites

• GitHub repository with Actions enabled
• AssemblyAI API key for testing
• npm/pnpm project configured

Instructions

Step 1: GitHub Actions Workflow

# .github/workflows/assemblyai-tests.yml
name: AssemblyAI Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test -- --coverage
        # Unit tests use mocked AssemblyAI client — no API key needed

  integration-tests:
    runs-on: ubuntu-latest
    # Only run on main branch to limit API costs
    if: github.ref == 'refs/heads/main'
    env:
      ASSEMBLYAI_API_KEY: ${{ secrets.ASSEMBLYAI_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run test:integration
        timeout-minutes: 5

Step 2: Configure Secrets

# Store API key as GitHub secret
gh secret set ASSEMBLYAI_API_KEY --body "your-test-api-key"

# Use a separate test key with lower quota to control costs
# Get one from https://www.assemblyai.com/app/account

Step 3: Unit Tests (Mocked — Free, Fast)

// tests/unit/transcription.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { AssemblyAI } from 'assemblyai';

vi.mock('assemblyai', () => ({
  AssemblyAI: vi.fn().mockImplementation(() => ({
    transcripts: {
      transcribe: vi.fn().mockResolvedValue({
        id: 'mock-transcript-id',
        status: 'completed',
        text: 'Hello world, this is a test transcription.',
        audio_duration: 5.2,
        words: [
          { text: 'Hello', start: 0, end: 300, confidence: 0.99 },
          { text: 'world', start: 310, end: 600, confidence: 0.98 },
        ],
        utterances: [
          { speaker: 'A', text: 'Hello world, this is a test transcription.', start: 0, end: 5200 },
        ],
      }),
      get: vi.fn(),
      list: vi.fn().mockResolvedValue({
        transcripts: [{ id: 'mock-1', status: 'completed' }],
      }),
      delete: vi.fn().mockResolvedValue({}),
    },
    lemur: {
      task: vi.fn().mockResolvedValue({
        request_id: 'mock-lemur-id',
        response: 'This is a greeting recording.',
 …

AssemblyAI Common Errors

Overview

Quick reference for the most common AssemblyAI errors across transcription, streaming, and LeMUR APIs with real error messages and solutions.

Scope and Safety

Troubleshoot only the transcript, streaming session, or request identifier already placed in scope. Treat audio URLs, transcript text, prompts, and authorization headers as sensitive. Use redacted request metadata and official status information; never paste API keys into terminal commands, tickets, or chat.

Prerequisites

• assemblyai package installed
• API key configured
• Access to application logs or console

Instructions

Error 1: Authentication Failed

Error: Authentication error: Invalid API key
Status: 401

Cause: API key is missing, invalid, or revoked.

Solution:

# Test directly without printing the key; load the header from a protected file
# or use the SDK in a local developer session.
curl --config <(printf '%s\n' "header = Authorization: $ASSEMBLYAI_API_KEY") \
  https://api.assemblyai.com/v2/transcript \
  -X GET

Error 2: Transcription Status Error

{ "status": "error", "error": "Download error: unable to download..." }

Cause: The audio URL is not publicly accessible, has expired, or returned non-audio content.

Solution:

// Verify URL is accessible
const response = await fetch(audioUrl, { method: 'HEAD' });
console.log('Content-Type:', response.headers.get('content-type'));
console.log('Status:', response.status);
// Content-Type should be audio/* or video/*

// For private files, upload directly
const transcript = await client.transcripts.transcribe({
  audio: './local-file.mp3',  // SDK handles upload
});

Error 3: Could Not Process Audio

{ "status": "error", "error": "Audio file could not be processed" }

Cause: Corrupted file, unsupported codec, file too short (<200ms), or audio is entirely silent.

Solution:

# Check file with ffprobe
ffprobe -v quiet -print_format json -show_format -show_streams input.mp3

# Convert to a known-good format
ffmpeg -i input.unknown -ar 16000 -ac 1 -f wav output.wav

Error 4: Rate Limit Exceeded

Error: Rate limit exceeded
Status: 429
Header: Retry-After: 30

Cause: Too many concurrent requests. Free tier: 5 streams/min. Paid: 100 streams/min (auto-scales).

…

AssemblyAI Core Workflow A — Async Transcription

Overview

Primary money-path workflow: submit audio for async transcription with audio intelligence features. The SDK handles file upload (for local files), queues the transcription job, and polls until completion.

Prerequisites

• assemblyai package installed
• API key configured in ASSEMBLYAI_API_KEY

Instructions

Step 1: Basic Async Transcription

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

// Remote URL — SDK queues and polls automatically
const transcript = await client.transcripts.transcribe({
  audio: 'https://example.com/meeting-recording.mp3',
});

console.log(transcript.text);
console.log(`Duration: ${transcript.audio_duration}s`);
console.log(`Words: ${transcript.words?.length}`);

Step 2: Local File Upload

// The SDK uploads the file and transcribes in one call
const transcript = await client.transcripts.transcribe({
  audio: './recordings/interview.wav',
});

// Or from a buffer/stream
import fs from 'fs';
const buffer = fs.readFileSync('./recordings/interview.wav');
const transcript2 = await client.transcripts.transcribe({
  audio: buffer,
});

Step 3: Speaker Diarization

const transcript = await client.transcripts.transcribe({
  audio: audioUrl,
  speaker_labels: true,
  speakers_expected: 3,  // Optional: hint for expected speaker count
});

// Utterances are grouped by speaker
for (const utterance of transcript.utterances ?? []) {
  console.log(`Speaker ${utterance.speaker}: ${utterance.text}`);
  // Speaker A: Good morning, thanks for joining.
  // Speaker B: Happy to be here.
}

Step 4: Full Audio Intelligence Stack

const transcript = await client.transcripts.transcribe({
  audio: audioUrl,

  // Speaker identification
  speaker_labels: true,

  // Content analysis
  sentiment_analysis: true,
  entity_detection: true,
  auto_highlights: true,
  iab_categories: true,       // Topic detection (IAB taxonomy)
  content_safety: true,        // Flag sensitive content
  summarization: true,
  summary_model: 'informative',
  summary_type: 'bullets',

  // Formatting
  punctuate: true,
  format_text: true,
  language_code: 'en',

  // Word boost for domain terms
  word_boost: ['AssemblyAI', 'LeMUR', 'transcription'],
  boost_param: 'high',
});

// --- Access results ---

// Sentiment per sentence
for (const s of transcript.sentiment_analysis_results ?? []) {
  console.log(`[${s.sentiment}] ${s.text}`);
  // [POSITIVE] I really enjoyed working on this project.
}

// Named entities
for (const e of transcrip…

AssemblyAI Core Workflow B — Streaming & LeMUR

Overview

Two advanced workflows: (1) real-time streaming transcription via WebSocket for live captioning and voice agents, and (2) LeMUR for applying LLMs to transcripts — summarization, Q&A, action items, and custom tasks.

Prerequisites

• assemblyai package installed (npm install assemblyai)
• API key configured in ASSEMBLYAI_API_KEY
• For streaming: microphone or audio stream source

Instructions

Use server-controlled credentials for service-to-service work and short-lived tokens for browser streaming. Obtain consent for live audio, set a session and retention limit, and process only the transcript IDs and prompts approved for the stated purpose. Validate structured LeMUR output before taking any downstream action.

Part 1: Real-Time Streaming Transcription

Step 1: Basic Streaming Setup

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

const transcriber = client.streaming.createService({
  // Model options: 'nova-3' (default), 'nova-3-pro' (highest accuracy)
  speech_model: 'nova-3',
  sample_rate: 16000,
});

transcriber.on('open', ({ sessionId }) => {
  console.log('Session opened:', sessionId);
});

transcriber.on('transcript', (message) => {
  // message_type: 'PartialTranscript' or 'FinalTranscript'
  if (message.message_type === 'FinalTranscript') {
    console.log('[Final]', message.text);
  } else {
    process.stdout.write(`\r[Partial] ${message.text}`);
  }
});

transcriber.on('error', (error) => {
  console.error('Streaming error:', error);
});

transcriber.on('close', (code, reason) => {
  console.log('Session closed:', code, reason);
});

await transcriber.connect();

// Send audio chunks (16-bit PCM, 16kHz mono)
// transcriber.sendAudio(audioBuffer);

// When done:
// await transcriber.close();

Step 2: Stream from Microphone (Node.js)

import { AssemblyAI } from 'assemblyai';
import { spawn } from 'child_process';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

const transcriber = client.streaming.createService({
  speech_model: 'nova-3',
  sample_rate: 16000,
});

transcriber.on('transcript', (msg) => {
  if (msg.message_type === 'FinalTranscript' && msg.text) {
    console.log(msg.text);
  }
});

await transcriber.connect();

// Use SoX to capture microphone audio as raw PCM
const mic = spawn('sox', [
  '-d',                  // default audio device
  '-t', 'raw',           // raw PCM output
  '-b', '16',  …

AssemblyAI Cost Tuning

Overview

Optimize AssemblyAI costs through model selection, feature-aware billing, and usage monitoring. AssemblyAI charges per audio hour with add-on pricing for intelligence features.

Prerequisites

• Access to AssemblyAI billing dashboard at https://www.assemblyai.com/app
• Understanding of current usage patterns

Actual Pricing (Pay-As-You-Go)

Speech-to-Text (Async)

Streaming Speech-to-Text

Audio Intelligence Add-Ons

LeMUR

Instructions

Step 1: Cost Estimation Calculator

interface CostEstimate {
  baseTranscriptionCost: number;
  featuresCost: number;
  totalCost: number;
  breakdown: Record<string, number>;
}

function estimateTranscriptionCost(
  audioHours: number,
  options: {
    model?: 'best' | 'nano';
    speakerLabels?: boolean;
    sentimentAnalysis?: boolean;
    entityDetection?: boolean;
    contentSafety?: boolean;
    iabCategories?: boolean;
    piiRedaction?: boolean;
  } = {}
): CostEstimate {
  const model = options.model ?? 'best';
  const baseRate = model === 'best' ? 0.37 : 0.12;
  const baseCost = audioHours * baseRate;

  const breakdown: Record<string, number> = {
    [`transcription (${model})`]: baseCost,
  };

  let featuresCost = 0;

  if (options.speakerLabels) {
    const cost = audioHours * 0.02;
    breakdown['speaker_labels'] = cost;
    featuresCost += cost;
  }
  if (options.sentimentAnalysis) {
    const cost = audioHours * 0.02;
    breakdown['sentiment_analysis'] = cos…

AssemblyAI Debug Bundle

Overview

Collect all diagnostic information needed to resolve AssemblyAI issues — SDK version, transcript status, API connectivity, and configuration — packaged for support tickets.

Prerequisites

• assemblyai package installed
• Access to application logs
• Failed transcript ID (if applicable)

Instructions

Step 1: Create Debug Bundle Script

#!/bin/bash
# assemblyai-debug-bundle.sh
set -euo pipefail

BUNDLE_DIR="assemblyai-debug-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BUNDLE_DIR"

echo "=== AssemblyAI Debug Bundle ===" > "$BUNDLE_DIR/summary.txt"
echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

# Environment
echo "--- Runtime ---" >> "$BUNDLE_DIR/summary.txt"
node --version >> "$BUNDLE_DIR/summary.txt" 2>&1 || echo "Node.js not found" >> "$BUNDLE_DIR/summary.txt"
echo "Platform: $(uname -s) $(uname -m)" >> "$BUNDLE_DIR/summary.txt"
echo "ASSEMBLYAI_API_KEY: ${ASSEMBLYAI_API_KEY:+[SET (${#ASSEMBLYAI_API_KEY} chars)]}" >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

# SDK version
echo "--- SDK Version ---" >> "$BUNDLE_DIR/summary.txt"
npm list assemblyai 2>/dev/null >> "$BUNDLE_DIR/summary.txt" || echo "assemblyai not in node_modules" >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

# API connectivity
echo "--- API Connectivity ---" >> "$BUNDLE_DIR/summary.txt"
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: ${ASSEMBLYAI_API_KEY:-none}" \
  https://api.assemblyai.com/v2/transcript 2>/dev/null || echo "FAILED")
echo "GET /v2/transcript: HTTP $HTTP_CODE" >> "$BUNDLE_DIR/summary.txt"

STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
  https://api.assemblyai.com/v2 2>/dev/null || echo "FAILED")
echo "GET /v2: HTTP $STATUS_CODE" >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

# AssemblyAI service status
echo "--- Service Status ---" >> "$BUNDLE_DIR/summary.txt"
curl -s https://status.assemblyai.com/api/v2/status.json 2>/dev/null \
  | python3 -m json.tool 2>/dev/null >> "$BUNDLE_DIR/summary.txt" \
  || echo "Could not fetch status" >> "$BUNDLE_DIR/summary.txt"

# Package bundle
tar -czf "$BUNDLE_DIR.tar.gz" "$BUNDLE_DIR"
rm -rf "$BUNDLE_DIR"
echo …

AssemblyAI Deploy Integration

Overview

Deploy AssemblyAI-powered transcription services to Vercel (serverless), Google Cloud Run (containers), and Fly.io with proper secrets management and webhook configuration.

Prerequisites

• AssemblyAI API key for production
• Platform CLI installed (vercel, gcloud, or fly)
• Application with working AssemblyAI integration

Instructions

Vercel Deployment (Serverless)

# Add secrets
vercel env add ASSEMBLYAI_API_KEY production
vercel env add ASSEMBLYAI_WEBHOOK_SECRET production

# Deploy
vercel --prod

API Route for Transcription:

// app/api/transcribe/route.ts (Next.js App Router)
import { AssemblyAI } from 'assemblyai';
import { NextRequest, NextResponse } from 'next/server';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

export async function POST(req: NextRequest) {
  const { audioUrl, features } = await req.json();

  if (!audioUrl) {
    return NextResponse.json({ error: 'audioUrl required' }, { status: 400 });
  }

  // Use submit() + webhook for production (non-blocking)
  const transcript = await client.transcripts.submit({
    audio: audioUrl,
    webhook_url: `${process.env.NEXT_PUBLIC_APP_URL}/api/webhooks/assemblyai`,
    webhook_auth_header_name: 'X-Webhook-Secret',
    webhook_auth_header_value: process.env.ASSEMBLYAI_WEBHOOK_SECRET!,
    speaker_labels: features?.speakerLabels ?? false,
    sentiment_analysis: features?.sentiment ?? false,
  });

  return NextResponse.json({
    transcriptId: transcript.id,
    status: transcript.status,
  });
}

Webhook Handler:

// app/api/webhooks/assemblyai/route.ts
import { AssemblyAI } from 'assemblyai';
import { NextRequest, NextResponse } from 'next/server';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

export async function POST(req: NextRequest) {
  // Verify webhook authenticity
  const secret = req.headers.get('x-webhook-secret');
  if (secret !== process.env.ASSEMBLYAI_WEBHOOK_SECRET) {
    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
  }

  const { transcript_id, status } = await req.json();

  if (status === 'completed') {
    const transcript = await client.transcripts.get(transcript_id);
    // Store transcript, notify user, trigger downstream processing
    console.log(`Transcript ${transcript_id} completed: ${transcript.text?.length} chars`);
  } else if (status === 'error') {
    console.error(`Transcript ${transcript_id} failed`);
  }

  return NextResponse.json({ received: true });
}

Vercel config:

{
  …

AssemblyAI Hello World

Overview

Minimal working examples demonstrating AssemblyAI's three core capabilities: async transcription, audio intelligence features, and LeMUR (LLM-powered analysis).

Prerequisites

• Completed assemblyai-install-auth setup
• Valid API key configured in ASSEMBLYAI_API_KEY

Instructions

Step 1: Basic Transcription (Remote URL)

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

async function transcribeUrl() {
  const transcript = await client.transcripts.transcribe({
    audio: 'https://storage.googleapis.com/aai-web-samples/5_common_sports_702.wav',
  });

  if (transcript.status === 'error') {
    throw new Error(`Transcription failed: ${transcript.error}`);
  }

  console.log('Transcript:', transcript.text);
  console.log('Duration:', transcript.audio_duration, 'seconds');
  console.log('Word count:', transcript.words?.length);
}

transcribeUrl().catch(console.error);

Step 2: Transcribe a Local File

async function transcribeLocal() {
  // The SDK handles upload automatically when you pass a local path
  const transcript = await client.transcripts.transcribe({
    audio: './recording.mp3',
  });

  console.log('Transcript:', transcript.text);

  // Access word-level timestamps
  for (const word of transcript.words ?? []) {
    console.log(`[${word.start}ms - ${word.end}ms] ${word.text} (${word.confidence})`);
  }
}

Step 3: Enable Audio Intelligence Features

async function transcribeWithIntelligence() {
  const transcript = await client.transcripts.transcribe({
    audio: 'https://storage.googleapis.com/aai-web-samples/5_common_sports_702.wav',
    speaker_labels: true,       // Who said what
    auto_highlights: true,       // Key phrases extraction
    sentiment_analysis: true,    // Sentiment per sentence
    entity_detection: true,      // Named entities (people, orgs, locations)
    summarization: true,         // Auto-summary
    summary_model: 'informative',
    summary_type: 'bullets',
  });

  // Speaker diarization
  for (const utterance of transcript.utterances ?? []) {
    console.log(`Speaker ${utterance.speaker}: ${utterance.text}`);
  }

  // Key phrases
  for (const result of transcript.auto_highlights_result?.results ?? []) {
    console.log(`Key phrase: "${result.text}" (mentioned ${result.count} times)`);
  }

  // Sentiment analysis
  for (const result of transcript.sentiment_analysis_results ?? []) {
    console.log(`${result.sentiment}: "${result.text}"`);
  }

  // Summary
  console.log('Summary:', transcript.summary);
}

Step 4: LeM…

AssemblyAI Install & Auth

Overview

Install the assemblyai npm package and configure API key authentication for transcription, LeMUR, and streaming APIs.

Prerequisites

• Node.js 18+ or Python 3.10+
• Package manager (npm, pnpm, yarn, or pip)
• AssemblyAI account — sign up at https://www.assemblyai.com/dashboard/signup
• API key from https://www.assemblyai.com/app/account

Instructions

Step 1: Install the SDK

# Node.js (official SDK)
npm install assemblyai

# Python
pip install assemblyai

Step 2: Configure API Key

# Set environment variable (recommended)
export ASSEMBLYAI_API_KEY="your-api-key-here"

# Or add to .env file
echo 'ASSEMBLYAI_API_KEY=your-api-key-here' >> .env

Add to .gitignore:

.env
.env.local
.env.*.local

Step 3: Initialize the Client

// src/assemblyai/client.ts
import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

export default client;

Step 4: Verify Connection

// verify-connection.ts
import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

async function verify() {
  // Transcribe a short public audio to confirm everything works
  const transcript = await client.transcripts.transcribe({
    audio: 'https://storage.googleapis.com/aai-web-samples/5_common_sports_702.wav',
  });

  if (transcript.status === 'error') {
    console.error('Transcription failed:', transcript.error);
    process.exit(1);
  }

  console.log('Connection verified. Transcript ID:', transcript.id);
  console.log('Status:', transcript.status);
  console.log('Text preview:', transcript.text?.slice(0, 100));
}

verify().catch(console.error);

Python Setup

import assemblyai as aai
import os

# Configure globally
aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]

# Or pass per-client
transcriber = aai.Transcriber()
transcript = transcriber.transcribe(
    "https://storage.googleapis.com/aai-web-samples/5_common_sports_702.wav"
)
print(transcript.text)

Output

• Installed assemblyai package in node_modules or site-packages
• API key stored in environment variable or .env file
• Client initialized and connection verified with a test transcription

Examples

For local setup, create a dedicated development credential in the provider dashboard, store it only in the approved secret manager or ignored local environment file, and run a minima…

AssemblyAI Local Dev Loop

Overview

Set up a fast, reproducible local development workflow for AssemblyAI transcription and LeMUR projects with mocking, caching, and hot reload.

Prerequisites

• Completed assemblyai-install-auth setup
• Node.js 18+ with npm/pnpm
• TypeScript project with tsx or ts-node

Instructions

Step 1: Project Structure

my-assemblyai-project/
├── src/
│   ├── assemblyai/
│   │   ├── client.ts       # Singleton client
│   │   ├── transcribe.ts   # Transcription helpers
│   │   └── lemur.ts        # LeMUR helpers
│   └── index.ts
├── tests/
│   ├── transcribe.test.ts
│   └── fixtures/
│       └── sample-transcript.json  # Cached API response
├── .env.local              # Local secrets (git-ignored)
├── .env.example            # Template for team
└── package.json

Step 2: Dev Scripts

{
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "test": "vitest",
    "test:watch": "vitest --watch",
    "transcribe": "tsx src/assemblyai/transcribe.ts"
  },
  "devDependencies": {
    "tsx": "^4.7.0",
    "vitest": "^1.6.0",
    "dotenv": "^16.4.0"
  },
  "dependencies": {
    "assemblyai": "^4.8.0"
  }
}

Step 3: Singleton Client with Env Loading

// src/assemblyai/client.ts
import 'dotenv/config';
import { AssemblyAI } from 'assemblyai';

let instance: AssemblyAI | null = null;

export function getClient(): AssemblyAI {
  if (!instance) {
    if (!process.env.ASSEMBLYAI_API_KEY) {
      throw new Error('ASSEMBLYAI_API_KEY not set. Copy .env.example to .env.local');
    }
    instance = new AssemblyAI({
      apiKey: process.env.ASSEMBLYAI_API_KEY,
    });
  }
  return instance;
}

Step 4: Cache Transcription Results for Fast Iteration

// src/assemblyai/transcribe.ts
import fs from 'fs';
import path from 'path';
import { getClient } from './client';
import type { Transcript } from 'assemblyai';

const CACHE_DIR = path.join(process.cwd(), '.assemblyai-cache');

export async function transcribeWithCache(
  audioUrl: string,
  options: Record<string, any> = {}
): Promise<Transcript> {
  const cacheKey = Buffer.from(audioUrl + JSON.stringify(options))
    .toString('base64url')
    .slice(0, 32);
  const cachePath = path.join(CACHE_DIR, `${cacheKey}.json`);

  // Return cached result in dev
  if (process.env.NODE_ENV !== 'production' && fs.existsSync(cachePath)) {
    console.log('[dev] Using cached transcript:', cacheKey);
    return JSON.pa…

AssemblyAI Performance Tuning

Overview

Optimize AssemblyAI transcription performance through model selection, parallel processing, caching, and webhook-based architectures.

Prerequisites

• assemblyai package installed
• Understanding of async patterns
• Redis or in-memory cache available (optional)

Latency Benchmarks (Actual)

Async Transcription

Streaming

Model Speed vs. Accuracy

Instructions

Step 1: Choose the Right Model

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

// For highest accuracy (default)
const accurate = await client.transcripts.transcribe({
  audio: audioUrl,
  speech_model: 'best',
});

// For fastest processing and lowest cost
const fast = await client.transcripts.transcribe({
  audio: audioUrl,
  speech_model: 'nano',
});

Step 2: Parallel Batch Processing

import PQueue from 'p-queue';

const queue = new PQueue({ concurrency: 10 });

async function batchTranscribe(audioUrls: string[]) {
  const results = await Promise.all(
    audioUrls.map(url =>
      queue.add(() =>
        client.transcripts.transcribe({ audio: url, speech_model: 'nano' })
      )
    )
  );

  return results.filter(t => t.status === 'completed');
}

// Process 100 files with 10 concurrent jobs
const urls = Array.from({ length: 100 }, (_, i) => `https://storage.example.com/audio-${i}.mp3`);
const transcripts = await batchTranscribe(urls);
con…

AssemblyAI Production Checklist

Overview

Complete checklist for deploying AssemblyAI-powered transcription services to production with health checks, monitoring, and rollback procedures.

Prerequisites

• Staging environment tested and verified
• Production API key from https://www.assemblyai.com/app/account
• Deployment pipeline configured
• Monitoring stack ready

Instructions

Pre-Deployment Checklist

API Key & Auth

• [ ] Production API key stored in secrets manager (not env files)
• [ ] Key is separate from dev/staging keys
• [ ] Temporary token endpoint configured for browser streaming
• [ ] API key rotation procedure documented

Code Quality

• [ ] All transcript.status === 'error' cases handled
• [ ] Rate limit retry with exponential backoff implemented
• [ ] No hardcoded API keys or audio URLs
• [ ] PII redaction enabled for sensitive audio content
• [ ] Webhook URL uses HTTPS
• [ ] Audio file upload size validated before submission

Error Handling

• [ ] 429 (rate limit) triggers retry with backoff
• [ ] 5xx (server error) triggers retry with backoff
• [ ] 401 (auth error) triggers alert, no retry
• [ ] transcript.status === 'error' logged with transcript ID and error message
• [ ] WebSocket disconnect triggers reconnection for streaming
• [ ] LeMUR errors handled (invalid transcript ID, context too long)

Performance

• [ ] Transcript results cached where appropriate
• [ ] Concurrent transcription jobs limited via queue (p-queue or similar)
• [ ] Webhook processing is async (don't block the response)
• [ ] Long audio files processed with webhook_url instead of polling

Health Check Implementation

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

export async function healthCheck(): Promise<{
  status: 'healthy' | 'degraded' | 'down';
  assemblyai: { connected: boolean; latencyMs: number };
}> {
  const start = Date.now();
  try {
    // List transcripts as a lightweight connectivity check
    await client.transcripts.list({ limit: 1 });
    return {
      status: 'healthy',
      assemblyai: { connected: true, latencyMs: Date.now() - start },
    };
  } catch (error) {
    return {
      status: 'degraded',
      assemblyai: { connected: false, latencyMs: Date.now() - start },
    };
  }
}

Webhook-Based Processing (Recommended for Production)

// Instead of polling, use webhooks for transcription completion
const transcript = await client.trans…

AssemblyAI Rate Limits

Overview

Handle AssemblyAI rate limits with exponential backoff, queue-based throttling, and concurrency management. AssemblyAI auto-scales limits for paid users.

Prerequisites

• assemblyai package installed
• Understanding of async/await patterns

Rate Limit Tiers (Actual)

Async Transcription API

Streaming (WebSocket)

LeMUR

Note: AssemblyAI auto-scales paid limits. At 70%+ utilization, the new session rate limit increases by 10% every 60 seconds with no ceiling cap.

Instructions

Step 1: Exponential Backoff with Jitter

import { AssemblyAI, type Transcript } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

async function transcribeWithBackoff(
  audioUrl: string,
  options: Record<string, any> = {},
  config = { maxRetries: 5, baseDelayMs: 1000, maxDelayMs: 30000 }
): Promise<Transcript> {
  for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
    try {
      return await client.transcripts.transcribe({
        audio: audioUrl,
        ...options,
      });
    } catch (err: any) {
      if (attempt === config.maxRetries) throw err;

      const status = err.status ?? err.statusCode;
      // Only retry on 429 (rate limit) and 5xx (server errors)
      if (status && status !== 429 && (status < 500 || status >= 600)) throw err;

      const exponentialDelay = config.baseDelayMs * Math.pow(2, attempt);
      const jitter = Math.random() * config.baseDelayMs;
      const delay = Math.min(exponentialDelay + jitter, config.maxDelayMs);

      console.warn(`[${attempt + 1}/${config.maxRetries}] Retrying in ${delay.toFixed(0)}ms...`);
      await new Promise(r => setTimeout(r, delay));
    }
  }
  throw new Error('Unreachable');
}

Step 2: Queue-Based Concurre…

AssemblyAI Reference Architecture

Overview

Production-ready architecture for AssemblyAI-powered transcription services with layered design, webhook-driven processing, and LeMUR analysis pipelines.

Prerequisites

• Understanding of layered architecture
• assemblyai npm package
• TypeScript project setup
• Database for transcript storage

Project Structure

my-transcription-service/
├── src/
│   ├── assemblyai/
│   │   ├── client.ts              # Singleton client
│   │   ├── transcription.ts       # Transcription service
│   │   ├── streaming.ts           # Streaming service
│   │   ├── lemur.ts               # LeMUR analysis service
│   │   └── types.ts               # Domain types
│   ├── api/
│   │   ├── transcribe.ts          # POST /api/transcribe
│   │   ├── transcripts.ts         # GET /api/transcripts/:id
│   │   ├── streaming-token.ts     # GET /api/streaming-token
│   │   └── webhooks/
│   │       └── assemblyai.ts      # POST /webhooks/assemblyai
│   ├── services/
│   │   ├── audio-processor.ts     # Audio validation & preprocessing
│   │   └── transcript-store.ts    # Database storage
│   ├── jobs/
│   │   └── batch-transcriber.ts   # Background batch processing
│   └── config.ts
├── tests/
│   ├── unit/
│   │   ├── transcription.test.ts
│   │   └── lemur.test.ts
│   └── integration/
│       └── assemblyai.test.ts
└── package.json

Architecture Layers

┌──────────────────────────────────────────────────────┐
│                    API Layer                          │
│  Transcribe endpoint, Webhook handler, Stream token  │
├──────────────────────────────────────────────────────┤
│                  Service Layer                        │
│  TranscriptionService, LeMURService, AudioProcessor  │
├──────────────────────────────────────────────────────┤
│                AssemblyAI SDK Layer                   │
│  client.transcripts, client.streaming, client.lemur  │
├──────────────────────────────────────────────────────┤
│               Infrastructure Layer                    │
│  Database, Redis Cache, Job Queue, Monitoring        │
└──────────────────────────────────────────────────────┘

Instructions

Step 1: Client Layer

// src/assemblyai/client.ts
import { AssemblyAI } from 'assemblyai';

let client: AssemblyAI | null = null;

export function getClient(): AssemblyAI {
  if (!client) {
    client = new AssemblyAI({
      apiKey: process.env.ASSEMBLYAI_API_KEY!,
    });
  }
  return client;
}

Step 2: Transcription Service

// src/assemblyai/transcription.ts
import { type Transcript, type TranscriptParams } from 'assemblyai';
import { getClient } from './client';

export interface TranscriptionOptions {
  speakerLabels?: boolean;
  sentimentAnal…

AssemblyAI SDK Patterns

Overview

Production-ready patterns for the assemblyai npm package covering client initialization, type-safe wrappers, error handling, and multi-tenant architectures.

Prerequisites

• assemblyai package installed (npm install assemblyai)
• Familiarity with async/await and TypeScript generics

Instructions

Step 1: Type-Safe Singleton Client

// src/assemblyai/client.ts
import { AssemblyAI } from 'assemblyai';

let instance: AssemblyAI | null = null;

export function getAssemblyAI(): AssemblyAI {
  if (!instance) {
    const apiKey = process.env.ASSEMBLYAI_API_KEY;
    if (!apiKey) throw new Error('ASSEMBLYAI_API_KEY is required');
    instance = new AssemblyAI({ apiKey });
  }
  return instance;
}

Step 2: Transcription Service Wrapper

// src/assemblyai/transcription-service.ts
import { AssemblyAI, type Transcript, type TranscriptParams } from 'assemblyai';

export interface TranscriptionResult {
  id: string;
  text: string;
  duration: number;
  words: Array<{ text: string; start: number; end: number; confidence: number }>;
  speakers?: Array<{ speaker: string; text: string }>;
}

export class TranscriptionService {
  constructor(private client: AssemblyAI) {}

  async transcribe(
    audio: string,
    options: Partial<TranscriptParams> = {}
  ): Promise<TranscriptionResult> {
    const transcript = await this.client.transcripts.transcribe({
      audio,
      ...options,
    });

    if (transcript.status === 'error') {
      throw new Error(`Transcription failed: ${transcript.error}`);
    }

    return {
      id: transcript.id,
      text: transcript.text ?? '',
      duration: transcript.audio_duration ?? 0,
      words: (transcript.words ?? []).map(w => ({
        text: w.text,
        start: w.start,
        end: w.end,
        confidence: w.confidence,
      })),
      speakers: transcript.utterances?.map(u => ({
        speaker: u.speaker,
        text: u.text,
      })),
    };
  }

  async getTranscript(id: string): Promise<Transcript> {
    return this.client.transcripts.get(id);
  }

  async listTranscripts(params?: { limit?: number; status?: string }) {
    const page = await this.client.transcripts.list(params);
    return page.transcripts;
  }

  async deleteTranscript(id: string): Promise<void> {
    await this.client.transcripts.delete(id);
  }
}

Step 3: Error Handling Wrapper

// src/assemblyai/errors.ts

export class AssemblyAIServiceError extends Error {
  constructor(
    message: string,
    public readonly statusCode?: number,
    public readonly retryable: boolean = false,
    public readonly transcriptId?: string
  ) {
    super(message);
    th…

AssemblyAI Security Basics

Overview

Security best practices for AssemblyAI: API key management, temporary tokens for browser clients, PII redaction, and data retention policies.

Prerequisites

• assemblyai package installed
• Understanding of environment variables
• AssemblyAI dashboard access

Instructions

Step 1: API Key Management

# .env (NEVER commit)
ASSEMBLYAI_API_KEY=your-api-key-here

# .gitignore
.env
.env.local
.env.*.local

// Never hardcode API keys
// BAD:
const client = new AssemblyAI({ apiKey: 'sk_abc123...' });

// GOOD:
import { AssemblyAI } from 'assemblyai';
const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

Step 2: Temporary Tokens for Browser Streaming

Never expose your API key in frontend code. Use temporary tokens for browser-side streaming:

// Server-side: /api/assemblyai-token.ts
import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

export async function GET() {
  // Token expires after 5 minutes
  const token = await client.streaming.createTemporaryToken({
    expires_in_seconds: 300,
  });

  return Response.json({ token });
}

// Client-side: use the temporary token
// const { token } = await fetch('/api/assemblyai-token').then(r => r.json());
// const transcriber = new StreamingTranscriber({ token });

Step 3: PII Redaction in Transcripts

const transcript = await client.transcripts.transcribe({
  audio: audioUrl,
  redact_pii: true,
  redact_pii_policies: [
    'email_address',
    'phone_number',
    'person_name',
    'credit_card_number',
    'social_security_number',
    'date_of_birth',
    'medical_condition',
    'banking_information',
    'us_social_security_number',
  ],
  redact_pii_sub: 'entity_name', // or 'hash'
  // 'entity_name': "My name is [PERSON_NAME]"
  // 'hash':        "My name is ####"
});

// Also redact the audio itself
const transcriptWithRedactedAudio = await client.transcripts.transcribe({
  audio: audioUrl,
  redact_pii: true,
  redact_pii_policies: ['person_name', 'phone_number'],
  redact_pii_audio: true, // Generates audio with PII beeped out
});

Step 4: Data Retention and Deletion

// Delete transcript data for GDPR/privacy compliance
await client.transcripts.delete(transcriptId);
// This permanently removes the transcript text and metadata
// The audio file at your source URL is NOT deleted (you manage that)

// List and bulk-delete old transcripts
const page =…

AssemblyAI Upgrade & Migration

Overview

Guide for upgrading the assemblyai npm package and migrating between SDK versions, including the major v3 to v4 migration and speech model transitions.

Prerequisites

• Current assemblyai package installed
• Git for version control
• Test suite available

Instructions

Step 1: Check Current Version

# Check installed version
npm list assemblyai

# Check latest available version
npm view assemblyai version

# See what changed
npm view assemblyai --json | jq '.versions[-5:]'

Step 2: Review Changelog

# View release notes
open https://github.com/AssemblyAI/assemblyai-node-sdk/releases

# Or check from CLI
gh release list --repo AssemblyAI/assemblyai-node-sdk --limit 10

Step 3: Create Upgrade Branch

git checkout -b upgrade/assemblyai-sdk-v$(npm view assemblyai version)
npm install assemblyai@latest
npm test

Step 4: Major Migration — Old SDK to Current (v4.x)

If migrating from @assemblyai/sdk (old package) to assemblyai (current):

// BEFORE (old package — @assemblyai/sdk)
import { AssemblyAI as OldClient } from '@assemblyai/sdk';
const client = new OldClient({ apiKey: '...' });

// AFTER (current package — assemblyai)
import { AssemblyAI } from 'assemblyai';
const client = new AssemblyAI({ apiKey: '...' });

# Remove old package, install new
npm uninstall @assemblyai/sdk
npm install assemblyai

Key changes in the migration:

Step 5: Transcription Method Changes

// v3 pattern: create + poll loop
const job = await client.transcripts.create({ audio_url: url });
let transcript;
while (true) {
  transcript = await client.transcripts.get(job.id);
  if (transcript.status === 'completed…

AssemblyAI Webhooks & Events

Overview

Handle AssemblyAI webhooks for transcription completion. When you submit a transcript with webhook_url, AssemblyAI sends a POST request to your URL when the transcript is completed or fails. One webhook per transcript — no complex event routing needed.

Prerequisites

• HTTPS endpoint accessible from the internet
• assemblyai package installed
• API key configured

How AssemblyAI Webhooks Work

1. You submit a transcription with webhook_url parameter
2. AssemblyAI processes the audio asynchronously
3. When done (completed or error), AssemblyAI sends a POST to your URL
4. Your endpoint receives transcript ID and status, then fetches the full transcript

Key difference from other APIs: AssemblyAI webhooks are per-transcript (set at submission time), not a global webhook registration. There are no event types to subscribe to — you get one callback per transcript.

Instructions

Step 1: Submit Transcription with Webhook

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

// submit() queues the job and returns immediately (doesn't poll)
const transcript = await client.transcripts.submit({
  audio: 'https://example.com/meeting-recording.mp3',
  webhook_url: 'https://your-app.com/webhooks/assemblyai',

  // Optional: auth header for webhook verification
  webhook_auth_header_name: 'X-Webhook-Secret',
  webhook_auth_header_value: process.env.ASSEMBLYAI_WEBHOOK_SECRET!,

  // Enable features — results will be available when webhook fires
  speaker_labels: true,
  sentiment_analysis: true,
  auto_highlights: true,
});

console.log('Submitted:', transcript.id);
// Returns immediately, webhook fires when processing completes

Step 2: Webhook Endpoint (Express.js)

import express from 'express';
import { AssemblyAI, type Transcript } from 'assemblyai';

const app = express();
const client = new AssemblyAI({
  apiKey: process.env.ASSEMBLYAI_API_KEY!,
});

app.post('/webhooks/assemblyai', express.json(), async (req, res) => {
  // Step 1: Verify authenticity via custom auth header
  const secret = req.headers['x-webhook-secret'];
  if (secret !== process.env.ASSEMBLYAI_WEBHOOK_SECRET) {
    console.warn('Webhook auth failed');
    return res.status(401).json({ error: 'Unauthorized' });
  }

  // Step 2: Extract payload
  const { transcript_id, status } = req.body;
  console.log(`Webhook received: ${transcript_id} — ${status}`);

  // Step 3: Respond quickly (within 10 seconds)
  res.status(200).json({ received: true });

  // Step 4: Process asynchronously
  try {
    if (st…