Cohere CI Integration

Overview

Set up CI/CD pipelines with automated unit tests (mocked) and integration tests (real API) for Cohere API v2 applications.

Prerequisites

• GitHub repository with Actions enabled
• Cohere trial or production API key
• cohere-ai package installed

Instructions

Step 1: GitHub Actions Workflow

# .github/workflows/cohere-ci.yml
name: Cohere CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test -- --coverage
        # Unit tests use mocked Cohere responses — no API key needed

  integration-tests:
    runs-on: ubuntu-latest
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    env:
      CO_API_KEY: ${{ secrets.CO_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - name: Run Cohere integration tests
        run: npm run test:integration
        timeout-minutes: 5

Step 2: Configure GitHub Secrets

# Store Cohere API key as a repo secret
gh secret set CO_API_KEY --body "your-production-key-here"

# Verify it was set
gh secret list

Step 3: Unit Tests (Mocked — No API Key)

// tests/unit/cohere-chat.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';

// Mock the entire SDK
vi.mock('cohere-ai', () => ({
  CohereClientV2: vi.fn().mockImplementation(() => ({
    chat: vi.fn().mockResolvedValue({
      message: { content: [{ type: 'text', text: 'mocked response' }] },
      finishReason: 'COMPLETE',
      usage: { billedUnits: { inputTokens: 5, outputTokens: 3 } },
    }),
    embed: vi.fn().mockResolvedValue({
      embeddings: { float: [[0.1, 0.2, 0.3]] },
    }),
    rerank: vi.fn().mockResolvedValue({
      results: [{ index: 0, relevanceScore: 0.95 }],
    }),
  })),
}));

describe('Chat service', () => {
  it('returns text from chat completion', async () => {
    const { CohereClientV2 } = await import('cohere-ai');
    const cohere = new CohereClientV2();

    const response = await cohere.chat({
      model: 'command-a-03-2025',
      messages: [{ role: 'user', content: 'test' }],
    });

    expect(response.message?.content?.[0]?.text).toBe('mocked response');
    expect(cohere.chat).toHaveBeenCalledWith(
      expect.objectContaining({ model: 'command-a-03-2025' })
    );
  …

Cohere Common Errors

Overview

Quick reference for real Cohere API v2 errors with exact messages, causes, and fixes.

Prerequisites

• cohere-ai SDK installed
• CO_API_KEY configured
• Access to error logs

Error Reference

400 — Bad Request: Missing Required Field

CohereError: model is required

Cause: API v2 requires model for all endpoints (Chat, Embed, Rerank, Classify).

Fix:

// Wrong (v1 style)
await cohere.chat({ messages: [...] });

// Correct (v2)
await cohere.chat({ model: 'command-a-03-2025', messages: [...] });

400 — Embed: Missing embedding_types

CohereError: embedding_types is required for embed models v3 and higher

Fix:

await cohere.embed({
  model: 'embed-v4.0',
  texts: ['hello'],
  inputType: 'search_document',
  embeddingTypes: ['float'],  // Required for v3+
});

400 — Embed: Missing input_type

CohereError: input_type is required for embed models v3 and higher

Fix: Use one of: search_document, search_query, classification, clustering, image.

401 — Invalid API Token

CohereError: invalid api token

Cause: CO_API_KEY is missing, wrong, or revoked.

Fix:

# Verify key is set
echo $CO_API_KEY

# Test directly
curl -H "Authorization: Bearer $CO_API_KEY" \
  https://api.cohere.com/v2/chat \
  -H "Content-Type: application/json" \
  -d '{"model":"command-r7b-12-2024","messages":[{"role":"user","content":"hi"}]}'

429 — Rate Limit Exceeded

CohereError: You are using a Trial key, which is limited to N calls/minute

Rate limits by key type:

Fix:

import { CohereError } from 'cohere-ai';

try {
  await cohere.chat({ model: 'command-a-03-2025', messages: [...] });
} catch (err) {
  if (err instanceof CohereError && err.statusCode === 429) {
    // Back off and retry
    await new Promise(r => setTimeout(r, 60_000…

Cohere RAG Pipeline (Core Workflow A)

Overview

End-to-end Retrieval-Augmented Generation using Cohere's three core endpoints: Embed (vectorize), Rerank (sort by relevance), Chat (generate grounded answer with citations).

Prerequisites

• Completed cohere-install-auth setup
• cohere-ai package installed
• Understanding of vector similarity search

Instructions

Step 1: Embed Your Documents

import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2();

// Your knowledge base
const documents = [
  { id: 'doc1', text: 'Cohere Command A has 256K context and supports tool use.' },
  { id: 'doc2', text: 'Embed v4 generates 1024-dim vectors with 128K token context.' },
  { id: 'doc3', text: 'Rerank v3.5 scores relevance from 0 to 1 across 100+ languages.' },
  { id: 'doc4', text: 'The Chat API v2 requires model as a mandatory parameter.' },
  { id: 'doc5', text: 'Cohere supports structured JSON output via response_format.' },
];

// Embed documents for storage
const docEmbeddings = await cohere.embed({
  model: 'embed-v4.0',
  texts: documents.map(d => d.text),
  inputType: 'search_document',
  embeddingTypes: ['float'],
});

// Store vectors alongside document text in your vector DB
const vectors = docEmbeddings.embeddings.float;
console.log(`Embedded ${vectors.length} docs, ${vectors[0].length} dimensions each`);

Step 2: Search — Embed the Query

async function searchDocuments(query: string, topK = 10) {
  // Embed the query (note: inputType is 'search_query', not 'search_document')
  const queryEmbedding = await cohere.embed({
    model: 'embed-v4.0',
    texts: [query],
    inputType: 'search_query',
    embeddingTypes: ['float'],
  });

  const queryVector = queryEmbedding.embeddings.float[0];

  // Cosine similarity search (replace with your vector DB query)
  const scores = vectors.map((vec, i) => ({
    index: i,
    score: cosineSimilarity(queryVector, vec),
  }));

  return scores
    .sort((a, b) => b.score - a.score)
    .slice(0, topK)
    .map(s => documents[s.index]);
}

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0, magA = 0, magB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    magA += a[i] * a[i];
    magB += b[i] * b[i];
  }
  return dot / (Math.sqrt(magA) * Math.sqrt(magB));
}

Step 3: Rerank Retrieved Documents

async function rerankResults(query: string, candidates: typeof documents) {
  const response = await cohere.rerank({
    model: 'rerank-v3.5',
    query,
    documents: candidates.map(d => d.text),
    topN: 3,
…

Cohere Tool Use & Agents (Core Workflow B)

Overview

Build multi-step tool-using agents with Cohere's Chat API v2. The model decides which tools to call, you execute them, and feed results back in a loop until the task is complete.

Prerequisites

• Completed cohere-install-auth setup
• Understanding of cohere-core-workflow-a (RAG)
• Command R7B or newer model (required for tool use)

Instructions

Step 1: Define Tools

import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2();

// Define tools the model can call
const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'get_weather',
      description: 'Get current weather for a city',
      parameters: {
        type: 'object' as const,
        properties: {
          city: { type: 'string', description: 'City name' },
          unit: { type: 'string', enum: ['celsius', 'fahrenheit'], description: 'Temperature unit' },
        },
        required: ['city'],
      },
    },
  },
  {
    type: 'function' as const,
    function: {
      name: 'search_database',
      description: 'Search internal database for records',
      parameters: {
        type: 'object' as const,
        properties: {
          query: { type: 'string', description: 'Search query' },
          limit: { type: 'number', description: 'Max results' },
        },
        required: ['query'],
      },
    },
  },
];

Step 2: Implement Tool Executors

// Map tool names to actual implementations
const toolExecutors: Record<string, (args: any) => Promise<string>> = {
  get_weather: async ({ city, unit = 'celsius' }) => {
    // Replace with real weather API call
    return JSON.stringify({
      city,
      temperature: unit === 'celsius' ? 22 : 72,
      unit,
      condition: 'partly cloudy',
    });
  },

  search_database: async ({ query, limit = 5 }) => {
    // Replace with real database query
    return JSON.stringify({
      results: [
        { id: 1, title: `Result for: ${query}`, relevance: 0.95 },
      ],
      total: 1,
    });
  },
};

Step 3: Single-Step Tool Use

async function singleStepToolUse(userMessage: string) {
  // 1. Send message with tools
  const response = await cohere.chat({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: userMessage }],
    tools,
  });

  // 2. Check if model wants to call tools
  if (response.finishReason === 'TOOL_CALL') {
    const toolCalls = response.message?.toolCalls ?? [];

    // 3. Execute each tool call
   …

Cohere Cost Tuning

Overview

Optimize Cohere costs through model selection, token budgets, embedding compression, and usage monitoring. Cohere pricing is token-based with separate input/output rates.

Prerequisites

• Cohere production key (trial is free but limited)
• Access to dashboard.cohere.com billing page

Cohere Pricing Model

Key principle: Cohere charges per token. Input tokens and output tokens have different rates. Embed, Rerank, and Classify have separate pricing based on search units.

Model Cost Comparison

Non-Chat Pricing

Instructions

Strategy 1: Model Tiering

type CostTier = 'economy' | 'standard' | 'premium';

function selectModel(tier: CostTier): string {
  switch (tier) {
    case 'economy':  return 'command-r7b-12-2024';    // ~5x cheaper
    case 'standard': return 'command-r-08-2024';       // Good balance
    case 'premium':  return 'command-a-03-2025';       // Best quality
  }
}

// Route by use case
function routeModel(task: string): string {
  // High-volume, simple tasks → cheapest model
  if (['classify', 'extract', 'summarize-short'].includes(task)) {
    return selectModel('economy');
  }
  // RAG, moderate complexity
  if (['rag', 'search', 'qa'].includes(task)) {
    return selectModel('standard');
  }
  // Complex reasoning, user-facing
  return selectModel('premium');
}

…

Cohere Data Handling

Overview

Handle sensitive data when calling Cohere APIs. Cohere processes text server-side for Chat, Embed, Rerank, and Classify — any PII in your input reaches their servers. This skill covers pre-call redaction, post-call scrubbing, and compliance patterns.

Prerequisites

• Understanding of GDPR/CCPA requirements
• cohere-ai SDK installed
• Database for audit logging

Data Flow Awareness

Your App → [PII Redaction] → Cohere API → [Response Scrubbing] → Your App → User

Key point: Everything you send to cohere.chat(), cohere.embed(), etc.
is processed on Cohere's servers. Redact BEFORE the API call.

Instructions

Step 1: PII Detection

interface PIIFinding {
  type: string;
  match: string;
  start: number;
  end: number;
}

const PII_PATTERNS: Array<{ type: string; regex: RegExp }> = [
  { type: 'email', regex: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g },
  { type: 'phone', regex: /\b(\+\d{1,3}[-.]?)?\d{3}[-.]?\d{3}[-.]?\d{4}\b/g },
  { type: 'ssn', regex: /\b\d{3}-\d{2}-\d{4}\b/g },
  { type: 'credit_card', regex: /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g },
  { type: 'ip_address', regex: /\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b/g },
];

function detectPII(text: string): PIIFinding[] {
  const findings: PIIFinding[] = [];
  for (const { type, regex } of PII_PATTERNS) {
    for (const match of text.matchAll(new RegExp(regex))) {
      findings.push({
        type,
        match: match[0],
        start: match.index!,
        end: match.index! + match[0].length,
      });
    }
  }
  return findings;
}

Step 2: Pre-Call Redaction

function redactPII(text: string): { redacted: string; map: Map<string, string> } {
  const map = new Map<string, string>();
  let redacted = text;
  let counter = 0;

  for (const { type, regex } of PII_PATTERNS) {
    redacted = redacted.replace(new RegExp(regex), (match) => {
      const placeholder = `[${type.toUpperCase()}_${counter++}]`;
      map.set(placeholder, match);
      return placeholder;
    });
  }

  return { redacted, map };
}

// Usage: redact before sending to Cohere
async function safeCohereChat(userInput: string) {
  const { redacted, map } = redactPII(userInput);

  const response = await cohere.chat({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: redacted }],
  });

  // Optionally restore PII in response (for internal use only)
  let answer = response.message?.content?.[0]?.text ?? '';
  for (const [placeholder, original] of map) {
    answer = answer.replace(placeholder, original);
  }

  return answer;
}

Step 3: Safe Embedding

// Embeddings are stored long-term in ve…

Cohere Debug Bundle

Overview

Collect all diagnostic information needed to resolve Cohere API v2 issues. Generates a support-ready bundle with environment info, request/response logs, and SDK version data.

Prerequisites

• cohere-ai SDK installed
• Access to application logs
• curl and jq available

Instructions

Step 1: Create Debug Bundle Script

#!/bin/bash
# cohere-debug-bundle.sh
set -euo pipefail

BUNDLE_DIR="cohere-debug-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BUNDLE_DIR"

echo "=== Cohere Debug Bundle ===" > "$BUNDLE_DIR/summary.txt"
echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$BUNDLE_DIR/summary.txt"
echo "" >> "$BUNDLE_DIR/summary.txt"

Step 2: Collect Environment and SDK Info

# Runtime versions
echo "--- Runtime ---" >> "$BUNDLE_DIR/summary.txt"
node --version >> "$BUNDLE_DIR/summary.txt" 2>&1 || echo "Node.js: not found" >> "$BUNDLE_DIR/summary.txt"
python3 --version >> "$BUNDLE_DIR/summary.txt" 2>&1 || echo "Python: not found" >> "$BUNDLE_DIR/summary.txt"

# SDK version
echo "--- SDK ---" >> "$BUNDLE_DIR/summary.txt"
npm list cohere-ai 2>/dev/null >> "$BUNDLE_DIR/summary.txt" || echo "cohere-ai: not installed (npm)" >> "$BUNDLE_DIR/summary.txt"
pip show cohere 2>/dev/null | grep Version >> "$BUNDLE_DIR/summary.txt" || echo "cohere: not installed (pip)" >> "$BUNDLE_DIR/summary.txt"

# API key status (never log the actual key)
echo "--- Auth ---" >> "$BUNDLE_DIR/summary.txt"
if [ -n "${CO_API_KEY:-}" ]; then
  echo "CO_API_KEY: SET (${#CO_API_KEY} chars, starts with ${CO_API_KEY:0:4}...)" >> "$BUNDLE_DIR/summary.txt"
else
  echo "CO_API_KEY: NOT SET" >> "$BUNDLE_DIR/summary.txt"
fi

Step 3: Test API Connectivity

echo "--- API Connectivity ---" >> "$BUNDLE_DIR/summary.txt"

# Test each endpoint
for endpoint in chat embed rerank classify; do
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
    -X POST "https://api.cohere.com/v2/$endpoint" \
    -H "Authorization: Bearer ${CO_API_KEY:-invalid}" \
    -H "Content-Type: application/json" \
    -d '{}' 2>/dev/null || echo "UNREACHABLE")
  echo "$endpoint: HTTP $STATUS" >> "$BUNDLE_DIR/summary.txt"
done

# Check service status
echo "" >> "$BUNDLE_DIR/summary.txt"
echo "--…

Cohere Deploy Integration

Overview

Deploy Cohere API v2 applications to Vercel, Fly.io, and Google Cloud Run with proper secrets management and health checks.

Prerequisites

• Cohere production API key (not trial)
• Platform CLI installed (vercel, fly, or gcloud)
• Application tested locally with real API calls

Instructions

Vercel Deployment

# Add Cohere API key as Vercel environment variable
vercel env add CO_API_KEY production
# Paste your production key when prompted

# Deploy
vercel --prod

vercel.json:

{
  "env": {
    "CO_API_KEY": "@co_api_key"
  },
  "functions": {
    "api/**/*.ts": {
      "maxDuration": 30
    }
  }
}

Vercel API Route (streaming chat):

// api/chat/route.ts
import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2();

export async function POST(req: Request) {
  const { message } = await req.json();

  const stream = await cohere.chatStream({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: message }],
  });

  const encoder = new TextEncoder();
  const readable = new ReadableStream({
    async start(controller) {
      for await (const event of stream) {
        if (event.type === 'content-delta') {
          const text = event.delta?.message?.content?.text ?? '';
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ text })}\n\n`));
        }
      }
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });

  return new Response(readable, {
    headers: { 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-cache' },
  });
}

Fly.io Deployment

# Set Cohere API key
fly secrets set CO_API_KEY="your-production-key"

# Deploy
fly deploy

fly.toml:

app = "my-cohere-app"
primary_region = "iad"

[env]
  NODE_ENV = "production"

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 1

[[services.http_checks]]
  interval = 30000
  timeout = 5000
  path = "/api/health"

Google Cloud Run Deployment

#!/bin/bash
PROJECT_ID="${GOOGLE_CLOUD_PROJECT}"
SERVICE="cohere-app"
REGION="us-central1"

# Store key in Secret Manager
echo -n "$CO_API_KEY" | gcloud secrets create cohere-api-key --data-file=-

# Build and deploy
gcloud builds submit --tag gcr.io/$PR…

Cohere Enterprise RBAC

Overview

Configure enterprise-grade access control for Cohere API v2 with multi-team API key management, per-team model/budget restrictions, and audit trails.

Prerequisites

• Cohere production API keys
• Understanding of your team/service structure
• Secret management infrastructure

Cohere Access Model

Cohere uses API key-based access control (no built-in RBAC or SSO). Enterprise patterns are implemented in your application layer.

Instructions

Step 1: Multi-Team Key Strategy

// Each team gets their own API key for tracking and revocation
interface TeamConfig {
  name: string;
  apiKeyEnvVar: string;
  allowedModels: string[];
  maxTokensPerCall: number;
  dailyBudgetUSD: number;
}

const teamConfigs: Record<string, TeamConfig> = {
  search: {
    name: 'Search Team',
    apiKeyEnvVar: 'CO_API_KEY_SEARCH',
    allowedModels: ['embed-v4.0', 'rerank-v3.5', 'command-r-08-2024'],
    maxTokensPerCall: 1000,
    dailyBudgetUSD: 50,
  },
  chatbot: {
    name: 'Chatbot Team',
    apiKeyEnvVar: 'CO_API_KEY_CHATBOT',
    allowedModels: ['command-a-03-2025', 'command-r7b-12-2024'],
    maxTokensPerCall: 4096,
    dailyBudgetUSD: 200,
  },
  ml: {
    name: 'ML Team',
    apiKeyEnvVar: 'CO_API_KEY_ML',
    allowedModels: ['embed-v4.0', 'embed-multilingual-v3.0'],
    maxTokensPerCall: 500,
    dailyBudgetUSD: 100,
  },
};

Step 2: Team-Scoped Client Factory

import { CohereClientV2 } from 'cohere-ai';

const clients = new Map<string, CohereClientV2>();

export function getCohereForTeam(teamId: string): CohereClientV2 {
  if (!clients.has(teamId)) {
    const config = teamConfigs[teamId];
    if (!config) throw new Error(`Unknown team: ${teamId}`);

    const apiKey = process.env[config.apiKeyEnvVar];
    if (!apiKey) throw new Error(`${config.apiKeyEnvVar} not set for team ${teamId}`);

    clients.set(teamId, new CohereClientV2({ token: apiKey }));
  }
  return clients.get(teamId)!;
}

Step 3: Model Access Enforcement

function enforceModelAccess(teamId: string, requestedModel: string): void {
  const config = teamConfigs[teamId];
  if (!config) throw new Error(`Unknown …

Cohere Hello World

Overview

Three minimal working examples: Chat completion, text embedding, and search reranking. Each demonstrates a core Cohere API v2 endpoint.

Prerequisites

• Completed cohere-install-auth setup
• cohere-ai package installed
• CO_API_KEY environment variable set

Instructions

Example 1: Chat Completion

import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2();

async function chat() {
  const response = await cohere.chat({
    model: 'command-a-03-2025',
    messages: [
      { role: 'system', content: 'You are a helpful coding assistant.' },
      { role: 'user', content: 'Explain what a closure is in JavaScript in 2 sentences.' },
    ],
  });

  console.log(response.message?.content?.[0]?.text);
}

chat().catch(console.error);

Example 2: Text Embedding

async function embed() {
  const response = await cohere.embed({
    model: 'embed-v4.0',
    texts: ['Cohere builds enterprise AI', 'LLMs power modern search'],
    inputType: 'search_document',
    embeddingTypes: ['float'],
  });

  const vectors = response.embeddings.float;
  console.log(`Generated ${vectors.length} embeddings`);
  console.log(`Dimensions: ${vectors[0].length}`);
}

embed().catch(console.error);

Example 3: Search Reranking

async function rerank() {
  const response = await cohere.rerank({
    model: 'rerank-v3.5',
    query: 'What is machine learning?',
    documents: [
      'Machine learning is a subset of artificial intelligence.',
      'The weather today is sunny and warm.',
      'Deep learning uses neural networks with many layers.',
      'I enjoy cooking Italian food on weekends.',
    ],
    topN: 2,
  });

  for (const result of response.results) {
    console.log(`[${result.relevanceScore.toFixed(3)}] ${result.index}`);
  }
}

rerank().catch(console.error);

Example 4: Streaming Chat

async function streamChat() {
  const stream = await cohere.chatStream({
    model: 'command-a-03-2025',
    messages: [
      { role: 'user', content: 'Write a haiku about APIs.' },
    ],
  });

  for await (const event of stream) {
    if (event.type === 'content-delta') {
      process.stdout.write(event.delta?.message?.content?.text ?? '');
    }
  }
  console.log(); // newline
}

streamChat().catch(console.error);

Python Equivalents

import cohere

co = cohere.ClientV2()

# Chat
response = co.chat(
    model="command-a-03-2025",
    messages=[{"role": …

Cohere Incident Runbook

Overview

Rapid incident response procedures for Cohere API v2 outages. Covers triage, mitigation, communication, and postmortem for Chat, Embed, Rerank, and Classify endpoints.

Prerequisites

• Access to status.cohere.com
• kubectl access to production cluster
• Prometheus/Grafana access
• PagerDuty/Slack communication channels

Severity Levels

Quick Triage (Run These First)

# 1. Check Cohere service status
curl -s https://status.cohere.com/api/v2/status.json | jq '.status.description'

# 2. Test each endpoint directly
echo "--- Chat ---"
curl -s -o /dev/null -w "%{http_code}" \
  -X POST https://api.cohere.com/v2/chat \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"command-r7b-12-2024","messages":[{"role":"user","content":"ping"}]}'

echo -e "\n--- Embed ---"
curl -s -o /dev/null -w "%{http_code}" \
  -X POST https://api.cohere.com/v2/embed \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"embed-v4.0","texts":["test"],"input_type":"search_document","embedding_types":["float"]}'

echo -e "\n--- Rerank ---"
curl -s -o /dev/null -w "%{http_code}" \
  -X POST https://api.cohere.com/v2/rerank \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"rerank-v3.5","query":"test","documents":["a","b"]}'

# 3. Check our app health
curl -sf https://api.yourapp.com/api/health | jq '.cohere'

# 4. Check error rate (last 5 min)
curl -s "localhost:9090/api/v1/query?query=rate(cohere_errors_total[5m])" | jq '.data.result'

Decision Tree

Cohere API returning errors?
├─ YES: Is status.cohere.com showing incident?
│   ├─ YES → Cohere-side outage. Enable fallback. Monitor status page.
│   └─ NO → Check our API key and…

Cohere Install & Auth

Overview

Set up the Cohere SDK (v2) and configure authentication for Chat, Embed, Rerank, and Classify endpoints.

Prerequisites

• Node.js 18+ or Python 3.10+
• Package manager (npm, pnpm, or pip)
• Cohere account at dashboard.cohere.com
• API key from Cohere dashboard (trial keys are free, production keys require billing)

Instructions

Step 1: Install SDK

# Node.js / TypeScript
npm install cohere-ai

# Python
pip install cohere

Step 2: Configure API Key

# Set environment variable
export CO_API_KEY="your-api-key-here"

# Or create .env file (add .env to .gitignore!)
echo 'CO_API_KEY=your-api-key-here' >> .env

Key types:

• Trial key — free, rate-limited (5-20 calls/min per endpoint, 1000/month others)
• Production key — metered billing, 1000 calls/min all endpoints, unlimited monthly

Step 3: Verify Connection (TypeScript)

import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2({
  token: process.env.CO_API_KEY,
});

async function verify() {
  const response = await cohere.chat({
    model: 'command-a-03-2025',
    messages: [
      { role: 'user', content: 'Say "connection verified" and nothing else.' },
    ],
  });
  console.log('Status:', response.message?.content?.[0]?.text);
}

verify().catch(console.error);

Step 4: Verify Connection (Python)

import cohere
import os

co = cohere.ClientV2(api_key=os.environ.get("CO_API_KEY"))

response = co.chat(
    model="command-a-03-2025",
    messages=[
        {"role": "user", "content": "Say 'connection verified' and nothing else."}
    ],
)
print("Status:", response.message.content[0].text)

Available Models

Cohere Local Dev Loop

Overview

Set up a fast, reproducible local development workflow with Cohere API v2 mocking, vitest testing, and hot reload.

Prerequisites

• Completed cohere-install-auth setup
• Node.js 18+ with npm/pnpm
• TypeScript project with tsx or ts-node

Instructions

Step 1: Project Structure

my-cohere-project/
├── src/
│   ├── cohere/
│   │   ├── client.ts       # CohereClientV2 wrapper
│   │   ├── chat.ts         # Chat completions
│   │   ├── embed.ts        # Embedding operations
│   │   └── rerank.ts       # Reranking operations
│   └── index.ts
├── tests/
│   ├── chat.test.ts
│   ├── embed.test.ts
│   └── fixtures/
│       └── responses.ts    # Mock API responses
├── .env.local              # Local secrets (git-ignored)
├── .env.example            # Template for team
└── package.json

Step 2: Package Setup

{
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "test": "vitest",
    "test:watch": "vitest --watch",
    "test:integration": "COHERE_INTEGRATION=1 vitest --run"
  },
  "dependencies": {
    "cohere-ai": "^7.0.0"
  },
  "devDependencies": {
    "tsx": "^4.0.0",
    "vitest": "^2.0.0",
    "typescript": "^5.5.0"
  }
}

Step 3: Client Wrapper

// src/cohere/client.ts
import { CohereClientV2 } from 'cohere-ai';

let instance: CohereClientV2 | null = null;

export function getCohere(): CohereClientV2 {
  if (!instance) {
    instance = new CohereClientV2({
      token: process.env.CO_API_KEY,
    });
  }
  return instance;
}

// Reset for testing
export function resetClient(): void {
  instance = null;
}

Step 4: Mock Fixtures

// tests/fixtures/responses.ts
export const mockChatResponse = {
  id: 'test-chat-id',
  message: {
    role: 'assistant' as const,
    content: [{ type: 'text' as const, text: 'Mocked response' }],
  },
  finishReason: 'COMPLETE' as const,
  usage: { billedUnits: { inputTokens: 10, outputTokens: 5 } },
};

export const mockEmbedResponse = {
  id: 'test-embed-id',
  embeddings: {
    float: [[0.1, 0.2, 0.3, 0.4]], // truncated for dev
  },
  meta: { billedUnits: { inputTokens: 4 } },
};

export const mockRerankResponse = {
  id: 'test-rerank-id',
  results: [
    { index: 0, relevanceScore: 0.95 },
    { index: 2, relevanceScore: 0.72 },
  ],
  meta: { billedUnits: { searchUnits: 1 } },
};

Step 5: Test with Mocks

// tests/chat.test.ts
import { describe, it, expect, vi, beforeEach } fro…

Cohere Migration Deep Dive

Overview

Comprehensive guide for migrating to Cohere from OpenAI, Anthropic, or other LLM providers, including embedding re-vectorization, prompt adaptation, and gradual traffic shifting.

Prerequisites

• Current LLM integration documented
• Cohere API key and SDK installed
• Feature flag infrastructure
• Rollback strategy

Migration Types

Instructions

Step 1: OpenAI to Cohere Chat Migration

// --- OpenAI (before) ---
import OpenAI from 'openai';
const openai = new OpenAI();

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are helpful.' },
    { role: 'user', content: 'Hello' },
  ],
  max_tokens: 500,
  temperature: 0.7,
});
const text = response.choices[0].message.content;

// --- Cohere (after) ---
import { CohereClientV2 } from 'cohere-ai';
const cohere = new CohereClientV2();

const response = await cohere.chat({
  model: 'command-a-03-2025',   // GPT-4o equivalent
  messages: [
    { role: 'system', content: 'You are helpful.' },  // Same format!
    { role: 'user', content: 'Hello' },
  ],
  maxTokens: 500,               // camelCase, not snake_case
  temperature: 0.7,
});
const text = response.message?.content?.[0]?.text;  // Different response shape

Step 2: Embedding Migration

// OpenAI embeddings: 3072 dims (text-embedding-3-large)
// Cohere embeddings: 1024 dims (embed-v4.0)
// IMPORTANT: You CANNOT mix embeddings from different models in the same vector DB

// Migration plan:
// 1. Create new vector collection with Cohere dimensions
// 2. Re-embed all documents with Cohere
// 3. Switch queries to new collection
// 4. Delete old collection

async function migrateEmbeddings(
  documents: Array<{ id: string; text: string }>,
  batchSize = 96
) {
  const cohere = new CohereClientV2();
  let processed = 0;

  for (let i = 0; i < documents.length; i += batchSize) {
    const batch = documents.slice(i, i + batchSize);

    const response = await cohere.embed({
      model: 'embed-v4.0',
      texts: batch.map(d => d.text),
      inputType: 'search_document',
 …

Cohere Multi-Environment Setup

Overview

Configure Cohere API v2 across dev/staging/prod with environment-specific API keys, model selection, and budget controls.

Prerequisites

• Separate Cohere API keys per environment (trial for dev, production for staging/prod)
• Secret management solution (Vault, AWS Secrets Manager, GCP Secret Manager)
• Environment detection in application

Environment Strategy

Instructions

Step 1: Configuration Structure

// src/config/cohere.ts
type Environment = 'development' | 'staging' | 'production';

interface CohereEnvConfig {
  chatModel: string;
  embedModel: string;
  rerankModel: string;
  maxTokens: number;
  cacheEnabled: boolean;
  cacheTtlMs: number;
  retries: number;
  timeoutSeconds: number;
}

const configs: Record<Environment, CohereEnvConfig> = {
  development: {
    chatModel: 'command-r7b-12-2024',    // Fastest, cheapest
    embedModel: 'embed-v4.0',
    rerankModel: 'rerank-v3.5',
    maxTokens: 200,                       // Limit for dev
    cacheEnabled: false,                   // See real responses
    cacheTtlMs: 0,
    retries: 1,                            // Fail fast in dev
    timeoutSeconds: 30,
  },
  staging: {
    chatModel: 'command-r-08-2024',       // Mid-tier for testing
    embedModel: 'embed-v4.0',
    rerankModel: 'rerank-v3.5',
    maxTokens: 1000,
    cacheEnabled: true,
    cacheTtlMs: 5 * 60 * 1000,           // 5 minutes
    retries: 3,
    timeoutSeconds: 60,
  },
  production: {
    chatModel: 'command-a-03-2025',       // Best quality
    embedModel: 'embed-v4.0',
    rerankModel: 'rerank-v3.5',
    maxTokens: 4096,
    cacheEnabled: true,
    cacheTtlMs: 15 * 60 * 1000,          // 15 minutes
    retries: 5,
    timeoutSeconds: 120,
  },
};

function detectEnvironment(): Environment {
  const env = process.env.NODE_ENV ?? 'development';
  if (['development', 'staging', 'production'].includes(env)) {
    return env as Environment;
  }
  return 'development';
}

export function getCohereConfig(): CohereEnvConfig & { environment: Environment } {
  const env = detectEnvironment();
  return { ...configs[env], environment: env };
}

Step 2: Environment-Aware Client

Cohere Observability

Overview

Set up production observability for Cohere API v2 with Prometheus metrics, OpenTelemetry tracing, and AlertManager rules. Tracks per-endpoint latency, token usage, error rates, and costs.

Prerequisites

• Prometheus or compatible metrics backend
• OpenTelemetry SDK installed
• cohere-ai SDK v7+

Instructions

Step 1: Metrics Collection

import { Registry, Counter, Histogram, Gauge } from 'prom-client';

const registry = new Registry();

// Per-endpoint request counter
const requestCounter = new Counter({
  name: 'cohere_requests_total',
  help: 'Total Cohere API requests',
  labelNames: ['endpoint', 'model', 'status'],
  registers: [registry],
});

// Latency histogram
const requestDuration = new Histogram({
  name: 'cohere_request_duration_seconds',
  help: 'Cohere request duration',
  labelNames: ['endpoint', 'model'],
  buckets: [0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30],
  registers: [registry],
});

// Token usage tracking
const tokenCounter = new Counter({
  name: 'cohere_tokens_total',
  help: 'Total tokens consumed',
  labelNames: ['endpoint', 'model', 'direction'], // direction: input|output
  registers: [registry],
});

// Error counter by type
const errorCounter = new Counter({
  name: 'cohere_errors_total',
  help: 'Cohere errors by status code',
  labelNames: ['endpoint', 'status_code'],
  registers: [registry],
});

// Rate limit headroom
const rateLimitGauge = new Gauge({
  name: 'cohere_rate_limit_remaining',
  help: 'Remaining rate limit capacity',
  labelNames: ['endpoint'],
  registers: [registry],
});

Step 2: Instrumented Client Wrapper

import { CohereClientV2, CohereError, CohereTimeoutError } from 'cohere-ai';

const cohere = new CohereClientV2();

async function instrumentedCall<T>(
  endpoint: string,
  model: string,
  operation: () => Promise<T>
): Promise<T> {
  const timer = requestDuration.startTimer({ endpoint, model });

  try {
    const result = await operation();
    requestCounter.inc({ endpoint, model, status: 'success' });
    timer();

    // Track tokens from response
    const usage = (result as any)?.usage?.billedUnits;
    if (usage) {
      if (usage.inputTokens) {
        tokenCounter.inc({ endpoint, model, direction: 'input' }, usage.inputTokens);
      }
      if (usage.outputTokens) {
        tokenCounter.inc({ endpoint, model, direction: 'output' }, usage.outputTokens);
      }
    }

    return result;
  } catch (err) {
    requestCounter.inc({ endpoint, model, status: 'error' });
    timer();

    if (err instanceof CohereError) {
      errorCoun…

Cohere Performance Tuning

Overview

Optimize Cohere API v2 performance through model selection, embedding batches, rerank pipelines, caching, and streaming for time-to-first-token.

Prerequisites

• cohere-ai SDK installed
• Understanding of Cohere endpoints (Chat, Embed, Rerank)
• Redis or in-memory cache (optional)

Latency Benchmarks (Typical)

Instructions

Strategy 1: Model Selection by Latency Budget

// Use smaller models for latency-sensitive paths
function selectModel(latencyBudgetMs: number): string {
  if (latencyBudgetMs < 1000) return 'command-r7b-12-2024';   // 7B, fastest
  if (latencyBudgetMs < 3000) return 'command-r-08-2024';      // Mid-tier
  return 'command-a-03-2025';                                    // Best quality
}

// Pair with maxTokens to control output length
await cohere.chat({
  model: selectModel(1500),
  messages: [{ role: 'user', content: query }],
  maxTokens: 200,  // Shorter output = lower latency
});

Strategy 2: Streaming for Time-to-First-Token

// Non-streaming: user waits for entire response (800ms-5s)
// Streaming: first token arrives in ~200ms

async function streamForUI(message: string): Promise<string> {
  const stream = await cohere.chatStream({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: message }],
  });

  let fullText = '';
  for await (const event of stream) {
    if (event.type === 'content-delta') {
      const text = event.delta?.message?.content?.text ?? '';
      fullText += text;
      // Emit to frontend immediately — perceived latency drops to ~200ms
    }
  }
  return fullText;
}

Strategy 3: Batch Embeddings (96 per Call)

// BAD: 1000 texts = 1000 API calls
for (const text of texts) {
  await cohere.embed({ model: 'embed-v4.0', texts: [text], ... });
}

// GOOD: 1000 texts = 11 API calls (96 per batch)
async function batchEmbed(texts: string[]): Promise<number[][]> {
  con…

Cohere Production Checklist

Overview

Complete go-live checklist for deploying Cohere API v2 integrations to production with safety gates, health checks, and rollback procedures.

Prerequisites

• Staging environment tested and verified
• Production API key (not trial) from dashboard.cohere.com
• Deployment pipeline configured
• Monitoring and alerting ready

Checklist

API & Authentication

• [ ] Using production API key (not trial — trial is rate-limited to 20 calls/min)
• [ ] CO_API_KEY stored in secret manager (Vault, AWS Secrets Manager, GCP Secret Manager)
• [ ] Key rotation procedure documented and tested
• [ ] Billing alerts configured at dashboard.cohere.com
• [ ] Using API v2 endpoints (CohereClientV2, not CohereClient)

Code Quality

• [ ] All API calls specify model parameter explicitly
• [ ] embeddingTypes set for all Embed calls (required for v3+)
• [ ] inputType set for all Embed calls (required for v3+)
• [ ] Error handling catches CohereError and CohereTimeoutError
• [ ] Retry logic with exponential backoff for 429 and 5xx
• [ ] No hardcoded API keys in source code
• [ ] Request/response logging excludes API keys and PII

Model Selection

• [ ] Correct model IDs used (not deprecated names):

Performance

• [ ] Embed calls batched (up to 96 texts per request)
• [ ] Rerank calls limited to 1000 documents per request
• [ ] Streaming enabled for user-facing chat (chatStream)
• [ ] Connection pooling / keep-alive configured
• [ ] Response caching for repeated embed/rerank queries
• [ ] maxTokens set to prevent runaway generation costs

Health Check Endpoint

// /api/health
import { CohereClientV2, CohereError } from 'cohere-ai';

const cohere = new CohereClientV2();

export async function GET() {
  const start = Date.now();
  let cohereStatus: 'healthy' | 'degraded' | 'down' = 'down';

  try {
    // Cheapest pos…

Cohere Rate Limits

Overview

Handle Cohere rate limits with exponential backoff, request queuing, and proactive throttling. Real rate limits from Cohere's documentation.

Prerequisites

• cohere-ai SDK installed
• Understanding of async/await patterns

Actual Cohere Rate Limits

Trial keys are free. Production keys require billing at dashboard.cohere.com.

Instructions

Step 1: Exponential Backoff with Jitter

import { CohereError, CohereTimeoutError } from 'cohere-ai';

interface RetryConfig {
  maxRetries: number;
  baseDelayMs: number;
  maxDelayMs: number;
}

const DEFAULT_RETRY: RetryConfig = {
  maxRetries: 5,
  baseDelayMs: 1000,
  maxDelayMs: 60_000,
};

async function withBackoff<T>(
  operation: () => Promise<T>,
  config = DEFAULT_RETRY
): Promise<T> {
  for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
    try {
      return await operation();
    } catch (err) {
      if (attempt === config.maxRetries) throw err;

      // Only retry on rate limits (429) and server errors (5xx)
      let shouldRetry = false;
      let retryAfterMs: number | undefined;

      if (err instanceof CohereError) {
        if (err.statusCode === 429) {
          shouldRetry = true;
          // Cohere returns Retry-After header (seconds)
          // SDK may expose this via err.headers
        } else if (err.statusCode && err.statusCode >= 500) {
          shouldRetry = true;
        }
      } else if (err instanceof CohereTimeoutError) {
        shouldRetry = true;
      }

      if (!shouldRetry) throw err;

      // Exponential delay with jitter
      const exponential = config.baseDelayMs * Math.pow(2, attempt);
      const jitter = Math.random() * config.baseDelayMs;
      const delay = Math.min(exponential + jitter, config.maxDelayMs);

      console.warn(`Cohere retry ${attempt + 1}/${config.maxRetries} in ${delay.toFixed(0)}ms`);
      await new Promise(r => setTimeout(r, retryAfterMs ?? delay));
    }
  }
  throw new Error('Unreachable');
}

Step 2: Request Queue (Concurrency-Limited)

import …

Cohere Reference Architecture

Overview

Production-ready architecture for Cohere API v2 applications covering RAG pipelines, tool-use agents, and multi-model orchestration.

Prerequisites

• Understanding of layered architecture
• cohere-ai SDK v7+
• TypeScript project with vitest

Project Structure

my-cohere-app/
├── src/
│   ├── cohere/
│   │   ├── client.ts           # CohereClientV2 singleton
│   │   ├── models.ts           # Model selection logic
│   │   ├── types.ts            # Cohere-specific types
│   │   └── errors.ts           # Error classification
│   ├── services/
│   │   ├── chat.ts             # Chat completions + streaming
│   │   ├── rag.ts              # RAG pipeline (embed → rerank → chat)
│   │   ├── agents.ts           # Tool-use agent loops
│   │   ├── embed.ts            # Batch embedding + caching
│   │   ├── rerank.ts           # Document reranking
│   │   └── classify.ts         # Few-shot classification
│   ├── tools/                  # Tool definitions for agents
│   │   ├── registry.ts         # Tool name → executor mapping
│   │   ├── search.ts
│   │   └── calculator.ts
│   ├── api/
│   │   ├── chat.ts             # POST /api/chat (streaming)
│   │   ├── embed.ts            # POST /api/embed
│   │   └── health.ts           # GET /api/health
│   └── cache/
│       └── embeddings.ts       # LRU cache for embeddings
├── tests/
│   ├── unit/
│   │   ├── chat.test.ts
│   │   ├── rag.test.ts
│   │   └── fixtures/           # Mocked API responses
│   └── integration/
│       └── cohere.test.ts      # Real API tests (gated)
├── config/
│   ├── models.json             # Model selection per environment
│   └── tools.json              # Tool definitions
└── package.json

Layer Architecture

┌─────────────────────────────────────────┐
│             API Layer                    │
│   (Express/Next.js routes, SSE stream)  │
├─────────────────────────────────────────┤
│           Service Layer                  │
│  (RAG pipeline, agent loop, classify)   │
├─────────────────────────────────────────┤
│          Cohere Layer                    │
│   (CohereClientV2, retry, model select) │
├─────────────────────────────────────────┤
│         Infrastructure Layer             │
│    (Embed cache, tool registry, queue)   │
└─────────────────────────────────────────┘

Core Components

Client Layer

// src/cohere/client.ts
import { CohereClientV2, CohereError, CohereTimeoutError } from 'cohere-ai';

let instance: CohereClientV2 | null = null;

export function getCohere(): CohereClientV2 {
  if (!instance) {
    instance = new CohereClientV2({
      token: process.env.CO_API_KEY,
    });
  }
  return instance;
}

// src/cohere/models.ts
export const MODELS = {
  chat: {
    premium: 'command-a-03-2025',
    standard: …

Cohere SDK Patterns

Overview

Production-ready patterns for the cohere-ai TypeScript SDK (CohereClientV2) and Python cohere package. Real model names, real API shapes, real error types.

Prerequisites

• cohere-ai v7+ installed (TypeScript) or cohere v5+ (Python)
• Familiarity with async/await patterns
• Understanding of Cohere API v2 endpoints

Instructions

Pattern 1: Singleton Client with Retry

// src/cohere/client.ts
import { CohereClientV2, CohereError, CohereTimeoutError } from 'cohere-ai';

let instance: CohereClientV2 | null = null;

export function getCohere(): CohereClientV2 {
  if (!instance) {
    if (!process.env.CO_API_KEY) {
      throw new Error('CO_API_KEY environment variable is required');
    }
    instance = new CohereClientV2({
      token: process.env.CO_API_KEY,
    });
  }
  return instance;
}

export async function withRetry<T>(
  operation: () => Promise<T>,
  maxRetries = 3,
  baseDelayMs = 1000
): Promise<T> {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await operation();
    } catch (err) {
      if (attempt === maxRetries) throw err;

      // Only retry on rate limits (429) and server errors (5xx)
      if (err instanceof CohereError) {
        const status = err.statusCode;
        if (status && status !== 429 && status < 500) throw err;
      } else if (!(err instanceof CohereTimeoutError)) {
        throw err;
      }

      const delay = baseDelayMs * Math.pow(2, attempt - 1);
      const jitter = Math.random() * 500;
      await new Promise(r => setTimeout(r, delay + jitter));
    }
  }
  throw new Error('Unreachable');
}

Pattern 2: Type-Safe Chat Wrapper

// src/cohere/chat.ts
import { getCohere, withRetry } from './client';

interface ChatOptions {
  message: string;
  systemPrompt?: string;
  model?: string;
  maxTokens?: number;
  temperature?: number;
  documents?: Array<{ id?: string; data: Record<string, string> }>;
}

export async function chat(options: ChatOptions): Promise<string> {
  const cohere = getCohere();

  const messages: Array<{ role: string; content: string }> = [];

  if (options.systemPrompt) {
    messages.push({ role: 'system', content: options.systemPrompt });
  }
  messages.push({ role: 'user', content: options.message });

  const response = await withRetry(() =>
    cohere.chat({
      model: options.model ?? 'command-a-03-2025',
      messages,
      maxTokens: options.maxTokens,
      temperature: options.temperature,
      documents: options.documents,
    })
  );

  return response.message?.content?.[0]?.text ?? '';
}

Pattern 3: Streaming Chat

Cohere Security Basics

Overview

Security best practices for Cohere API keys, request validation, and data protection. Cohere uses bearer token auth with trial and production key tiers.

Prerequisites

• Cohere account at dashboard.cohere.com
• Understanding of environment variables
• Secret management solution for production

Instructions

Step 1: API Key Management

# NEVER hardcode keys — use environment variables
export CO_API_KEY="your-key-here"

# .env file (MUST be git-ignored)
CO_API_KEY=your-key-here

# .gitignore (mandatory entries)
.env
.env.local
.env.*.local

Key types:

• Trial keys — free, rate-limited, for development only
• Production keys — metered billing, for live applications

Step 2: Runtime Validation

import { CohereClientV2 } from 'cohere-ai';

function createSecureClient(): CohereClientV2 {
  const apiKey = process.env.CO_API_KEY;

  if (!apiKey) {
    throw new Error('CO_API_KEY is required. Set it as an environment variable.');
  }

  // Basic key format check
  if (apiKey.length < 20) {
    throw new Error('CO_API_KEY appears malformed. Check dashboard.cohere.com.');
  }

  return new CohereClientV2({ token: apiKey });
}

Step 3: Key Rotation Procedure

# 1. Generate new key in Cohere dashboard
#    → dashboard.cohere.com → API Keys → Create new key

# 2. Deploy new key (keep old key active)
# Vercel:
vercel env add CO_API_KEY production

# AWS:
aws secretsmanager update-secret --secret-id cohere/api-key --secret-string "new-key"

# GCP:
echo -n "new-key" | gcloud secrets versions add cohere-api-key --data-file=-

# 3. Verify new key works
curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer NEW_KEY" \
  -H "Content-Type: application/json" \
  https://api.cohere.com/v2/chat \
  -d '{"model":"command-r7b-12-2024","messages":[{"role":"user","content":"test"}]}'
# Should return 200

# 4. Revoke old key in dashboard
# 5. Monitor for 401 errors after revocation

Step 4: Request Data Protection

// Scrub PII before sending to Cohere API
const PII_PATTERNS: [string, RegExp][] = [
  ['email', /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g],
  ['phone', /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g],
  ['ssn', /\b\d{3}-\d{2}-\d{4}\b/g],
];

function scrubPII(text: string): string {
  let scrubbed = text;
  for (const [type, regex] of PII_PATTERNS) {
    scrubbed = scrubbed.replace(regex, `[REDACTED_${type.toUpperCase()}]`);
  }
  return…

Cohere Upgrade & Migration

Overview

Guide for migrating from Cohere API v1 to v2 and upgrading the cohere-ai TypeScript / cohere Python SDK. Covers every breaking change with before/after code.

Prerequisites

• Current Cohere SDK installed
• Git for version control
• Test suite available

Instructions

Step 1: Check Current Version

# TypeScript
npm list cohere-ai

# Python
pip show cohere

# Latest versions
npm view cohere-ai version
pip index versions cohere

Step 2: Create Upgrade Branch

git checkout -b upgrade/cohere-v2-migration
npm install cohere-ai@latest
# or: pip install cohere --upgrade

Step 3: API v1 to v2 Breaking Changes

Client Import

// v1
import { CohereClient } from 'cohere-ai';
const cohere = new CohereClient({ token: '...' });

// v2
import { CohereClientV2 } from 'cohere-ai';
const cohere = new CohereClientV2({ token: '...' });

Chat Endpoint — Messages Format

// v1 — single message string
const response = await cohere.chat({
  message: 'Hello',
  preamble: 'You are helpful.',
  chatHistory: [
    { role: 'USER', message: 'Hi' },
    { role: 'CHATBOT', message: 'Hello!' },
  ],
});
console.log(response.text);

// v2 — OpenAI-compatible messages array, model required
const response = await cohere.chat({
  model: 'command-a-03-2025',  // REQUIRED in v2
  messages: [
    { role: 'system', content: 'You are helpful.' },
    { role: 'user', content: 'Hi' },
    { role: 'assistant', content: 'Hello!' },
    { role: 'user', content: 'Hello' },
  ],
});
console.log(response.message?.content?.[0]?.text);

Role Names

Embed Endpoint — Required Fields

// v1 — model optional, no embedding_types
const response = await cohere.embed({
  texts: ['hello'],
});

// v2 — model, inputType, embeddingTypes all REQUIRED
const response = await cohere.embed({
  model: 'embed-v4.0',          // Required
  texts: ['hello'],
  inputType: 'search_document', // Required for v3+ models
  embeddingTypes: ['float'],    // Required for v3+ models
});

Rerank Endpoint — Model Required

/…

Cohere Streaming Events & Connectors

Overview

Handle Cohere's streaming chat events (SSE), tool-call events, citation events, and register data connectors for RAG. Cohere does not use traditional webhooks — its event model is streaming-based.

Prerequisites

• cohere-ai SDK v7+
• Understanding of Server-Sent Events (SSE)
• For connectors: HTTPS endpoint accessible from internet

Instructions

Step 1: Chat Streaming Events

Cohere's chatStream returns a stream of typed events:

import { CohereClientV2 } from 'cohere-ai';

const cohere = new CohereClientV2();

async function handleStream(userMessage: string) {
  const stream = await cohere.chatStream({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: userMessage }],
  });

  for await (const event of stream) {
    switch (event.type) {
      // Text content streaming
      case 'content-start':
        console.log('--- Generation started ---');
        break;

      case 'content-delta':
        const text = event.delta?.message?.content?.text ?? '';
        process.stdout.write(text);
        break;

      case 'content-end':
        console.log('\n--- Generation complete ---');
        break;

      // Citation events (when using documents)
      case 'citation-start':
        console.log('Citation:', event.delta?.message?.citations);
        break;

      // Tool call events (when using tools)
      case 'tool-call-start':
        console.log('Tool call started:', event.delta?.message?.toolCalls?.function?.name);
        break;

      case 'tool-call-delta':
        // Streaming tool arguments
        break;

      case 'tool-call-end':
        console.log('Tool call complete');
        break;

      // Message lifecycle
      case 'message-start':
        console.log('Message ID:', event.id);
        break;

      case 'message-end':
        console.log('Finish reason:', event.delta?.finishReason);
        console.log('Usage:', event.delta?.usage);
        break;
    }
  }
}

Step 2: RAG Streaming with Citations

async function streamRAG(query: string, docs: string[]) {
  const stream = await cohere.chatStream({
    model: 'command-a-03-2025',
    messages: [{ role: 'user', content: query }],
    documents: docs.map((text, i) => ({
      id: `doc-${i}`,
      data: { text },
    })),
  });

  let fullText = '';
  const citations: Array<{ start: number; end: number; text: string; sources: string[] }> = [];

  for await (const event of stream) {
    if (event.type === 'content-delta') {
      const chunk = event.delta?.message?.content?.text ?? …