Langfuse Observability

Overview

Set up monitoring for your Langfuse integration: Prometheus metrics for trace/generation throughput, Grafana dashboards, alert rules, and integration with Langfuse's built-in analytics dashboards and Metrics API.

Prerequisites

• Langfuse SDK integrated and producing traces
• For custom metrics: Prometheus + Grafana (or compatible stack)
• For Langfuse analytics: access to the Langfuse UI dashboard

Instructions

Step 1: Langfuse Built-In Dashboards

Langfuse provides pre-built dashboards in the UI at https://cloud.langfuse.com (or your self-hosted URL):

• Overview: Total traces, generations, scores, and errors
• Cost Dashboard: Token usage and costs over time, broken down by model, user, session
• Latency Dashboard: Response times across models and user segments
• Custom Dashboards: Build your own with the query engine (multi-level aggregations, filters by user/model/tag)

Accessing via Metrics API:

import { LangfuseClient } from "@langfuse/client";

const langfuse = new LangfuseClient();

// Fetch aggregated metrics programmatically
const traces = await langfuse.api.traces.list({
  fromTimestamp: new Date(Date.now() - 3600000).toISOString(), // Last hour
  limit: 100,
});

console.log(`Traces in last hour: ${traces.data.length}`);

// Get observations with cost data
const observations = await langfuse.api.observations.list({
  type: "GENERATION",
  fromTimestamp: new Date(Date.now() - 86400000).toISOString(),
  limit: 500,
});

const totalCost = observations.data.reduce(
  (sum, obs) => sum + (obs.calculatedTotalCost || 0), 0
);
console.log(`Total cost (24h): $${totalCost.toFixed(4)}`);

Step 2: Prometheus Metrics for Your App

Track the health of your Langfuse integration with custom Prometheus metrics:

// src/lib/langfuse-metrics.ts
import { Counter, Histogram, Gauge, Registry } from "prom-client";

const registry = new Registry();

export const metrics = {
  tracesCreated: new Counter({
    name: "langfuse_traces_created_total",
    help: "Total traces created",
    labelNames: ["status"],
    registers: [registry],
  }),

  generationDuration: new Histogram({
    name: "langfuse_generation_duration_seconds",
    help: "LLM generation latency",
    labelNames: ["model"],
    buckets: [0.1, 0.5, 1, 2, 5, 10, 30],
    registers: [registry],
  }),

  tokensUsed: new Counter({
    name: "langfuse_tokens_total",
    help: "Total tokens used",
    labelNames: ["model", "type"],
    registers: [registry],
  }),

  costUsd: new Counter({
    name: "langfuse

Langfuse Performance Tuning

Overview

Optimize Langfuse tracing for minimal overhead and maximum throughput: benchmark measurement, batch tuning, non-blocking patterns, payload optimization, sampling, and memory management.

Prerequisites

• Existing Langfuse integration
• Performance baseline to compare against
• Understanding of async patterns

Performance Targets

Instructions

Step 1: Benchmark Current Performance

// scripts/benchmark-langfuse.ts
import { performance } from "perf_hooks";
import { startActiveObservation, updateActiveObservation } from "@langfuse/tracing";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

async function benchmark() {
  const sdk = new NodeSDK({
    spanProcessors: [new LangfuseSpanProcessor()],
  });
  sdk.start();

  const iterations = 1000;

  // Measure trace creation
  const timings: number[] = [];
  for (let i = 0; i < iterations; i++) {
    const start = performance.now();
    await startActiveObservation(`bench-${i}`, async () => {
      updateActiveObservation({ input: { i }, output: { done: true } });
    });
    timings.push(performance.now() - start);
  }

  const sorted = timings.sort((a, b) => a - b);
  console.log("=== Langfuse Performance Benchmark ===");
  console.log(`Iterations: ${iterations}`);
  console.log(`Mean:  ${(sorted.reduce((a, b) => a + b) / sorted.length).toFixed(3)}ms`);
  console.log(`P50:   ${sorted[Math.floor(sorted.length * 0.5)].toFixed(3)}ms`);
  console.log(`P95:   ${sorted[Math.floor(sorted.length * 0.95)].toFixed(3)}ms`);
  console.log(`P99:   ${sorted[Math.floor(sorted.length * 0.99)].toFixed(3)}ms`);

  const flushStart = performance.now();
  await sdk.shutdown();
  console.log(`Flush: ${(performance.now() - flushStart).toFixed(1)}ms`);
}

benchmark();

Step 2: Optimize Batch Configuration

// v4+: Tune OTel span processor
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

const processor = new LangfuseSpanProcessor({
  exportIntervalMillis: 10000,  // Flush every 10s (default: 5000)
  maxExportBatchSize: 100,      // Larger batches = fewer API calls
  maxQueueSize: 4096,           // Buffer more events before dropping
});

const sdk = new NodeSDK({ spanProcessors: [processor] });
sdk.start();

Langfuse Production Checklist

Overview

Comprehensive checklist for deploying Langfuse observability to production with verified configuration, error handling, graceful shutdown, monitoring, and a pre-deployment verification script.

Prerequisites

• Development and staging testing completed
• Production Langfuse project created with separate API keys
• Secret management solution in place

Production Configuration

Recommended SDK Settings

// v4+ Production Config
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

const processor = new LangfuseSpanProcessor({
  exportIntervalMillis: 5000,  // Flush every 5s
  maxExportBatchSize: 50,      // Batch size
  maxQueueSize: 2048,          // Buffer limit
});

const sdk = new NodeSDK({ spanProcessors: [processor] });
sdk.start();

// Graceful shutdown on all signals
for (const signal of ["SIGTERM", "SIGINT", "SIGUSR2"]) {
  process.on(signal, async () => {
    await sdk.shutdown();
    process.exit(0);
  });
}

// v3 Legacy Production Config
import { Langfuse } from "langfuse";

const langfuse = new Langfuse({
  flushAt: 25,            // Balance between latency and efficiency
  flushInterval: 5000,    // 5 second flush interval
  requestTimeout: 15000,  // 15s timeout
  enabled: true,          // Explicitly enable
});

process.on("beforeExit", () => langfuse.shutdownAsync());
process.on("SIGTERM", () => langfuse.shutdownAsync().then(() => process.exit(0)));

Production Error Handling

import { observe, updateActiveObservation, startActiveObservation } from "@langfuse/tracing";

// Wrap all traced operations with error safety
const tracedEndpoint = observe({ name: "api-endpoint" }, async (req: Request) => {
  try {
    updateActiveObservation({
      input: { path: req.url, method: req.method },
      metadata: { userId: req.userId },
    });

    const result = await processRequest(req);

    updateActiveObservation({ output: { status: 200 } });
    return result;
  } catch (error) {
    // Log error to trace -- don't let tracing error mask app error
    try {
      updateActiveObservation({
        output: { error: String(error) },
        metadata: { level: "ERROR" },
      });
    } catch {
      // Tracing failure must never break the app
    }
    throw error;
  }
});

Pre-Deployment Verification Script

// scripts/verify-langfuse-prod.ts
import { LangfuseClient } from "@langfuse/client";
import { startActiveObservation, updateActiveObservation } from "@langfuse/tracing";

async function verify() {
  const checks: Array<{ na

Langfuse Rate Limits

Overview

Handle Langfuse API rate limits with optimized SDK batching, exponential backoff with jitter, concurrent request limiting, and configurable sampling for ultra-high-volume workloads.

Prerequisites

• Langfuse SDK installed and configured
• High-volume trace workload (1,000+ events/minute)

Instructions

Step 1: Optimize SDK Batching Configuration

The Langfuse SDK batches events internally before sending. Tuning batch settings is the first defense against rate limits.

// v3 Legacy: Direct configuration
import { Langfuse } from "langfuse";

const langfuse = new Langfuse({
  flushAt: 50,           // Events per batch (default: 15, max ~200)
  flushInterval: 10000,  // Milliseconds between flushes (default: 10000)
  requestTimeout: 30000, // Timeout per batch request
});

// v4+: Configure via OTel span processor
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

const processor = new LangfuseSpanProcessor({
  exportIntervalMillis: 10000, // Flush interval
  maxExportBatchSize: 50,      // Events per batch
});

const sdk = new NodeSDK({ spanProcessors: [processor] });
sdk.start();

Step 2: Implement Retry with Exponential Backoff

For custom API calls (scores, datasets, prompts) that hit rate limits:

async function withRetry<T>(
  fn: () => Promise<T>,
  options: { maxRetries?: number; baseDelayMs?: number; maxDelayMs?: number } = {}
): Promise<T> {
  const { maxRetries = 5, baseDelayMs = 1000, maxDelayMs = 30000 } = options;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error: any) {
      const status = error?.status || error?.response?.status;

      // Only retry on rate limits (429) and server errors (5xx)
      if (attempt === maxRetries || (status && status < 429)) {
        throw error;
      }

      // Honor Retry-After header if present
      const retryAfter = error?.response?.headers?.["retry-after"];
      let delay: number;

      if (retryAfter) {
        delay = parseInt(retryAfter, 10) * 1000;
      } else {
        // Exponential backoff with jitter
        delay = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
        delay += Math.random() * 500; // Jitter
      }

      console.warn(`Rate limited. Retry ${attempt + 1}/${maxRetries} in ${Math.round(delay)}ms`);
      await new Promise((r) => setTimeout(r, delay));
    }
  }
  throw new Error("Unreachable");
}

// Usage with Langfuse client operations
const langfuse = new LangfuseClient();

await withRetry(() =>
  langfuse.score.create({
    traceId: "trace-123",
    name: "quality",
    value: 0.95,
    dataType: "NUMERIC",
  })

Langfuse Reference Architecture

Overview

Production-grade architecture patterns for Langfuse LLM observability: singleton SDK, context propagation with AsyncLocalStorage, cross-service trace correlation, multi-environment configurations, and scale strategies.

Prerequisites

• Understanding of distributed systems and async patterns
• Node.js 18+ with OpenTelemetry SDK
• For v4+: @langfuse/tracing, @langfuse/otel, @opentelemetry/sdk-node

Architecture Tiers

Instructions

Pattern 1: Singleton SDK with Context Propagation

// src/lib/tracing.ts -- Single module for all tracing
import { LangfuseClient } from "@langfuse/client";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";
import { AsyncLocalStorage } from "async_hooks";

// Singleton OTel SDK
let sdk: NodeSDK | null = null;

export function initTracing() {
  if (sdk) return sdk;

  sdk = new NodeSDK({
    spanProcessors: [
      new LangfuseSpanProcessor({
        exportIntervalMillis: 5000,
        maxExportBatchSize: 50,
      }),
    ],
  });
  sdk.start();

  // Graceful shutdown
  for (const signal of ["SIGTERM", "SIGINT"]) {
    process.on(signal, async () => {
      console.log(`Received ${signal}, flushing traces...`);
      await sdk?.shutdown();
      process.exit(0);
    });
  }

  return sdk;
}

// Singleton client for non-tracing operations
let client: LangfuseClient | null = null;

export function getLangfuseClient(): LangfuseClient {
  if (!client) client = new LangfuseClient();
  return client;
}

// Request context for user/session tracking
interface RequestContext {
  userId?: string;
  sessionId?: string;
  requestId: string;
}

const requestStore = new AsyncLocalStorage<RequestContext>();

export function getRequestContext(): RequestContext | undefined {
  return requestStore.getStore();
}

export function runWithContext<T>(ctx: RequestContext, fn: () => T): T {
  return requestStore.run(ctx, fn);
}

Pattern 2: Express Middleware for Automatic Tracing

// src/middleware/tracing.ts
import { startActiveObservation, updateActiveObservation } from "@langfuse/tracing";
import { runWithContext, getRequestContext } from "../lib/tracing";
impo

Langfuse SDK Patterns

Overview

Production-quality patterns for the Langfuse SDK: singleton clients, the observe wrapper, startActiveObservation for nested traces, session tracking, graceful shutdown, and error-safe tracing.

Prerequisites

• Completed langfuse-install-auth setup
• Understanding of async/await patterns
• For v4+: @langfuse/tracing, @langfuse/otel, @opentelemetry/sdk-node

Instructions

Pattern 1: Singleton Client with Graceful Shutdown

// src/lib/langfuse.ts -- single file, import everywhere
import { LangfuseClient } from "@langfuse/client";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

// Singleton client for prompts, datasets, scores
let client: LangfuseClient | null = null;
export function getLangfuseClient(): LangfuseClient {
  if (!client) {
    client = new LangfuseClient();
  }
  return client;
}

// One-time OTel setup (call at app entry point)
let sdk: NodeSDK | null = null;
export function initTracing(): NodeSDK {
  if (!sdk) {
    sdk = new NodeSDK({
      spanProcessors: [new LangfuseSpanProcessor()],
    });
    sdk.start();

    // Graceful shutdown on process exit
    const shutdown = async () => {
      await sdk?.shutdown();
      process.exit(0);
    };
    process.on("SIGTERM", shutdown);
    process.on("SIGINT", shutdown);
  }
  return sdk;
}

Legacy v3 singleton:

import { Langfuse } from "langfuse";

let instance: Langfuse | null = null;

export function getLangfuse(): Langfuse {
  if (!instance) {
    instance = new Langfuse({
      flushAt: 15,
      flushInterval: 10000,
    });
    process.on("beforeExit", () => instance?.shutdownAsync());
  }
  return instance;
}

Pattern 2: observe Wrapper for Existing Functions

The observe wrapper is the most ergonomic way to add tracing. It wraps any function and auto-creates a span.

import { observe, updateActiveObservation } from "@langfuse/tracing";

// Wrap existing functions -- no internal changes needed
const fetchUserProfile = observe(async (userId: string) => {
  updateActiveObservation({ input: { userId } });
  const profile = await db.users.findById(userId);
  updateActiveObservation({ output: { found: !!profile } });
  return profile;
});

// Mark LLM calls as generations
const summarize = observe(
  { name: "summarize-text", asType: "generation" },
  async (text: string) => {
    updateActiveObservation({ model: "gpt-4o-mini", input: text });
    const result = await openai.chat.completions.create({
      model: "gpt-4o-min

Langfuse Security Basics

Overview

Security practices for Langfuse LLM observability: credential management, PII scrubbing before tracing, self-hosted hardening, data retention, and secret scanning.

Prerequisites

• Langfuse instance (cloud or self-hosted)
• API keys provisioned
• Understanding of data privacy requirements (GDPR, SOC2, HIPAA)

Instructions

Step 1: Credential Security

Langfuse uses two keys with different security profiles:

// Startup validation -- catch misconfigurations early
function validateLangfuseCredentials() {
  const publicKey = process.env.LANGFUSE_PUBLIC_KEY;
  const secretKey = process.env.LANGFUSE_SECRET_KEY;

  if (!publicKey || !secretKey) {
    throw new Error("LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are required");
  }

  // Catch key swap (common mistake)
  if (secretKey.startsWith("pk-lf-")) {
    throw new Error("LANGFUSE_SECRET_KEY contains a public key (pk-lf-). Keys are swapped.");
  }

  if (publicKey.startsWith("sk-lf-")) {
    throw new Error("LANGFUSE_PUBLIC_KEY contains a secret key (sk-lf-). Keys are swapped.");
  }

  return { publicKey, secretKey };
}

// Use validated credentials
const { publicKey, secretKey } = validateLangfuseCredentials();

Key security rules:

• Public key (pk-lf-...): Identifies the project. Safe in client-side code.
• Secret key (sk-lf-...): Grants write access. Server-side only.
• Store in environment variables or secret manager -- never in source code.
• Rotate keys immediately if exposed in logs, git, or error reports.

Step 2: PII Scrubbing Before Tracing

Langfuse stores everything you send. Scrub PII from inputs and outputs before tracing.

// src/lib/pii-scrubber.ts

const PII_PATTERNS: Array<{ regex: RegExp; replacement: string }> = [
  { regex: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z]{2,}\b/gi, replacement: "[EMAIL]" },
  { regex: /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g, replacement: "[PHONE]" },
  { regex: /\b\d{3}-\d{2}-\d{4}\b/g, replacement: "[SSN]" },
  { regex: /\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b/g, replacement: "[CARD]" },
  { regex: /\b(?:sk|pk)-[a-zA-Z0-9_-]{20,}\b/g, replacement: "[API_KEY]" },
];

export function scrubPII(text: string): string {
  let scrubbed = text;
  for (const { regex, replacement } of PII_PATTERNS) {
    scrubbed = scrubbed.replace(regex, replacement);
  }
  return scrubbed;
}

export function scrubObject(obj: any): any {
  if (typeof obj === "string") return scrubPII(obj);
  if (Array.isArray(obj)) return obj.map(scrubObject);
  if (typeof obj === "object" && obj !== null) {
    const result: Record<string, an

Langfuse Upgrade & Migration

Current State

!npm list langfuse @langfuse/client @langfuse/tracing @langfuse/otel 2>/dev/null | head -10 || echo 'No langfuse packages found' !pip show langfuse 2>/dev/null | grep -E "Name|Version" || echo 'Python langfuse not installed'

Overview

Step-by-step guide for upgrading the Langfuse SDK across major versions. Covers v3 to v4 (OTel rewrite), v4 to v5, breaking changes, and automated codemods.

Prerequisites

• Existing Langfuse integration
• Test suite covering traced operations
• Git branch for the upgrade

Version Roadmap

Instructions

Step 1: Check Current Version and Plan

set -euo pipefail
# Check what you have
npm list langfuse @langfuse/client @langfuse/tracing 2>/dev/null

# Check latest available
npm info @langfuse/client version
npm info @langfuse/tracing version
npm info langfuse version

# Python
pip show langfuse 2>/dev/null | grep Version
pip index versions langfuse 2>/dev/null | head -3

Step 2: v3 to v4 Migration (TypeScript)

This is the biggest migration -- v4 rewrites tracing on OpenTelemetry.

2a. Install new packages:

set -euo pipefail
# Install v4+ packages
npm install @langfuse/client @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node

# Keep langfuse v3 temporarily for comparison
# Remove after migration: npm uninstall langfuse

2b. Update initialization:

// BEFORE (v3):
import { Langfuse } from "langfuse";
const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  secretKey: process.env.LANGFUSE_SECRET_KEY,
  baseUrl: process.env.LANGFUSE_HOST,
});

// AFTER (v4+):
import { LangfuseClient } from "@langfuse/client";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { NodeSDK } from "@opentelemetry/sdk-node";

// OTel setup (once at entry point)
const sdk = new NodeSDK({
  spanProcessors: [new LangfuseSpanProcessor()],
});
sdk.start();

// Client for prompts, datasets, scores
const langfuse = new LangfuseClient();

2c. Update tracing calls: