Langfuse Trace Ingestion Medic

Overview

Explain which exported spans are visible in supplied Langfuse readbacks, which facts conflict, and what bounded check would resolve the remaining uncertainty. Distinguish local span creation, transport acceptance, background processing and per-span read visibility. None of those alone proves complete instrumentation.

This workflow reads evidence. It never reruns customer operations, replays an export, changes sampling, rotates keys, flushes a live SDK or deploys a service.

Prerequisites

• A precise symptom, affected project/region and absolute incident window.
• An existing authorized private capture of one OTLP JSON batch and its scoped Observations API v2 pages, or an explicit statement that this evidence is missing.
• Python 3.10+ and the receipt contract.
• Permission to inspect the evidence. The analyzer needs no authentication; any separate provider read needs existing approved access and scope.

Read runtime diagnostics for SDK compatibility, sampling, lifecycle and authentication checks. Do not enable raw debug logging or duplicate production exporters merely to obtain a fixture.

Instructions

Step 1: Bound the incident and evidence

Record the expected operation, trace/span identities, capture times, project, region and SDK/runtime versions privately. State whether the capture covers a single export batch, a synthetic canary or actual affected traffic. A canary cannot certify customer coverage. Preserve the original artifacts.

If no batch or readback exists, return an evidence gap and the smallest useful next check. Do not invent native pages or execute the business operation again. Binary protobuf and console-exporter text are not the helper's JSON contract.

Step 2: Map the supplied receipts

Follow the contract. Keep native resource/scope span nesting, exact IDs, parent IDs and nanosecond times. Retain the response for that same batch when available, including a partial-success envelope.

Readbacks must use the reviewed project and explicit trace/window selectors, stable queries and complete cursor chains. Request only necessary fields. A first page, old recent trace or successful auth check cannot prove recovery.

Step 3: Reconcile without retrying

python3 scripts/inspect-ingestion.py private-ingestion-receipt.json

Use the packaged script relative to this skill's directory. Keep the private receipt out of Git. The script returns content-free findings and counts; it does not query the provider or execute callbacks.

Inspect missing IDs, project/trace mismatches, cursor gaps, ended timing, physical-parent mismatches and export warnings independently. Timing matches are at millisecond resolution, not proof of nanosecond equality.

Step 4: Separate defects from hypotheses

A parent outside this batch is a capture boundary requiring review; it is not proof that Langfuse dropped it. An exported parent cycle or a different returned parent is separate evidence. A transport HTTP 200 or completed flush is not a per-span read acknowledgement. Missing readback rows remain “not observed in these pages,” not a universal freshness breach or proven data loss.

Inspect tracing enablement, sampling/export filters, compatibility and lifecycle configuration next. Preserve business exceptions. An instrumentation failure must never trigger a second call to the application callback.

Step 5: Return the evidence ledger

For each finding give the report code, private evidence pointer, consequence, confidence and next discriminating check. Preserve investigation and held states. Any further read must remain within existing authorization; production fixes require their own implementation and rollout review.

Output

Return langfuse-ingestion-report.json plus a short supplied-data assessment: expected/matched span counts, coverage limits, confirmed receipt defects, unresolved hypotheses and next check. observed-supplied-batch establishes only the captured batch's correspondence with supplied pages, not service health.

Examples

• Flush succeeded; one span is missing: report its absence from the supplied pages, check ended-span/export lifecycle and retained pagination; do not replay the order-processing function.
• Child appears at root: distinguish a parent absent from this batch from a different physical parent in the readback. Investigate export filtering and context propagation before blaming storage.
• Export accepted an empty batch: retain empty-export-batch; a successful transport response does not demonstrate ingestion.

Error Handling

Exit 0 means the supplied batch was observed; exit 1 needs investigation; exit 2 holds invalid or oversized input with a fixed error code. Correct the capture or mapping, not observed values. Missing credentials or native evidence is a gap, not permission to create keys, widen access or synthesize a passing report.

Resources

• Receipt contract and primary sources.
• Runtime, access and lifecycle diagnostics.
• Evaluation specification: proposed behavioral checks, not evidence that they ran.