Langfuse Cost Forensics

Overview

Explain the recorded cost supported by a supplied observation set and what prevents a complete-set subtotal. Missing cost is not zero, a first page is not a project total, and recorded USD cost is not a verified provider invoice. This operator diagnoses evidence; it does not route models or promise savings.

Prerequisites

• The disputed cost claim, reviewed project and exact absolute selection window.
• Native Observations API v2 pages with actual request selectors, usage fields, cursor metadata and collection times.
• The receipt contract and Python 3.10+.
• Permission to inspect the private evidence. Offline analysis needs no authentication; separate reads require existing approved project access.

Use Metrics API v2 for project-wide aggregate questions. Do not manufacture an aggregate by fetching only one observation page or issuing unbounded per-trace requests. Preserve the source and selection behind any supplied aggregate.

Instructions

Step 1: Define the cost question

Identify whether the claim concerns a captured observation set, a project aggregate, an inferred model price, an ingested cost or a provider invoice. Record time window, project, filters, fields and capture times. Do not assume the report and invoice measure the same population or accounting event.

Step 2: Map native pages and coverage

Read the contract. Retain the initial query, each opaque cursor and terminal omission. Usage fields must actually have been requested. Keep missing values missing and preserve explicit numeric zero.

Check inclusive start/exclusive end, exact project binding, compound trace/observation IDs, query stability and collection completion. The analyzer does not replay arbitrary advanced filters or authenticate supplied declarations. An open/future window or a missing page remains incomplete evidence.

Step 3: Run the deterministic audit

python3 scripts/inspect-costs.py private-cost-receipt.json

Run from this skill's directory. The standard-library analyzer makes no network requests and echoes no raw prompts, outputs, model names, trace IDs or cursors. Inspect eligible generation/embedding rows separately from other observations.

Use totalCost as recorded; do not sum overlapping costDetails entries or arbitrary token/audio/total usage components. Missing prices or usage do not justify guessed rates, and negative adjustments need their own interpretation.

Step 4: Interpret subtotals honestly

known_rows_recorded_cost_usd is a subtotal of known rows. The complete retrieved set subtotal is withheld when evidence has gaps. Neither is a project-wide invoice or a savings estimate. Model-group output is bounded and ordered by hashed identity, not a ranking of the most expensive models.

Investigate unavailable costs, omitted usage fields, mismatched scope, cursor gaps and pricing provenance independently. A stored cost can be ingested or inferred; matching numbers alone do not establish invoice reconciliation.

Step 5: Return the cost evidence ledger

For each discrepancy provide the finding code, private receipt pointer, affected count, arithmetic limit and next check. Separate confirmed receipt defects, observed recorded costs and hypotheses about provider pricing or billing. Further collection must stay within approved read scope.

Output

Return langfuse-cost-report.json with eligible/known/zero/missing coverage, known-row subtotal, complete-set subtotal when supported, bounded model groups, findings and source limitations. Name supplied-data mode and do not convert unknown cost into zero or claim that static coverage proves complete tracing.

Examples

• One row has null cost and one has zero: preserve unavailable versus explicit-zero categories; withhold a complete-set subtotal.
• Cursor remains after page one: report partial coverage rather than a monthly project total or a percentage of savings.
• Usage components include input, output and total: inspect their coverage without adding all three into a fabricated token count.
• Stored cost differs from an invoice: identify scope and ingested/inferred provenance gaps; do not invent a current model price to explain the difference.

Error Handling

Exit 0 means reviewable retrieved-set evidence, exit 1 requires investigation, and exit 2 holds malformed/unreadable/oversized input with a fixed code. Preserve the result, correct capture defects and retain original evidence. Do not alter cost observations, credentials or model routing as part of this analysis.

Resources

• Native receipt contract, numeric semantics and primary references.
• Metrics API: aggregate questions.
• Evaluation specification: no paid or live evaluation is implied.