Langfuse Data Boundary Auditor

Overview

Compare supplied observation response fields with an explicitly reviewed owner policy and identify project/retention/coverage gaps. Unknown fields, empty containers and nested metadata are still data. A matching field policy is not anonymization, permission to publish, proof of deletion or legal compliance.

This operator produces an evidence ledger. It never creates an export, rewrites customer data, rotates credentials, changes retention or deletes records.

Prerequisites

• The reviewed purpose, project and absolute selection window.
• A privately reviewed field allowlist, evidence-age bound and retention limit.
• Native project lookup and observation pages with stable queries and cursor metadata, using the receipt contract.
• Separate access, dataset, audit-log, external-copy and retention-execution review references, or explicit gaps for missing evidence.
• Python 3.10+ and authorization to inspect the supplied material.

Offline analysis needs no authentication. Any separate read needs existing approved scope; a user role or key prefix does not establish API permissions. Keep the original receipt private even when its diagnostic output contains hashes.

Instructions

Step 1: Define what is being reviewed

Identify the exact captured observation selection and intended use. Preserve the source artifacts and mapping receipt. Do not present one page, a filtered query or an API access window as a complete historical project inventory.

State which stores and exports are outside this scan. Project lookup and review references are supplied evidence; the helper does not independently authenticate them or inspect the referenced controls.

Step 2: Review the field policy before scanning

Use exact JSON Pointer leaf paths with ~1 and ~0 escaping. The local * extension matches array positions only, not arbitrary object keys or subtrees. Empty objects, arrays and null remain leaves. Read the contract before approving paths.

Do not expand the allowlist automatically after a failure. A permitted string can still contain secrets or personal data; field approval is not value-level PII detection. Regex replacement, object spread and late updates cannot prove that raw content was never transmitted.

Step 3: Run the bounded audit

python3 scripts/audit-boundary.py private-boundary-review.json

Run from this skill's directory. Inspect unapproved path hashes/counts, project and window correspondence, query stability, terminal cursors, capture age and declared collection completeness. Resolve path hashes privately against the original evidence; do not copy raw keys or values into a shared report.

Step 4: Assess retention and other-store coverage

An omitted, null or zero retentionDays does not establish configured deletion. Positive configuration must fit the reviewed owner bound, but configuration alone is not evidence that deletion executed. Cloud access windows limit reads; they are not deletion policies.

Review saved dataset items, audit logs and external copies separately. Missing coverage references remain findings; present references still require actual evidence review. Trace deletion or retention cannot be generalized into a claim that all copies of a person's data are gone.

Source-side export-stage masking, permission enforcement and database/disk/blob encryption require their own evidence. Do not treat an ENCRYPTION_KEY setting as proof of full storage encryption or turn this report into a security/DoD certification.

Step 5: Return the bounded finding ledger

For each finding record its fixed code, private evidence pointer, impact, confidence, missing coverage and next check. Keep selected-query correspondence, source-side exposure, configured retention and actual deletion evidence separate. Any remediation or destructive request requires its own authorized workflow.

Output

Return langfuse-boundary-review.json and a supplied-data summary of field-policy correspondence, unapproved leaf/path counts, selection completeness, configured retention and independent coverage gaps. matches-supplied-policy is limited to the supplied payload and policy; it does not authorize sharing the input.

Examples

• An anonymized export still contains nested metadata: report unapproved leaves, preserve the original evidence privately and review actual values; do not declare the remaining payload anonymous.
• The UI shows only 30 days: obtain the configured project retention evidence and review other stores instead of claiming older records were deleted.
• Page one ends with a cursor: retain incomplete-selection findings; do not certify the export or deletion inventory as complete.
• Retention is configured, dataset review is absent: report the configuration separately from the unresolved dataset scope and execution evidence.

Error Handling

Exit 0 matches the supplied policy, exit 1 needs investigation, and exit 2 holds invalid/unreadable/oversized evidence with a fixed code. Preserve held and partial states. Do not manufacture review references, truncate a manifest, alter source values or relax the policy just to obtain a passing result.

Resources

• Receipt, field-path policy, bounds and primary references.
• Evaluation specification: no live privacy/deletion evaluation is implied.