Logging API Requests
Overview
Implement structured API request logging with correlation IDs, performance timing, security audit trails, and PII redaction. Capture request/response metadata in JSON format suitable for aggregation in ELK Stack, Loki, or CloudWatch Logs, enabling debugging, performance analysis, and compliance auditing across distributed services.
Prerequisites
- Structured logging library: Pino or Winston (Node.js), structlog (Python), Logback with JSON encoder (Java)
- Log aggregation system: ELK Stack (Elasticsearch, Logstash, Kibana), Grafana Loki, or CloudWatch Logs
- Correlation ID propagation mechanism (middleware-injected or from incoming
X-Request-ID header)
- PII data classification for the API domain (which fields contain personal data requiring redaction)
- Log retention and rotation policy defined per compliance requirements
Instructions
- Examine existing logging configuration using Grep and Read to identify current log format, output destinations, and any structured logging already in place.
- Implement request logging middleware that captures: timestamp (ISO 8601), correlation ID, HTTP method, URL path (without query string PII), status code, response time (ms), request size, response size, and client IP.
- Generate a unique correlation ID (
X-Request-ID) for each request if not provided by the caller, and propagate it to all downstream service calls and log entries within the request scope.
- Add PII redaction rules that mask sensitive fields (passwords, tokens, SSNs, email addresses) in logged request/response bodies using configurable field-path patterns.
- Implement log levels per context:
info for successful requests, warn for 4xx client errors, error for 5xx server errors with stack traces, and debug for request/response bodies (development only).
- Configure response body logging for error responses only (4xx/5xx), capturing the error payload for debugging while skipping successful response bodies to reduce log volume.
- Add security audit logging for sensitive operations: authentication attempts, permission changes, data exports, and admin actions, tagged with
audit: true for separate indexing.
- Set up log rotation and retention policies: 30 days for application logs, 90 days for audit logs, with automatic compression of logs older than 7 days.
- Write tests verifying that PII redaction works correctly, correlation IDs propagate through nested calls, and log output matches expected JSON structure.
See ${CLAUDESKILLDIR}/references/implementation.md for the full implementation guide.
Output
${CLAUDESKILLDIR}/src/middleware/request-logger.js - Structured request/response logging middleware