OpenEvidence Performance Tuning

Overview

OpenEvidence's clinical API handles evidence query response times, citation batch retrieval, and complex multi-condition query optimization. Clinical evidence queries can take 2-5 seconds as the system searches across thousands of medical studies and synthesizes responses. Citation batch retrieval for systematic reviews generates heavy load when fetching 50-200 references per query. Caching evidence responses, batching citation fetches, and optimizing query specificity reduces clinician wait times by 50-70% and keeps complex queries within acceptable latency bounds.

Caching Strategy

const cache = new Map<string, { data: any; expiry: number }>();
const TTL = { evidence: 1_800_000, citations: 3_600_000, queries: 300_000 };

async function cached(key: string, ttlKey: keyof typeof TTL, fn: () => Promise<any>) {
  const entry = cache.get(key);
  if (entry && entry.expiry > Date.now()) return entry.data;
  const data = await fn();
  cache.set(key, { data, expiry: Date.now() + TTL[ttlKey] });
  return data;
}
// Citations are stable (1hr). Evidence summaries update with new studies (30 min).

Batch Operations

async function fetchCitationsBatch(client: any, citationIds: string[], batchSize = 25) {
  const results = [];
  for (let i = 0; i < citationIds.length; i += batchSize) {
    const batch = citationIds.slice(i, i + batchSize);
    const res = await Promise.all(batch.map(id => client.getCitation(id)));
    results.push(...res);
    if (i + batchSize < citationIds.length) await new Promise(r => setTimeout(r, 500));
  }
  return results;
}

Connection Pooling

import { Agent } from 'https';
const agent = new Agent({ keepAlive: true, maxSockets: 6, maxFreeSockets: 3, timeout: 30_000 });
// Moderate socket count — evidence queries are sequential, citations parallel

Rate Limit Management

async function withRateLimit(fn: () => Promise<any>): Promise<any> {
  try { return await fn(); }
  catch (err: any) {
    if (err.status === 429) {
      const retryMs = parseInt(err.headers?.['retry-after'] || '10') * 1000;
      await new Promise(r => setTimeout(r, retryMs));
      return fn();
    }
    throw err;
  }
}

Monitoring

const metrics = { queries: 0, citationFetches: 0, cacheHits: 0, avgLatencyMs: 0, errors: 0 };
function track(op: 'query' | 'citation', startMs: number, cached: boolean) {
  metrics[op === 'query' ? 'queries' : 'citationFetches']++;
  const lat = Date.now() - startMs;
  metrics.avgLatencyMs = (metrics.avgLatencyMs * (metrics.queries - 1) + lat) / metrics.queries;
  if (cached) metrics.cacheHits++;
}

Performance Checklist

• [ ] Cache evidence responses with 30-min TTL (studies update periodically)
• [ ] Cache citation metadata with 1-hour TTL (stable once published)
• [ ] Batch citation retrieval in groups of 25 with 500ms pauses
• [ ] Use specific condition + intervention queries instead of broad searches
• [ ] Prefetch commonly queried drug interaction evidence
• [ ] Enable HTTP keep-alive for persistent API connections
• [ ] Monitor average query latency (target < 3s for simple queries)
• [ ] Set client timeout to 30s for complex multi-condition queries

Error Handling

Resources

• OpenEvidence Platform
• OpenEvidence API Docs

Next Steps

See openevidence-reference-architecture.