Hex CI Integration

Overview

Set up CI/CD for Hex data analytics integrations: run unit tests with mocked project run and connection responses on every PR, trigger live Hex project runs and validate outputs on merge to main. Hex provides collaborative data notebooks with scheduled runs and API-triggered execution, so CI pipelines verify data transform logic, trigger post-deploy dashboard refreshes, and monitor run status.

GitHub Actions Workflow

# .github/workflows/hex-ci.yml
name: Hex CI
on:
  pull_request:
    paths: ['src/hex/**', 'tests/**']
  push:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npm test -- --reporter=verbose

  trigger-hex-refresh:
    if: github.ref == 'refs/heads/main'
    needs: unit-tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npm run test:integration
        env:
          HEX_API_TOKEN: ${{ secrets.HEX_API_TOKEN }}
          HEX_PROJECT_ID: ${{ vars.HEX_PROJECT_ID }}

Mock-Based Unit Tests

// tests/hex-service.test.ts
import { describe, it, expect, vi } from 'vitest';
import { triggerProjectRun, getRunStatus } from '../src/hex-service';

vi.mock('../src/hex-client', () => ({
  HexClient: vi.fn().mockImplementation(() => ({
    runProject: vi.fn().mockResolvedValue({
      runId: 'run_abc123',
      projectId: 'proj_xyz',
      status: 'running',
      startedAt: '2026-04-01T10:00:00Z',
    }),
    getRunStatus: vi.fn().mockResolvedValue({
      runId: 'run_abc123',
      status: 'completed',
      elapsedMs: 4500,
      outputs: { row_count: 1250, last_updated: '2026-04-01T10:00:04Z' },
    }),
    listProjects: vi.fn().mockResolvedValue({
      projects: [{ id: 'proj_xyz', title: 'Revenue Dashboard' }],
    }),
  })),
}));

describe('Hex Service', () => {
  it('triggers a project run and returns run ID', async () => {
    const result = await triggerProjectRun('proj_xyz', { triggered_by: 'ci' });
    expect(result.runId).toBe('run_abc123');
    expect(result.status).toBe('running');
  });

  it('polls run status until complete', async () => {
    const status = await getRunStatus('run_abc123');
    expect(status.status).toBe('completed');
    expect(status.outputs.row_count).toBe(1250);
  });
});

Integration Tests

// tests/integration/hex.integration.test.ts
import { describe, it, expect } from 'vitest…

Hex Common Errors

Error Reference

401 Unauthorized

Cause: Token invalid, expired, or missing. Fix: Regenerate token in Hex workspace settings.

403 Forbidden — Read-Only Token

Cause: Token has "Read projects" scope but RunProject requires "Run projects". Fix: Create new token with "Run projects" scope.

404 Not Found — Project

Cause: Project ID wrong or project not published. Fix: Verify project ID. Only published projects can be run via API.

429 Too Many Requests

Cause: RunProject is limited to 20 requests/min, 60/hr. Fix: Queue runs with delays. See hex-rate-limits.

Run Status: ERRORED

Cause: SQL query, Python code, or connection error in the project. Fix: Open the project in Hex UI and check the error in the run history.

Run Status: KILLED

Cause: Run exceeded timeout or was manually cancelled. Fix: Optimize slow queries. Increase timeout in API trigger.

Quick Diagnostics

# Test token
curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $HEX_API_TOKEN" \
  https://app.hex.tech/api/v1/projects

# List recent runs for a project
curl -s -H "Authorization: Bearer $HEX_API_TOKEN" \
  https://app.hex.tech/api/v1/project/PROJECT_ID/runs | python3 -m json.tool

Overview

This reference triages project-run failures using status, aggregate telemetry, and opaque run IDs. It does not authorize copying SQL, cell output, data previews, or tokens into a ticket.

Prerequisites

• A safe sandbox probe, redacted correlation ID, endpoint class, and HTTP or terminal-run status.
• A scoped read-only diagnostic credential and named owner for any project, schedule, or access change.

Instructions

1. Classify authentication, authorization, project lookup, validation, quota, timeout, and terminal-run errors before changing configuration.
2. Reproduce once with the safe probe and capture only status, latency, quota, and opaque run ID.
3. Check scope, parameter schema, project state, quota, and downstream connection health in that order.
4. Make one reversible change at a time and escalate a redacted bundle when the error persists.

Output

Return error class, correlation ID, project scope, probe outcome, remediation attempted, and next owner. Do not include SQL, output, credentials, or identities.

Error Handling

Treat unknown destination, failed redaction, access expansion, or repeated non-idempotent run as a stop condition. Cancel or restore the prior revision rather than retrying with broader scope.

Hex Project Orchestration

Overview

Trigger Hex project runs from external orchestration tools (Airflow, Dagster, cron) with input parameters, status polling, and error handling. This is the primary integration pattern for embedding Hex in data pipelines.

Instructions

Step 1: Parameterized Project Runs

import 'dotenv/config';
const TOKEN = process.env.HEX_API_TOKEN!;
const BASE = 'https://app.hex.tech/api/v1';

interface RunConfig {
  projectId: string;
  inputParams?: Record<string, any>;
  updateCache?: boolean;
  killRunning?: boolean;
}

async function triggerRun(config: RunConfig) {
  const response = await fetch(`${BASE}/project/${config.projectId}/run`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${TOKEN}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({
      inputParams: config.inputParams || {},
      updateCacheResult: config.updateCache ?? true,
      killRunningExecution: config.killRunning ?? false,
    }),
  });
  if (!response.ok) throw new Error(`Trigger failed: ${response.status} ${await response.text()}`);
  return response.json();
}

Step 2: Synchronous Run Helper

async function runAndWait(config: RunConfig, timeoutMs = 600000): Promise<any> {
  const { runId, projectId } = await triggerRun(config);
  const startTime = Date.now();

  while (Date.now() - startTime < timeoutMs) {
    const res = await fetch(`${BASE}/project/${projectId}/run/${runId}`, {
      headers: { 'Authorization': `Bearer ${TOKEN}` },
    });
    const status = await res.json();

    switch (status.status) {
      case 'COMPLETED': return { success: true, runId, duration: Date.now() - startTime };
      case 'ERRORED': throw new Error(`Run ${runId} errored: ${status.statusMessage || 'unknown'}`);
      case 'KILLED': throw new Error(`Run ${runId} was killed`);
      default: await new Promise(r => setTimeout(r, 5000));
    }
  }
  throw new Error(`Run ${runId} timed out after ${timeoutMs}ms`);
}

Step 3: Pipeline Orchestration

// Run multiple Hex projects in sequence (data pipeline)
async function runPipeline(steps: RunConfig[]) {
  const results = [];
  for (const step of steps) {
    console.log(`Running: ${step.projectId}`);
    const result = await runAndWait(step);
    console.log(`Completed in ${result.duration}ms`);
    results.push(result);
  }
  return results;
}

// Example: ETL pipeline
await runPipeline([
  { projectId: 'extract-project-id', inputParams: { date: '2025-01-01' } },
  { projectId: 'transform-project-id' },
  { projectId: 'load-project-id', updateCache: true },
]);

Step 4: Cancel Long-Running Projects

as…

Hex Scheduled Runs & Admin API

Overview

Configure scheduled runs and manage workspace resources via the Hex Admin API. Scheduled runs execute projects on cron-based intervals. The Admin API manages users, groups, and data connections.

Instructions

Step 1: List Project Runs

const TOKEN = process.env.HEX_API_TOKEN!;
const BASE = 'https://app.hex.tech/api/v1';

async function getProjectRuns(projectId: string) {
  const response = await fetch(`${BASE}/project/${projectId}/runs`, {
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  const runs = await response.json();
  for (const run of runs) {
    console.log(`${run.runId}: ${run.status} (${run.startTime} → ${run.endTime || 'running'})`);
  }
  return runs;
}

Step 2: Scheduled Runs (via Hex UI + API Trigger)

Schedules are configured in the Hex UI. For API-based scheduling, use external cron:

// cron-trigger.ts — run via cron job or CI
import cron from 'node-cron';

// Daily at 6 AM UTC
cron.schedule('0 6 * * *', async () => {
  console.log('Triggering daily report...');
  await triggerRun({
    projectId: 'daily-report-project-id',
    inputParams: { date: new Date().toISOString().split('T')[0] },
    updateCache: true,
  });
});

// Weekly on Monday at 9 AM
cron.schedule('0 9 * * 1', async () => {
  await triggerRun({ projectId: 'weekly-summary-project-id' });
});

Step 3: User Management (Admin API)

// List workspace users
async function listUsers() {
  const response = await fetch(`${BASE}/workspace/users`, {
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  return response.json();
}

// List groups
async function listGroups() {
  const response = await fetch(`${BASE}/workspace/groups`, {
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  return response.json();
}

Step 4: Data Connection Management

// List configured data connections
async function listConnections() {
  const response = await fetch(`${BASE}/workspace/connections`, {
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  return response.json();
}

Scheduling Options

Prerequisites

• A named data/project owner, approved parameter and destination po…

Hex Cost Tuning

Overview

Hex pricing combines per-seat licensing with compute-based charges for notebook runs and data connections. Each scheduled or ad-hoc notebook execution consumes compute credits proportional to query complexity and data volume processed. Organizations running dozens of notebooks on hourly schedules — many producing identical results from unchanged data — accumulate unnecessary compute costs. Caching run results, optimizing schedules, and consolidating redundant notebooks are the highest-leverage cost reduction strategies.

Cost Breakdown

API Call Reduction

class HexRunOptimizer {
  private resultCache = new Map<string, { data: any; timestamp: number }>();
  private dataHashes = new Map<string, string>();

  async runIfChanged(projectId: string, runFn: () => Promise<any>): Promise<any> {
    const currentHash = await this.getSourceDataHash(projectId);
    if (this.dataHashes.get(projectId) === currentHash) {
      const cached = this.resultCache.get(projectId);
      if (cached) return cached.data; // Source unchanged — serve cached result
    }
    const result = await runFn();
    this.resultCache.set(projectId, { data: result, timestamp: Date.now() });
    this.dataHashes.set(projectId, currentHash);
    return result;
  }

  private async getSourceDataHash(projectId: string): Promise<string> {
    const res = await fetch(`/api/v1/project/${projectId}/status`);
    return (await res.json()).sourceDataHash;
  }
}

Usage Monitoring

class HexCostMonitor {
  private runs = new Map<string, number[]>();
  private weeklyBudget = 500; // max runs per week

  recordRun(projectId: string): void {
    const timestamps = this.runs.get(projectId) || [];
    timestamps.push(Date.now());
    this.runs.set(projectId, timestamps);
  }

  getWeeklyReport(): { totalRuns: number; byProject: Record<string, number> } {
    const weekAgo = Date.now() - 7 * 24 * 60 * 60 * 1000;
    const byProject: Record<string, number> = {};
    l…

Hex Debug Bundle

Overview

Collect Hex API connectivity status, project listing, run history, data connection health, and rate limit state into a single diagnostic archive. This bundle helps troubleshoot failed notebook runs, stale data connections, scheduled run failures, and API authentication issues.

Debug Collection Script

#!/bin/bash
set -euo pipefail
BUNDLE="debug-hex-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BUNDLE"

# Environment check
echo "=== Hex Debug Bundle ===" | tee "$BUNDLE/summary.txt"
echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$BUNDLE/summary.txt"
echo "HEX_API_KEY: ${HEX_API_KEY:+[SET]}" >> "$BUNDLE/summary.txt"

# API connectivity
HTTP=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer ${HEX_API_KEY}" \
  https://app.hex.tech/api/v1/projects 2>/dev/null || echo "000")
echo "API Status: HTTP $HTTP" >> "$BUNDLE/summary.txt"

# Project listing
curl -s -H "Authorization: Bearer ${HEX_API_KEY}" \
  "https://app.hex.tech/api/v1/projects" \
  > "$BUNDLE/projects.json" 2>&1 || true

# Recent runs (across projects)
curl -s -H "Authorization: Bearer ${HEX_API_KEY}" \
  "https://app.hex.tech/api/v1/runs?limit=10" \
  > "$BUNDLE/recent-runs.json" 2>&1 || true

# Data connections and rate limit headers
curl -s -D "$BUNDLE/rate-headers.txt" -H "Authorization: Bearer ${HEX_API_KEY}" \
  "https://app.hex.tech/api/v1/connections" > "$BUNDLE/connections.json" 2>&1 || true

tar -czf "$BUNDLE.tar.gz" "$BUNDLE" && rm -rf "$BUNDLE"
echo "Bundle: $BUNDLE.tar.gz"

Analyzing the Bundle

tar -xzf debug-hex-*.tar.gz
cat debug-hex-*/summary.txt                       # Auth + connectivity
jq '.[] | {id, title, status}' debug-hex-*/projects.json   # Project inventory
jq '.[] | {id, status, started_at}' debug-hex-*/recent-runs.json  # Run history
jq '.[] | {name, type, status}' debug-hex-*/connections.json      # Data source health

Common Issues

Hex Deploy Integration

Overview

Deploy Hex orchestration services that trigger project runs from web endpoints or cron jobs.

Instructions

Vercel — On-Demand Data Refresh

// api/refresh.ts
export default async function handler(req, res) {
  const response = await fetch(`https://app.hex.tech/api/v1/project/${process.env.HEX_PROJECT_ID}/run`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${process.env.HEX_API_TOKEN}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ inputParams: req.body || {}, updateCacheResult: true }),
  });
  res.json(await response.json());
}

vercel env add HEX_API_TOKEN production
vercel env add HEX_PROJECT_ID production

Cloud Run — Scheduled Orchestrator

gcloud run deploy hex-orchestrator \
  --image gcr.io/$PROJECT_ID/hex-orchestrator \
  --set-secrets=HEX_API_TOKEN=hex-api-token:latest \
  --timeout=600

Prerequisites

• An approved deployment change, secret references, project/destination allowlist, and immutable client/configuration revision.
• Safe canary project, baselines for health/latency/quota/output assertions, and a tested rollback/cancel artifact.

Output

Produce a deployment receipt with artifact digest, environment, canary project, health/latency/quota/aggregate assertion outcomes, owner approval, rollout state, and rollback reference. Exclude SQL, output, and secrets.

Error Handling

Halt for unknown project/destination, unauthorized scope, failed canary assertion, unbounded retry, or leaked output in telemetry. Cancel affected execution and restore the previous revision rather than bypassing a gate.

Examples

artifact=sha256:opaque; env=staging; canary=proj-sandbox-12; health=pass; assertions=pass; quota=pass; rollback=release-r31 supports controlled promotion.

Resources

• Hex API

Hex Hello World

Overview

List projects, trigger a project run, and poll for results using the Hex API. The core workflow is: trigger a run of a published project, poll for completion, then access the results.

Prerequisites

• Completed hex-install-auth setup
• At least one published Hex project

Instructions

Step 1: List Projects

// hello-hex.ts
import 'dotenv/config';

const TOKEN = process.env.HEX_API_TOKEN!;
const BASE = 'https://app.hex.tech/api/v1';

async function listProjects() {
  const response = await fetch(`${BASE}/projects`, {
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  const projects = await response.json();
  for (const p of projects) {
    console.log(`${p.name} (${p.projectId}) — ${p.status}`);
  }
  return projects;
}

Step 2: Trigger a Project Run

async function runProject(projectId: string, inputParams?: Record<string, any>) {
  const response = await fetch(`${BASE}/project/${projectId}/run`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${TOKEN}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({
      inputParams: inputParams || {},
      updateCacheResult: true,
    }),
  });
  const { runId, projectId: pid, runStatusUrl } = await response.json();
  console.log(`Run started: ${runId}`);
  console.log(`Status URL: ${runStatusUrl}`);
  return { runId, projectId: pid, runStatusUrl };
}

Step 3: Poll for Run Completion

async function waitForRun(projectId: string, runId: string, timeoutMs = 300000): Promise<any> {
  const startTime = Date.now();
  while (Date.now() - startTime < timeoutMs) {
    const response = await fetch(`${BASE}/project/${projectId}/run/${runId}`, {
      headers: { 'Authorization': `Bearer ${TOKEN}` },
    });
    const status = await response.json();
    console.log(`Run ${runId}: ${status.status}`);

    if (status.status === 'COMPLETED') return status;
    if (status.status === 'ERRORED' || status.status === 'KILLED') {
      throw new Error(`Run failed: ${status.status}`);
    }

    await new Promise(r => setTimeout(r, 5000)); // Poll every 5s
  }
  throw new Error('Run timed out');
}

Step 4: Complete Flow

async function main() {
  const projects = await listProjects();
  if (projects.length === 0) { console.log('No projects found'); return; }

  const project = projects[0];
  const { runId } = await runProject(project.projectId, { date: '2025-01-01' });
  const result = await waitForRun(project.projectId, runId);
  console.log('Run completed:', result);
}

main().catch(console.error);

Step 5: curl Quick Tes…

Hex Install & Auth

Overview

Configure Hex API authentication using OAuth 2.0 Bearer tokens. The Hex API at app.hex.tech/api/v1/ lets you programmatically trigger project runs, check status, manage users, and configure connections. Tokens are generated per-user in the Hex workspace settings.

Prerequisites

• Hex account (Team or Enterprise plan)
• Workspace admin access for API token generation
• At least one published Hex project

Instructions

Step 1: Generate API Token

1. Open Hex workspace settings
2. Navigate to API tokens section
3. Click New Token
4. Set description and expiration
5. Select scopes: "Read projects" and/or "Run projects"

Step 2: Configure Environment

# .env (NEVER commit)
HEX_API_TOKEN=hex_token_abc123...
HEX_WORKSPACE_URL=https://app.hex.tech

# .gitignore
.env
.env.local

Step 3: Verify Connection

// verify-hex.ts
import 'dotenv/config';

const TOKEN = process.env.HEX_API_TOKEN!;

async function verify() {
  const response = await fetch('https://app.hex.tech/api/v1/projects', {
    headers: { 'Authorization': `Bearer ${TOKEN}`, 'Content-Type': 'application/json' },
  });
  if (!response.ok) throw new Error(`Hex API ${response.status}`);
  const projects = await response.json();
  console.log(`Connected! Found ${projects.length} projects`);
  return projects;
}

verify().catch(console.error);

# curl verification
curl -s -H "Authorization: Bearer $HEX_API_TOKEN" \
  https://app.hex.tech/api/v1/projects | python3 -m json.tool

Token Scopes

Error Handling

Output

Return an authentication setup receipt with environment, secret reference rather than value, token-scope class, approved project/destination result, synthetic connectivity status, and rotation/cleanup owner. Never print an access token, SQL, or output.

Examples

…

Hex Local Dev Loop

Overview

Set up a development workflow for Hex API orchestration with mocked API responses and testing.

Instructions

Step 1: Project Structure

hex-orchestrator/
├── src/hex/
│   ├── client.ts       # API client
│   ├── orchestrator.ts # Pipeline runner
│   └── types.ts        # TypeScript interfaces
├── tests/
│   ├── fixtures/       # Mock API responses
│   └── orchestrator.test.ts
├── .env.local
└── package.json

Step 2: Typed Hex Client

// src/hex/client.ts
export class HexClient {
  constructor(private token: string, private baseUrl = 'https://app.hex.tech/api/v1') {}

  async listProjects() {
    return this.get('/projects');
  }

  async runProject(projectId: string, inputParams?: Record<string, any>) {
    return this.post(`/project/${projectId}/run`, { inputParams: inputParams || {}, updateCacheResult: true });
  }

  async getRunStatus(projectId: string, runId: string) {
    return this.get(`/project/${projectId}/run/${runId}`);
  }

  async cancelRun(projectId: string, runId: string) {
    return this.delete(`/project/${projectId}/run/${runId}`);
  }

  private async get(path: string) {
    const res = await fetch(`${this.baseUrl}${path}`, { headers: { 'Authorization': `Bearer ${this.token}` } });
    if (!res.ok) throw new Error(`Hex API ${res.status}`);
    return res.json();
  }

  private async post(path: string, body: any) {
    const res = await fetch(`${this.baseUrl}${path}`, { method: 'POST', headers: { 'Authorization': `Bearer ${this.token}`, 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
    if (!res.ok) throw new Error(`Hex API ${res.status}`);
    return res.json();
  }

  private async delete(path: string) {
    const res = await fetch(`${this.baseUrl}${path}`, { method: 'DELETE', headers: { 'Authorization': `Bearer ${this.token}` } });
    return res.ok;
  }
}

Step 3: Mocked Tests

import { describe, it, expect, vi } from 'vitest';
const mockFetch = vi.fn();
vi.stubGlobal('fetch', mockFetch);

describe('HexClient', () => {
  it('should list projects', async () => {
    mockFetch.mockResolvedValueOnce({ ok: true, json: async () => [{ projectId: 'p1', name: 'Test' }] });
    const client = new HexClient('test-token');
    const projects = await client.listProjects();
    expect(projects).toHaveLength(1);
  });

  it('should trigger a run', async () => {
    mockFetch.mockResolvedValueOnce({ ok: true, json: async () => ({ runId: 'r1', projectId: 'p1' }) });
    const client = new HexClient('test-token');
    const run = await client.runProject('p1', { date: '2025-01-01' });
    expect(run.runId).toBe('r1'…

Hex Performance Tuning

Latency Benchmarks

Instructions

Cache Project Lists

import { LRUCache } from 'lru-cache';
const projectCache = new LRUCache<string, any>({ max: 50, ttl: 300000 }); // 5 min

async function getCachedProjects(client: HexClient) {
  const cached = projectCache.get('projects');
  if (cached) return cached;
  const projects = await client.listProjects();
  projectCache.set('projects', projects);
  return projects;
}

Parallel Independent Runs

// Run independent projects in parallel (respecting rate limits)
async function parallelRuns(client: HexClient, configs: Array<{ id: string; params: any }>) {
  return Promise.allSettled(
    configs.map(c => runWithRetry(client, c.id, c.params))
  );
}

Optimize Poll Interval

// Adaptive polling: start fast, slow down
async function adaptivePoll(client: HexClient, projectId: string, runId: string) {
  let interval = 2000; // Start at 2s
  while (true) {
    const status = await client.getRunStatus(projectId, runId);
    if (['COMPLETED', 'ERRORED', 'KILLED'].includes(status.status)) return status;
    await new Promise(r => setTimeout(r, interval));
    interval = Math.min(interval * 1.5, 30000); // Max 30s
  }
}

Overview

Tune run latency and throughput using safe sandbox projects and aggregate metrics. A gain is invalid if it expands data scope, compromises output correctness, exceeds quota, or makes rollback impossible.

Prerequisites

• Baseline latency/run metrics, safe fixture revision, approved error budget, and a rollback revision for cache, concurrency, parameters, and retry policy.

Output

Return a tuning receipt with baseline/canary percentile bands, cache/concurrency/parameter revisions, quota/error outcomes, aggregate assertion, owner approval, and rollback reference. Use aggregates only.

Error Handling

Roll back for quota saturation, increased errors, changed output assertion, access drift, or duplicate runs. Do not raise concurrency or cache duration to hide a failing dependency.

Examples

env=sandbox; p95=420ms->310ms; concurrency=2; cache=r4; quota=within-budget; assertions=pass; rollback=perf-r3 documents a safe canary.

Resources

• Hex API

Hex Production Checklist

Overview

Hex is a collaborative data analytics platform where teams build notebooks, dashboards, and scheduled data pipelines. A production integration triggers project runs, retrieves results, and monitors pipeline health via the Hex API. Failures mean stale dashboards, broken scheduled reports, or data pipelines that silently stop producing output for downstream consumers.

Authentication & Secrets

• [ ] HEX_API_KEY stored in secrets manager (not config files)
• [ ] Token scoped to "Run projects" permission only
• [ ] Token expiration set > 90 days with calendar renewal reminder
• [ ] Separate tokens for dev/staging/prod workspaces
• [ ] Key rotation schedule documented (quarterly cycle)

API Integration

• [ ] Production base URL configured (https://app.hex.tech/api/v1)
• [ ] Rate limits respected (20 requests/min, 60/hour)
• [ ] All orchestrated projects published before triggering runs
• [ ] Input parameters documented and validated before submission
• [ ] Run status polling with configurable interval (default: 5s)
• [ ] Result retrieval handles large datasets (pagination/streaming)
• [ ] Project version pinning prevents unexpected behavior changes

Error Handling & Resilience

• [ ] Circuit breaker configured for Hex API outages
• [ ] Retry with exponential backoff for 429/5xx responses
• [ ] ERRORED and KILLED run states handled with alert + re-queue
• [ ] Run timeout detection (alert if run exceeds 2x typical duration)
• [ ] Database connection errors in Hex projects surface to caller
• [ ] Graceful degradation serves cached results when API is down

Monitoring & Alerting

• [ ] API latency tracked per project run
• [ ] Error rate alerts set (any ERRORED run = notification)
• [ ] Run duration regression tracked (>50% increase = warning)
• [ ] Scheduled run completion rate monitored daily
• [ ] API quota usage tracked against plan limits

Validation Script

async function checkHexReadiness(): Promise<void> {
  const checks: { name: string; pass: boolean; detail: string }[] = [];
  // API connectivity
  try {
    const res = await fetch('https://app.hex.tech/api/v1/projects', {
      headers: { Authorization: `Bearer ${process.env.HEX_API_KEY}` },
    });
    checks.push({ name: 'Hex API', pass: res.ok, detail: res.ok ? 'Connected' : `HTTP ${res.status}` });
  } catch (e: any) { checks.push({ name: 'Hex API', pass: false, detail: e.message }); }
  // Credentials present
  checks.push({ name: 'API Key Set', pass: !!process.env.HEX_API_KEY, detail: process.env.HEX_API_KEY ? 'Present' : 'MISSING'…

Hex Rate Limits

Overview

Hex's API enforces tight limits on project run triggers (20 per minute, 60 per hour) while leaving read operations like status checks and project listing largely unthrottled. Data teams scheduling batch analytics runs or triggering parameterized notebooks from CI/CD pipelines must carefully manage the hourly cap, since a single pipeline triggering 15 projects can consume a quarter of the hourly budget. Polling run status is free, but triggering runs is the bottleneck that shapes integration architecture.

Rate Limit Reference

Rate Limiter Implementation

class HexRateLimiter {
  private minuteTokens: number = 20;
  private hourlyTokens: number = 60;
  private lastMinuteRefill: number = Date.now();
  private lastHourlyRefill: number = Date.now();
  private queue: Array<{ resolve: () => void }> = [];

  async acquire(): Promise<void> {
    this.refill();
    if (this.minuteTokens >= 1 && this.hourlyTokens >= 1) {
      this.minuteTokens -= 1;
      this.hourlyTokens -= 1;
      return;
    }
    return new Promise(resolve => this.queue.push({ resolve }));
  }

  private refill() {
    const now = Date.now();
    this.minuteTokens = Math.min(20, this.minuteTokens + ((now - this.lastMinuteRefill) / 60_000) * 20);
    this.lastMinuteRefill = now;
    this.hourlyTokens = Math.min(60, this.hourlyTokens + ((now - this.lastHourlyRefill) / 3_600_000) * 60);
    this.lastHourlyRefill = now;
    while (this.minuteTokens >= 1 && this.hourlyTokens >= 1 && this.queue.length) {
      this.minuteTokens -= 1;
      this.hourlyTokens -= 1;
      this.queue.shift()!.resolve();
    }
  }
}

const runLimiter = new HexRateLimiter();

Retry Strategy

async function hexRunWithRetry(
  projectId: string, params: Record<string, any>, maxRetries = 3
): Promise<any> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    await runLimiter.acquire();
    const res = await fetch(`${HEX_BASE}/api/v1/run/${projectId}`, {
      method: "POST", headers,
      body: JSON.stringify({ inputParams: params }),
    });
    if (res.ok) return res.json();
    if (res.status === 429) {
      const delay = 30_000 * Math.pow(2, attempt) + Math.random() * 5000;
  …

Hex Reference Architecture

Architecture

┌────────────────────────────────────────┐
│          Orchestration Layer            │
│  (Airflow, Dagster, GitHub Actions,    │
│   Cron, Custom API)                    │
├────────────────────────────────────────┤
│           Hex API Client               │
│  (Run, Poll, Cancel, List)             │
├────────────────────────────────────────┤
│            Hex Platform                │
│  ┌──────────┐  ┌───────────────────┐  │
│  │ Projects  │  │ Data Connections  │  │
│  │ (SQL,     │  │ (Snowflake,      │  │
│  │  Python,  │  │  BigQuery,       │  │
│  │  R)       │  │  Postgres, etc.) │  │
│  └──────────┘  └───────────────────┘  │
└────────────────────────────────────────┘

Project Structure

hex-orchestrator/
├── src/hex/
│   ├── client.ts         # API client
│   ├── orchestrator.ts   # Pipeline runner
│   ├── scheduler.ts      # Cron-based triggers
│   └── types.ts          # TypeScript interfaces
├── src/notify/
│   └── slack.ts          # Completion notifications
├── tests/
├── config/
│   └── pipelines.json    # Pipeline definitions
└── .env.example

Integration Patterns

Overview

This architecture separates project triggers, parameter validation, scoped execution, bounded orchestration, aggregate observability, and cancellation/rollback. Workspace data remains inside approved project boundaries and never becomes architectural telemetry.

Prerequisites

• A project/data owner for each source, environment allowlist, secret manager, and owner for every execution edge.
• Separate sandbox/staging/production configuration plus rollback for project, schedule, queue, and client controls.

Instructions

1. Map every trigger-to-project edge with owner, data class, allowed parameters, authorization scope, retry behavior, observability, and rollback.
2. Start with a sandbox project and fail closed on unknown project, parameter schema, destination, or response shape.
3. Make asynchronous execution idempotent and bounded; quarantine uncertainty rather than resubmitting or exporting output for debugging.
4. Canary one project with aggregate signals before promotion and retain the prior revision.
5. Re-evaluate controls whenever credentials, project ownership, schedules, or parameters change.

Output

Produce an architecture record with owners, opaque project IDs, policy r…

Hex SDK Patterns

Overview

Production patterns for Hex API: typed client, pipeline orchestration, retry logic, and Python integration.

Instructions

Step 1: Run with Retry

async function runWithRetry(client: HexClient, projectId: string, params: Record<string, any>, maxRetries = 2) {
  for (let i = 0; i <= maxRetries; i++) {
    try {
      const { runId } = await client.runProject(projectId, params);
      const result = await pollUntilComplete(client, projectId, runId);
      return result;
    } catch (err: any) {
      if (i === maxRetries || !err.message.includes('429')) throw err;
      await new Promise(r => setTimeout(r, 30000)); // Wait 30s on rate limit
    }
  }
}

Step 2: Python Client (hextoolkit)

# pip install hextoolkit
from hextoolkit import HexAPI

hex_api = HexAPI(token=os.environ['HEX_API_TOKEN'])

# List projects
projects = hex_api.list_projects()

# Run project
run = hex_api.run_project('project-id', input_params={'date': '2025-01-01'})

# Poll for completion
status = hex_api.get_run_status('project-id', run['runId'])

Step 3: Airflow Integration

# Using the hex-inc/airflow-provider-hex package
from airflow_provider_hex.operators.hex import HexRunProjectOperator

run_task = HexRunProjectOperator(
    task_id='run_hex_project',
    project_id='your-project-id',
    input_params={'date': '{{ ds }}'},
    hex_conn_id='hex_default',
    wait_for_completion=True,
    timeout=600,
)

Prerequisites

• A typed client boundary, secret-manager reference, environment allowlist, and a sandbox project using fictional or approved non-sensitive data.
• An explicit execution budget, idempotency/correlation convention, and policy that unknown project scope or response shape fails closed.

Output

Produce an SDK receipt with client revision, environment, opaque project/run IDs, parameter schema revision, execution state, aggregate result checks, and rollback/cancel reference. Do not include API tokens, SQL, cell output, or workspace data.

Error Handling

Classify authentication, authorization, validation, quota, timeout, and terminal-run failures separately. Do not retry a write-like project run without idempotency, broaden scope to bypass a denial, or log notebook output for diagnosis.

Examples

sdk=v3; env=sandbox; project=proj-opaque-11; params=r4; run=succeeded; row_count=within-range; cancel=not-needed is a safe SDK execution receipt.

Resources

• Hex API
• Airflow Provider

Next Steps

…

Hex Security Basics

Overview

Hex is a collaborative data analytics platform where notebooks query production databases, generate visualizations, and share results across teams. Security concerns center on API token management (read vs run scopes), protecting database connection credentials embedded in Hex projects, and ensuring query results containing sensitive business data are not leaked through logs or exports. A compromised run-scope token can trigger arbitrary queries against connected databases.

API Key Management

function createHexClient(scope: "read" | "run"): { token: string; baseUrl: string } {
  const envVar = scope === "run" ? "HEX_RUN_TOKEN" : "HEX_READ_TOKEN";
  const token = process.env[envVar];
  if (!token) {
    throw new Error(`Missing ${envVar} — store in secrets manager, never in code`);
  }
  // Run tokens can trigger queries — use read tokens for monitoring
  console.log(`Hex client initialized with ${scope} scope (token suffix: ${token.slice(-4)})`);
  return { token, baseUrl: "https://app.hex.tech/api/v1" };
}

Webhook Signature Verification

import crypto from "crypto";
import { Request, Response, NextFunction } from "express";

function verifyHexWebhook(req: Request, res: Response, next: NextFunction): void {
  const signature = req.headers["x-hex-signature"] as string;
  const secret = process.env.HEX_WEBHOOK_SECRET!;
  const expected = crypto.createHmac("sha256", secret).update(req.body).digest("hex");
  if (!signature || !crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))) {
    res.status(401).send("Invalid signature");
    return;
  }
  next();
}

Input Validation

import { z } from "zod";

const HexRunRequestSchema = z.object({
  project_id: z.string().uuid(),
  input_params: z.record(z.string(), z.unknown()).optional(),
  notify_on_completion: z.boolean().default(false),
  update_cache: z.boolean().default(false),
});

function validateHexRunRequest(data: unknown) {
  return HexRunRequestSchema.parse(data);
}

Data Protection

const HEX_SENSITIVE_FIELDS = ["db_connection_string", "query_results", "api_token", "input_params", "export_url"];

function redactHexLog(record: Record<string, unknown>): Record<string, unknown> {
  const redacted = { ...record };
  for (const field of HEX_SENSITIVE_FIELDS) {
    if (field in redacted) redacted[field] = "[REDACTED]";
  }
  return redacted;
}

Security Checklist

• [ ] API tokens stored in secrets vault, never in code
• [ ] Read-only tokens for monitoring, run tokens for orchest…

Hex Upgrade & Migration

Overview

Hex API is versioned at /api/v1/. Monitor the Hex changelog for new endpoints and deprecations.

Instructions

Check API Usage

grep -r "app.hex.tech" src/ --include="*.ts" --include="*.py"

Airflow Provider Updates

pip install --upgrade airflow-provider-hex

Prerequisites

• Pinned current and target versions, a compatibility assessment, sandbox project fixtures, and an owner for each changed contract.
• Versioned configuration backups plus approved downgrade, cancellation, and integration-disable procedures.

Output

Produce an upgrade receipt with from/to versions, affected contracts, fixture/canary outcomes, aggregate assertions, owner approval, compatibility decision, and rollback revision. Exclude SQL, workspace output, and credentials.

Error Handling

Stop for incompatible parameter/output contracts, authorization drift, unbounded replays, or a failed canary assertion. Restore the exact pinned prior revision and retain redacted evidence rather than forcing migration.

Examples

from=client-r12; to=client-r13; sandbox=pass; staging=pass; assertions=pass; canary=held; rollback=r12 is a defensible upgrade record.

Resources

• Hex Changelog
• API Reference

Hex Webhooks & Events

Overview

Hex doesn't provide push webhooks. For event-driven integrations, poll run status or build your own notification system around run completions.

Instructions

Run Status Polling with Callback

async function runWithCallback(
  client: HexClient,
  projectId: string,
  params: Record<string, any>,
  onComplete: (result: any) => void,
  onError: (error: Error) => void
) {
  try {
    const { runId } = await client.runProject(projectId, params);
    const poll = async () => {
      const status = await client.getRunStatus(projectId, runId);
      if (status.status === 'COMPLETED') { onComplete(status); return; }
      if (status.status === 'ERRORED' || status.status === 'KILLED') { onError(new Error(status.status)); return; }
      setTimeout(poll, 5000);
    };
    poll();
  } catch (err) { onError(err as Error); }
}

Notify on Completion

runWithCallback(client, 'project-id', { date: '2025-01-01' },
  (result) => {
    // Send Slack notification, email, etc.
    fetch(process.env.SLACK_WEBHOOK_URL!, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ text: `Hex project completed: ${result.runId}` }),
    });
  },
  (error) => console.error('Run failed:', error)
);

Prerequisites

• A secret-manager webhook secret, origin allowlist, replay-window policy, and opaque event ledger.
• A sandbox receiver plus a tested disable/cancel control for downstream project execution.

Output

Return an event receipt with type, opaque event/run ID, signature/timestamp result, idempotency outcome, queue state, canary result, and rollback reference. Exclude payload data, SQL, output, and signatures.

Error Handling

Reject unknown origin, stale/replayed delivery, malformed payload, unknown project destination, or non-idempotent retry. Quarantine the opaque event and disable the consumer if integrity is uncertain.

Examples

type=run.completed; event=evt-opaque-9; signature=pass; replay=absent; project=proj-sandbox-12; enqueue=once; rollback=consumer-disabled proves the event boundary.

Resources

• Hex API