ai-experiment-logger

The AI Experiment Logger helps you systematically track experiments across different AI tools (ChatGPT, Claude, Gemini, etc.), analyze effectiveness patterns, and make data-driven decisions about which tools and prompting strategies work best for your use cases.

Key Features:

• 🎯 Structured Experiment Logging - Capture tool, prompt, result, rating, and tags
• 📊 Rich Statistics Dashboard - Visualize tool performance, rating distribution, and trends
• 🔍 Advanced Search & Filtering - Find experiments by any field or search term
• 📈 Performance Analytics - Track which AI tools perform best for your needs
• 💾 Data Persistence - Local JSON storage with CSV export capability
• 🌐 Dual Interface - MCP tools for Claude Code + web UI for visual exploration
• 🎨 Modern UI - Clean, responsive interface built with Tailwind CSS

Using MCP Tools (in Claude Code)

Log an Experiment

Use the log_experiment tool to record:
- AI Tool: "ChatGPT o1-preview"
- Prompt: "Write a Python function to calculate Fibonacci numbers recursively"
- Result: "Provided clean recursive implementation with base cases. Included time complexity analysis (O(2^n))."
- Rating: 4
- Tags: ["code-generation", "python", "algorithms"]

List Recent Experiments

Use the list_experiments tool to show my last 10 experiments

Search Experiments

Use the list_experiments tool with searchQuery="code-generation" to find all coding experiments

Get Statistics

Use the get_statistics tool to show me which AI tools perform best

Export Data

Use the export_experiments tool to generate a CSV file of all my experiments

Using the Web Dashboard

Start the Web Server

cd plugins/mcp/ai-experiment-logger
pnpm web

The web UI will be available at http://localhost:3000

Web Dashboard Features

1. Dashboard Tab

• View all experiments in a sortable table
• Search across all fields in real-time
• Filter by AI tool, rating, or date range
• Delete experiments with confirmation

1. Statistics Tab

• Total experiments and average rating
• AI tool performance comparison
• Rating distribution visualization
• Top tags analysis
• Recent activity chart (30 days)

1. Log Experiment Button

• Easy form with all fields
• Star rating selector (1-5)
• Tag management (comma-separated)
• Date/time picker (defaults to now)

1. Export CSV Button

• Download all experiments as CSV
• Compatible with Excel, Google Sheets, etc.
• Includes all metadata

• Prompt Engineering Research - Track which prompts work best across tools
• AI Tool Evaluation - Compare ChatGPT, Claude, Gemini, etc. for your needs
• Team Collaboration - Share experiment logs with team members
• Quality Tracking - Monitor AI output quality over time
• Cost Optimization - Identify most effective tools to reduce API costs
• Documentation - Maintain records of successful AI interactions
• Learning & Training - Build a knowledge base of effective prompts