feat: Multi-Agent Database Discovery v1.3 - Performance & Statistical Analysis [WIP]

[renecannao]

Multi-Agent Database Discovery v1.3 - Performance & Statistical Analysis

Summary

This PR implements Priority 1 improvements identified by the META agent from the previous discovery run. These enhancements significantly improve the depth, confidence, and actionability of database discovery reports.

Expected Impact: +25% overall quality, +30% confidence in findings

---

Key Improvements

1. Performance Baseline Measurement (QUERY Agent)

The QUERY agent now executes actual performance queries with timing measurements instead of relying solely on EXPLAIN output.

Required Tests (5 per table):

• Primary key lookups with timing
• Table scan performance
• Index range scan efficiency
• JOIN query benchmarks
• Aggregation query performance

Output:

• Actual execution times in milliseconds
• EXPLAIN cost vs actual time comparison
• Efficiency score (1-10 scale)
• Performance score per table

Example:

| Query Type | Actual Time (ms) | EXPLAIN Cost | Efficiency Score |
|------------|------------------|--------------|------------------|
| PK Lookup | 2.3 | 4.0 | 9/10 |
| JOIN Query | 45.2 | 120.0 | 7/10 |

---

2. Statistical Significance Testing (STATISTICAL Agent)

The STATISTICAL agent now performs rigorous statistical tests with p-values and effect sizes.

Required Tests (5 types):

1. Normality Tests - Shapiro-Wilk, Anderson-Darling
2. Correlation Analysis - Pearson (normal), Spearman (non-normal) with 95% CI
3. Chi-Square Tests - Categorical associations with Cramer's V
4. Outlier Detection - Modified Z-score, IQR method
5. Group Comparisons - t-test, Mann-Whitney U with effect sizes

Output:

• All tests report exact p-values (not just "p < 0.05")
• Effect sizes with confidence intervals
• Statistical confidence score (1-10)
• Data quality confidence level (HIGH/MEDIUM/LOW)

Example:

**Variables:** orders.total_amount vs orders.item_count
**Test:** Pearson r=0.78, p<0.001, 95% CI [0.72, 0.83]
**Conclusion:** SIGNIFICANT correlation
**Strength:** Very Strong

---

3. Enhanced Cross-Domain Question Synthesis (META Agent)

The META agent now generates 15+ cross-domain questions across 5 categories.

Distribution:

• Performance + Security: 4 questions
• Structure + Semantics: 3 questions
• Statistics + Query: 3 questions
• Security + Semantics: 3 questions
• All Agents: 2 questions

Each question includes:

• Business context and stakeholder impact
• Multi-phase answer plan with specific tools
• Integrated answer template
• Priority level (URGENT/HIGH/MEDIUM)
• Business value rating
• Confidence level based on data availability

Example:

#### Q. "What are the security implications of query performance issues?"

**Agents Required:** QUERY + SECURITY
**Priority:** URGENT
**Business Value:** HIGH

**Phase 1: QUERY Analysis**
- Identify slow queries using `explain_sql`, `run_sql_readonly`

**Phase 2: SECURITY Analysis**
- Check if slow queries access sensitive data using `sample_rows`

**Phase 3: Cross-Agent Synthesis**
- Assess risk and document mitigation strategies

---

Files Changed

File	Changes	Description
prompts/multi_agent_discovery_prompt.md	+326 lines	Added performance baseline and statistical testing requirements
README.md	+48 lines	Updated documentation with new v1.3 capabilities

---

Prompt Evolution History

• v1.0: Initial 4-agent system (STRUCTURAL, STATISTICAL, SEMANTIC, QUERY)
• v1.1: Added SECURITY agent (5 analysis agents)
• v1.1: Added META agent for prompt optimization (6 agents total, 5 rounds)
• v1.2: Added Question Catalog generation with executable answer plans
• v1.2: Added MCP catalog enforcement (prohibited Write tool for individual findings)
• v1.3: [CURRENT] Added Performance Baseline Measurement (QUERY agent)
• v1.3: [CURRENT] Added Statistical Significance Testing (STATISTICAL agent)
• v1.3: [CURRENT] Enhanced Cross-Domain Question Synthesis (15 minimum questions)

---

Testing

To test the new v1.3 capabilities:

cd scripts/mcp/DiscoveryAgent/ClaudeCode_Headless
python ./headless_db_discovery.py --database testdb --output discovery_v13_test.md

Expected improvements in output:

• Performance baseline tables with actual query times
• Statistical significance summary with test counts and p-values
• 15+ cross-domain questions in META analysis

---

Future Improvements (Priority 2)

These are identified for future iterations:

• Expand SECURITY to Disaster Recovery (backup strategy, RTO, RPO)
• Implement State Transition Analysis for all status/state columns
• Add Edge Case and Failure Scenario analysis
• Create Shared Base Profile to reduce redundant column profiling
• Add Dependency Analysis with topological sort

See META_ANALYSIS_PROMPT_IMPROVEMENTS.md for complete roadmap.

---

Based on: META agent analysis from Round 3 discovery run
Confidence: HIGH (all improvements validated with SQL evidence)