Scenario 07: Test Coverage

CPG-based untested code detection, test prioritization, test generation recommendations, runtime coverage import, and hybrid analysis.

Table of Contents¶

• Quick Start
• How It Works
• Architecture
• Intent Detection
• Handler Registry
• Detection Modes
• Heuristic Detection
• Hybrid Detection
• Handlers
• UntestedCodeHandler
• TestGenerationHandler
• TestPriorityHandler
• Impact Analysis
• CPG-Based Recommendations
• Runtime Coverage Import
• Configuration
• CLI Usage
• REST API
• Use Cases
• Example Questions
• Related Scenarios

Quick Start¶

/select 07

How It Works¶

Architecture¶

The test coverage module (src/workflow/scenarios/coverage_handlers/, 13 files) consists of 6 components:

User Query
    |
    v
CoverageIntentDetector (5 intent types, bilingual)
    |
    v
HandlerRegistry (priority-ordered dispatch)
    |
    +---> TestGenerationHandler (priority 5)
    |         generate test recommendations for specific functions
    |
    +---> UntestedCodeHandler (priority 10)
    |         find untested code via heuristic or hybrid detection
    |
    +---> TestPriorityHandler (priority 20)
    |         rank untested functions by criticality
    |
    v
CoverageReportFormatter (bilingual markdown output)
    |
    v
CallGraphAnalyzer ──> ImpactAnalysis (impact_score, callers, callees)

Intent Detection¶

CoverageIntentDetector classifies queries into 5 intent types using morphological keyword matching with Cyrillic word boundaries:

Additional extraction: - _extract_criticality(query) — returns "critical", "high", "medium", or "all" - _extract_scope(query) — returns "method", "class", "module", or "all"

Handler Registry¶

Handlers are registered via HandlerRegistry("coverage") with priority-ordered dispatch. Lower priority = higher precedence:

@coverage_registry.register(priority=5)
class TestGenerationHandlerRegistered(TestGenerationHandler): ...

@coverage_registry.register(priority=10)
class UntestedCodeHandlerRegistered(UntestedCodeHandler): ...

@coverage_registry.register(priority=20)
class TestPriorityHandlerRegistered(TestPriorityHandler): ...

Each handler implements can_handle(query_info) -> bool and handle(query_info) -> HandlerResult. The registry tries handlers in priority order and uses the first one that matches.

Detection Modes¶

Heuristic Detection¶

Default mode when no runtime coverage data is available. Methods without test_* callers in edges_call are flagged as untested:

-- Methods with no test callers
SELECT m.id, m.name, m.full_name
FROM nodes_method m
WHERE NOT EXISTS (
    SELECT 1 FROM edges_call ec
    JOIN nodes_method caller ON ec.source_id = caller.id
    WHERE ec.target_id = m.id
    AND caller.name LIKE 'test_%'
)

Hybrid Detection¶

Automatically activated when coverage_percent column exists in nodes_method (populated via coverage import):

• Methods with coverage_percent < 1.0 → flagged via runtime data
• Methods with NULL coverage_percent → fallback to heuristic test-caller analysis
• Each candidate tagged with detection_method: "runtime" or "heuristic"
• Coverage estimate uses AVG(coverage_percent) from runtime data

The _has_coverage_data() guard ensures zero behavior change when no data is imported.

Handlers¶

UntestedCodeHandler¶

Handles untested_code_scan intent. Key methods:

Enriches top 20 candidates with CPG-based test recommendations (branch coverage, parameter boundaries, error paths).

TestGenerationHandler¶

Handles test_generation intent. Key methods:

TestPriorityHandler¶

Handles test_priority intent. Ranks untested functions using 4 scoring factors:

Score is converted to priority level via _score_to_rating(): high (≥ threshold), medium, low.

Impact Analysis¶

The workflow uses CallGraphAnalyzer (Graph Method #2) from src/analysis/callgraph/analyzer.py for impact analysis on untested methods.

ImpactAnalysis dataclass:

3 graph insight categories tracked in state["metadata"]:

CPG-Based Recommendations¶

For each untested method (top 20 by criticality), the handler generates specific recommendations:

Branch Coverage Analysis¶

Counts control structures (IF, FOR, WHILE, SWITCH) from nodes_control_structure and estimates test cases needed.

Parameter Boundary Analysis¶

Maps parameter types from nodes_param to boundary test suggestions:

Error Path Analysis¶

Counts TRY blocks in nodes_control_structure and RETURN statements in nodes_return. Multiple return statements suggest error handling paths.

Runtime Coverage Import¶

Import coverage data from external tools to enable hybrid detection.

Supported Formats¶

How It Works¶

1. The parser reads the coverage report and extracts per-file line-level hit data
2. The importer adds a coverage_percent column to nodes_method (if absent)
3. Each method is matched to coverage data by suffix-matching the filename and intersecting the method’s line range with covered lines
4. coverage_percent is computed as covered_lines_in_range / total_lines_in_range * 100

Path normalization: Coverage reports often contain absolute paths while the CPG stores relative paths. The importer normalizes paths (strips ./, converts backslashes) and falls back to suffix matching. Use --source-root to strip a common prefix.

Configuration¶

Coverage-related parameters from get_unified_config():

Thresholds¶

Scoring Weights¶

Handler Limits¶

CLI Usage¶

# Import pytest-cov JSON report
python -m src.cli.import_commands coverage import --file coverage.json --format pytest-cov --db data/projects/postgres.duckdb

# Import lcov trace file
python -m src.cli.import_commands coverage import --file lcov.info --format lcov

# Import Cobertura XML (e.g., from Java/C# tooling)
python -m src.cli.import_commands coverage import --file coverage.xml --format cobertura --source-root /project

# View imported coverage data
python -m src.cli.import_commands coverage show
python -m src.cli.import_commands coverage show --uncovered-only

REST API¶

Test coverage queries are handled via the scenario router:

Request model (ScenarioQueryRequest):

Response model (ScenarioQueryResponse):

Example:

curl -X POST http://localhost:8000/api/v1/scenarios/test_coverage/query \
  -H "Content-Type: application/json" \
  -d '{"query": "Find untested functions", "language": "en"}'

Use Cases¶

Finding Untested Code¶

> What functions lack test coverage?

## Coverage Gaps

**Detection mode:** Heuristic
**Total untested:** 234 functions
**Coverage estimate:** 78%

### Critical (executor)
- ExecParallelHashJoinNewBatch()
- ExecReScanGather()

### High priority (storage)
- heap_lock_updated_tuple()
- heap_abort_speculative()

### Recommendations for `heap_lock_updated_tuple`
1. Add 5 test cases for branch coverage (IF: 3, SWITCH: 1)
2. Test boundary values for parameter `flags` (zero, negative, max)
3. Test error handling: 2 try/catch blocks

Test Prioritization¶

> Which critical functions need tests first?

## Test Priority Ranking

| Function | Score | Priority | Reason |
|----------|-------|----------|--------|
| heap_lock_updated_tuple | 8.5 | high | core module, 23 callers, public API |
| ExecParallelHashJoinNewBatch | 7.0 | high | engine module, complex signature |
| AtEOXact_RelationCache | 5.0 | medium | system module, 4 callers |

Generating Test Cases¶

> Generate test cases for heap_insert

## Test Recommendations: heap_insert()

**File:** src/backend/access/heap/heapam.c:2156

**Strategy:**
- Unit Tests: Test heap_insert in isolation by mocking 5 dependencies
- Integration Tests: Test through 23 callers to verify real-world usage
- Edge Cases: Test boundary conditions, null inputs, error handling

**Dependencies to Mock:**
- RelationGetBufferForTuple()
- heap_prepare_insert()
- XLogInsert()

**Test Scenarios from Callers:**
- simple_heap_insert() uses heap_insert for...
- toast_save_datum() uses heap_insert for...

Hybrid Detection (with runtime data)¶

> Find untested code

## Coverage Gaps (Hybrid)

**Detection mode:** Runtime + Heuristic
**Coverage estimate:** 62.3% (from runtime data)

| Method | Detection | Coverage | Reason |
|--------|-----------|----------|--------|
| parse_query() | Runtime | 0.0% | No lines covered |
| exec_plan() | Runtime | 12.5% | Partial coverage |
| helper_func() | Heuristic | --- | No test callers |

Example Questions¶

Untested code detection: - “What functions lack test coverage?” - “Find untested code” - “Show functions without tests” - “Which code is not covered by tests?”

Test prioritization: - “Which critical functions need tests first?” - “What should I test first?” - “Test priority ranking”

Test generation: - “Generate test cases for heap_insert” - “Create tests for palloc function” - “What edge cases should I test in ExecInitNode?” - “Write mutation tests for query parser”

Coverage overview: - “Show coverage gaps” - “Coverage report” - “How to improve test coverage?”

Related Scenarios¶

• Security Audit (S02) — Security testing
• Refactoring (S05) — Find unused code and refactor
• Debugging (S15) — Debug code issues