Scenario 07: Test Coverage

Scenario 07: Test Coverage

QA engineer identifying untested code paths, importing runtime coverage data, and getting CPG-based test recommendations.

Quick Start

# Select Test Coverage Scenario
/select 07

Finding Coverage Gaps

Identify Untested Functions

The handler uses heuristic detection by default: methods without test_* callers in the call graph are flagged as untested.

> What functions lack test coverage?

+--------------- Coverage Gaps ---------------+
| Functions without direct tests:             |
|                                             |
| Critical (executor):                        |
|   - ExecParallelHashJoinNewBatch()          |
|   - ExecReScanGather()                      |
|                                             |
| High priority (storage):                    |
|   - heap_lock_updated_tuple()               |
|   - heap_abort_speculative()                |
|                                             |
| Total untested: 234 functions               |
| Coverage estimate: 78%                      |
| Detection mode: Heuristic                   |
+---------------------------------------------+

Prioritize Testing

> Which untested functions have highest impact?

+--------------- Priority List ---------------+
| High Impact + No Tests:                     |
|                                             |
| 1. heap_lock_updated_tuple()                |
|    Impact: Transaction integrity            |
|    Callers: 23                              |
|                                             |
| 2. ExecParallelHashJoinNewBatch()           |
|    Impact: Parallel query correctness       |
|    Callers: 8                               |
|                                             |
| 3. AtEOXact_RelationCache()                 |
|    Impact: Cache consistency                |
|    Callers: 4                               |
+---------------------------------------------+

Find Untested Error Paths

> Find untested error handling paths
> Show entry points without tests

Runtime Coverage Import

Import coverage data from external tools to enable hybrid detection (runtime + heuristic).

Supported Formats

Format Tool File Type
pytest-cov pytest-cov (--cov-report=json) JSON
lcov gcov / lcov / geninfo Text (.info, .lcov)
cobertura Cobertura, JaCoCo, coverage.py XML XML

CLI Commands

# Import pytest-cov JSON report
python -m src.cli coverage import --file coverage.json --format pytest-cov --db data/projects/postgres.duckdb

# Import lcov trace file
python -m src.cli coverage import --file lcov.info --format lcov

# Import Cobertura XML (e.g., from Java/C# tooling)
python -m src.cli coverage import --file coverage.xml --format cobertura --source-root /project

# View imported coverage data
python -m src.cli coverage show
python -m src.cli coverage show --uncovered-only

How It Works

  1. The parser reads the coverage report and extracts per-file line-level hit data
  2. The importer adds a coverage_percent column to nodes_method (if absent)
  3. Each method is matched to coverage data by suffix-matching the filename and intersecting the method’s line range with covered lines
  4. coverage_percent is computed as covered_lines_in_range / total_lines_in_range * 100

Path normalization: Coverage reports often contain absolute paths while the CPG stores relative paths. The importer normalizes paths (strips ./, converts backslashes) and falls back to suffix matching. Use --source-root to strip a common prefix for better matching.

Import Result

+-------------- Coverage Import Summary --------------+
| Format: pytest-cov                                  |
| Methods updated: 1,247 / 3,891                      |
| Files matched: 89 / 112                             |
+-----------------------------------------------------+

Hybrid Detection

After importing runtime coverage data, untested code queries automatically switch to hybrid mode:

  • Methods with coverage_percent < 1.0 are flagged via runtime data
  • Methods with NULL coverage_percent (no runtime data) fall back to heuristic test-caller analysis
  • Each candidate is tagged with detection_method: "runtime" or "heuristic"
  • The coverage estimate uses AVG(coverage_percent) from runtime data instead of the heuristic ratio
> Find untested code

+--------------- Coverage Gaps (Hybrid) ---------------+
| Method              | Detection | Coverage | Reason  |
|---------------------|-----------|----------|---------|
| parse_query()       | Runtime   | 0.0%     | ...     |
| exec_plan()         | Runtime   | 12.5%    | ...     |
| helper_func()       | Heuristic | ---      | No test |
|                                                      |
| Detection mode: Runtime + Heuristic                  |
| Coverage estimate: 62.3% (from runtime data)         |
+------------------------------------------------------+

Backward compatibility: If no coverage data has been imported, the handler works identically to the heuristic-only mode. The _has_coverage_column() guard ensures zero behavior change.

CPG-Based Test Recommendations

For each untested method (top 20 by criticality), the handler analyzes CPG data and generates specific, actionable test recommendations.

Branch Coverage Analysis

Counts control structures (IF, FOR, WHILE, SWITCH) from nodes_control_structure and estimates the number of test cases needed for branch coverage.

> Find untested code

### `parse_query`
1. Add 8 test cases for branch coverage (IF: 3, FOR: 1, SWITCH: 1)
2. Test boundary values for parameter `query_len` (zero, negative, max, min)
3. Test error handling: 2 try/catch blocks, 4 return statements

Parameter Boundary Analysis

Maps parameter types from nodes_param to boundary test suggestions:

Type Boundary Tests
int, long, size_t, float zero, negative, max, min
char*, string, str empty, null, very long, special chars
Pointer types (*, ptr) null pointer
bool true, false
Variadic parameters zero args, one arg, many args

Error Path Analysis

Counts TRY blocks in nodes_control_structure and RETURN statements in nodes_return. Multiple return statements suggest error handling paths that need testing.

Generating Test Cases

The test generation handler provides detailed test recommendations for specific functions using call graph analysis.

> Generate test cases for heap_insert

+--------------- Test Cases -------------------------+
|                                                    |
| Function: heap_insert()                            |
| File: src/backend/access/heap/heapam.c:2156        |
|                                                    |
| Test Coverage Strategy:                            |
|   Unit Tests: Test heap_insert in isolation        |
|     by mocking its 5 dependencies                  |
|   Integration Tests: Test heap_insert through      |
|     its 23 callers to verify real-world usage      |
|   Edge Cases: Test boundary conditions,            |
|     null inputs, error handling                    |
|                                                    |
| Dependencies to Mock/Test:                         |
|   - RelationGetBufferForTuple()                    |
|   - heap_prepare_insert()                          |
|   - XLogInsert()                                   |
|                                                    |
| Test Scenarios from Callers:                       |
|   - simple_heap_insert() uses heap_insert for...   |
|   - toast_save_datum() uses heap_insert for...     |
+----------------------------------------------------+

Example Questions

Untested code detection: - “What functions lack test coverage?” - “Find untested code” - “Show functions without tests” - “Which code is not covered by tests?”

Test prioritization: - “Which critical functions need tests first?” - “What should I test first?” - “Test priority ranking”

Test generation: - “Generate test cases for [function_name]” - “Create tests for [function_name]” - “What edge cases should I test in [function_name]?”

Coverage overview: - “Show coverage gaps” - “Coverage report” - “How to improve test coverage?”