Skip to main content

Scenario 07: Find and prioritize test gaps

Map measured coverage to CPG methods and prioritize risky test gaps. Distinguish a missing artifact from measured zero coverage.

User Guides

Use this scenario to combine measured coverage with structural data such as callers, entry points and test references. This helps you choose the next test based on product behavior and the risk of failure.

Inspect measured coverage

Prepare the environment using the CLI guide. Run commands in PowerShell from the CodeGraph repository root. You need an existing CPG, reachable GoCPG gRPC, and a coverage artifact produced by the project’s test tool. The installed audit orchestrator selects artifacts through runtime_coverage_artifact_paths.

python -m src.cli coverage show --db "<cpg-path>" --uncovered-only

Replace <cpg-path> with the selected project database. The command queries method line spans through GoCPG, maps them to a coverage artifact, and prints the result in the terminal. --uncovered-only selects methods with no executed lines.

With available data, output contains the artifact path and identity, format, Methods summary, and method rows. The source_revision field identifies the CPG revision of the selected project. Compare it with the revision used to run the tests. UNMATCHED coverage source rows identify report files that could not be mapped to methods in the selection. Check those file paths.

If no artifact is found, the command prints:

No coverage data found: runtime coverage artifact is missing.

This case returns 0, but coverage was not measured. Other unavailable-data reasons print Coverage data unavailable: <reason> with exit 1; query failure prints Coverage query failed: <error> with exit 1. To measure coverage, produce the artifact with the project’s test tool, then verify its path, revision, and mapping to methods.

Prepare and interpret the data

Generate a coverage report with your project’s test tool and save it under the registered source root. The command searches that project using runtime_coverage_artifact_paths. The current configuration includes coverage.json, coverage.xml, and lcov.info, including files in nested directories. The CLI working directory does not determine the search root. Verify the registered source and database paths with python -m src.cli projects info <project-name>.

The Methods summary divides methods with at least one executed line by methods mapped to executable lines in the artifact. For example, a method with two executed lines out of ten counts as a covered method; its line coverage is 20%. Inspect the corresponding test assertions to verify behavior.

The command analyzes a selection of methods ordered by file and line number. The current configuration selects the first 500 methods. sql_query_limits.race_condition sets the selection limit. The summary covers methods in that selection that were mapped to the coverage report. Use the full report from your test tool when assessing the entire project.

Typed QA investigation

Configure the project and LLM provider using the programmatic guide. The request below is a template for an existing QA task. Replace its scope, task identifier, paths, and references with accessible evidence for that revision.

For graph-backed gap prioritization, invoke the required QA scenario:

from src.digital_employees.runtime.scenarios import (
    RoleBoundScenarioInvocationRequest,
    invoke_role_bound_scenario_request,
)

request = RoleBoundScenarioInvocationRequest(
    query="Prioritize untested behavior in the accepted change scope.",
    employee_id="codegraph_qa",
    scenario_id="scenario_07",
    event_type="qa_start",
    context={
        "project_key": "codegraph",
        "namespace": "default",
        "task_id": "<task-id>",
        "file_paths": ["<changed-source>", "<related-tests>"],
        "source_refs": ["<coverage-artifact-ref>", "<acceptance-criterion-ref>"],
    },
)

result = invoke_role_bound_scenario_request(request)
if result.get("error"):
    raise RuntimeError(result["error"])
print(result["scenario_id"])
print(result.get("answer") or "")

Missing project_key, namespace, or task_id raises RuntimeError containing role_bound_scenario_contract_missing. For a negative check, pass context={}, then restore the task scope.

A successful investigation returns scenario_07, a response in answer, and invocation details in metadata.role_bound_scenario_invocation. Read the recommendations alongside the coverage report: for each proposed test, identify the behavior, code location, and acceptance criterion it checks. Save the response and source revision with the coverage artifact.

Prioritize by behavior and risk

Review each candidate against:

  • active acceptance criteria and the changed revision;
  • public entry points and authorization boundaries;
  • error, retry, timeout and rollback paths;
  • persistence and serialization contracts;
  • high-impact callers and previously escaped defects;
  • whether the coverage evidence is current and actually includes the file.

A covered line shows that a test executed that part of the code. Test assertions define the behavior being checked. Record exclusions for generated code and defensive branches in the coverage settings and approve them during review.

Tests-first workflow

  1. Select one missing behavior tied to an acceptance criterion or explicit risk.
  2. Add a test that fails for the expected reason.
  3. Make the smallest product change only if the failure reveals missing implementation.
  4. Run the focused test and neighboring regression suite.
  5. Regenerate the project coverage artifact using the project’s own test tool.
  6. Refresh through the supported project path, then rerun the display command.
  7. Record the test execution, revision, artifact basis, and remaining gaps.

Attach the coverage report to the QA results. For acceptance, record Pass/Fail for every active criterion and the decision of the QA owner.

Use Feature Development for a missing implementation and Code Review for revision-bound structural review.

Report a documentation issue