Scenario 19: Reference-Guided Standards Check

Scenario 19: Reference-Guided Standards Check

Quality engineer or tech lead enforcing coding standards with document-driven analysis.

Table of Contents

Quick Start

/select 19

Overview

Scenario 19 provides standards-driven code analysis using YAML configuration files. Unlike generic linting, it allows teams to define custom coding standards with references to official guidelines, best practices documents, or internal policies.

Standards Document (YAML) → Rule Extraction → Code Analysis → Violation Report
                                    ↓
                          ┌─────────┴─────────┐
                          ↓                   ↓
                    Rule Database      Reference Links

Standards Documents

YAML Format Specification

Standards documents define rules with categories, severity, and references:

# company_standards.yaml
name: "Company Coding Standards v2.0"
version: "2.0.0"
last_updated: "2024-12-09"
author: "Engineering Team"

categories:
  - naming
  - security
  - style
  - documentation
  - error_handling

rules:
  - id: "SEC-001"
    title: "Use parameterized queries for database operations"
    category: security
    severity: error
    description: |
      All database queries must use parameterized queries or prepared
      statements to prevent SQL injection attacks.
    pattern: "execute\\s*\\(.*%s.*\\)|execute\\s*\\(.*.format\\("
    reference:
      url: "https://owasp.org/Top10/A03_2021-Injection/"
      document: "OWASP Top 10 2021"
      section: "A03:2021 – Injection"
    suggestion: "Use cursor.execute(query, params) with placeholders"
    examples:
      bad: 'cursor.execute("SELECT * FROM users WHERE id = %s" % user_id)'
      good: 'cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))'

  - id: "NAME-001"
    title: "Function names must use snake_case"
    category: naming
    severity: warning
    description: "Function names should follow snake_case convention."
    pattern: "def\\s+[a-z]+[A-Z]"
    reference:
      url: "https://peps.python.org/pep-0008/#function-and-variable-names"
      document: "PEP 8"
      section: "Function and Variable Names"
    suggestion: "Rename function to use snake_case"

  - id: "DOC-001"
    title: "Public functions must have docstrings"
    category: documentation
    severity: info
    description: "All public functions should have documentation."
    pattern: "^def\\s+[^_].*:\\s*$\\n\\s*[^\"']"
    reference:
      document: "Internal Guidelines"
      section: "Documentation Standards"

Import and Validate Documents

> Import standards document company_standards.yaml

╭─────────────── Standards Import ──────────────────────────╮
│                                                            │
│  Document: Company Coding Standards v2.0                   │
│  Version: 2.0.0                                            │
│  Rules imported: 15                                        │
│                                                            │
│  Categories:                                               │
│    - naming: 4 rules                                       │
│    - security: 5 rules                                     │
│    - style: 3 rules                                        │
│    - documentation: 2 rules                                │
│    - error_handling: 1 rule                                │
│                                                            │
│  Status: SUCCESS                                           │
│                                                            │
╰────────────────────────────────────────────────────────────╯

Rule Categories

Category Description Example Rules
naming Naming conventions snake_case, CONSTANT_CASE
security Security practices Input validation, SQL injection
style Code style Line length, indentation
documentation Documentation Docstrings, comments
error_handling Error handling Exception handling, logging

Severity Levels

Severity Description Action
error Must fix before merge Block CI/CD
warning Should fix Flag in review
info Consider fixing Informational

Code Analysis

Check Against Rules

> Check src/api/ against standards

╭─────────────── Standards Check Results ───────────────────╮
│                                                            │
│  Summary:                                                  │
│  - Total Violations: 23                                    │
│  - Errors: 5                                               │
│  - Warnings: 12                                            │
│  - Info: 6                                                 │
│                                                            │
│  Violations:                                               │
│                                                            │
│  - **SEC-001**: SQL injection risk                         │
│    `src/api/user_handler.py:67`                            │
│    *Suggestion: Use cursor.execute(query, params)*         │
│                                                            │
│  - **SEC-001**: SQL injection risk                         │
│    `src/api/data_handler.py:34`                            │
│    *Suggestion: Use cursor.execute(query, params)*         │
│                                                            │
│  - **NAME-001**: Function uses camelCase                   │
│    `src/api/helpers.py:23`                                 │
│    *Suggestion: Rename function to use snake_case*         │
│                                                            │
│  - **DOC-001**: Missing docstring                          │
│    `src/api/utils.py:45`                                   │
│    *Suggestion: Add function documentation*                │
│                                                            │
│  ... and 19 more violations                                │
│                                                            │
╰────────────────────────────────────────────────────────────╯

Violation Report with References

> Show details for violation SEC-001 at user_handler.py:67

╭─────────────── Violation Details ─────────────────────────╮
│                                                            │
│  Rule: SEC-001                                             │
│  Title: Use parameterized queries for database operations  │
│  Severity: ERROR                                           │
│  Category: security                                        │
│                                                            │
│  Location: src/api/user_handler.py:67                      │
│                                                            │
│  Violation:                                                │
│  ```python                                                 │
│  cursor.execute("SELECT * FROM users WHERE id = %s" % id)  │
│  ```                                                       │
│                                                            │
│  Suggestion:                                               │
│  Use cursor.execute(query, params) with placeholders       │
│                                                            │
│  Reference:                                                │
│  OWASP Top 10 2021 - A03:2021 – Injection                  │
│  https://owasp.org/Top10/A03_2021-Injection/               │
│                                                            │
╰────────────────────────────────────────────────────────────╯

Filter by Category or Severity

> Check src/ for security errors only

╭─────────────── Security Errors ───────────────────────────╮
│                                                            │
│  Filter: category=security, severity=error                 │
│  Total violations: 5                                       │
│                                                            │
│  - **SEC-001**: SQL injection in user_handler.py:67        │
│  - **SEC-001**: SQL injection in data_handler.py:34        │
│  - **SEC-002**: Missing input validation in api.py:89      │
│  - **SEC-003**: Hardcoded credentials in config.py:12      │
│  - **SEC-004**: Insecure deserialization in parser.py:56   │
│                                                            │
╰────────────────────────────────────────────────────────────╯

Composite Mode (Orchestrator)

Scenario 19 can operate as a composite orchestrator, combining compliance checking with file editing:

┌─────────────────────────────────────────────────────────────┐
│              S19 COMPOSITE ORCHESTRATOR                      │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │              STANDARDS DOCUMENT                         │ │
│  │                 (YAML rules)                            │ │
│  └────────────────────────────────────────────────────────┘ │
│                         ↓                                   │
│  ┌────────────────────────────────────────────────────────┐ │
│  │             SCENARIO INVOCATION                         │ │
│  │       ┌─────────┐           ┌─────────┐                │ │
│  │       │   S08   │           │   S17   │                │ │
│  │       │Complian.│           │  Edit   │                │ │
│  │       └────┬────┘           └────┬────┘                │ │
│  └────────────┼─────────────────────┼─────────────────────┘ │
│               │                     │                       │
│               ↓                     ↓                       │
│  ┌────────────────────────────────────────────────────────┐ │
│  │               DOCUMENT ENRICHMENT                       │ │
│  │   (Violations linked to official references)            │ │
│  └────────────────────────────────────────────────────────┘ │
│                         ↓                                   │
│                  ACTIONABLE REPORT                          │
└─────────────────────────────────────────────────────────────┘

Sub-Scenarios

Scenario Contribution
S08 (Compliance) Standard compliance rules
S17 (File Editing) Apply fixes

Running Composite Mode

# CLI
codegraph composition run "Check against company_standards.yaml" -o s19

# API
POST /api/v1/composition/query
{
  "query": "Check against company_standards.yaml",
  "orchestrator": "scenario_19",
  "context": {
    "document_path": "company_standards.yaml",
    "file_paths": ["src/"]
  }
}

CLI Commands

# Import standards document
codegraph standards import company_standards.yaml

# Validate document syntax
codegraph standards validate company_standards.yaml

# List imported rules
codegraph standards list

# List rules by category
codegraph standards list --category security

# Check code against standards
codegraph standards check src/

# Check with filters
codegraph standards check src/ --severity error
codegraph standards check src/ --category security

# Export violations report
codegraph standards report src/ --output violations.json

# Clear imported rules
codegraph standards clear

TUI Commands

Command Description
/standards import <file> Import standards document
/standards list List available rules
/standards check <path> Check code against rules
/standards report Generate violation report
/standards clear Clear imported rules

Example Questions

  • “Import standards from company_standards.yaml”
  • “Check src/ against the imported standards”
  • “List all security-related rules”
  • “Show violations of severity error only”
  • “What rules apply to the naming category?”
  • “Generate a compliance report for src/api/”

Creating Custom Standards

Minimal Document

name: "My Standards"
version: "1.0.0"

rules:
  - id: "CUSTOM-001"
    title: "Avoid magic numbers"
    category: style
    severity: warning
    pattern: "\\b\\d{3,}\\b"
    suggestion: "Extract to named constant"

Full Document Template

name: "Complete Standards Template"
version: "1.0.0"
last_updated: "2024-12-09"
author: "Your Team"
description: |
  Comprehensive coding standards for the project.

# Categories used in this document
categories:
  - naming
  - security
  - style
  - documentation
  - error_handling
  - performance

# Global configuration
config:
  default_severity: warning
  file_patterns:
    - "*.py"
    - "*.js"
  exclude_patterns:
    - "*_test.py"
    - "test_*.py"

rules:
  - id: "RULE-001"
    title: "Rule title"
    category: security
    severity: error  # error, warning, info
    description: "Detailed description of the rule."
    pattern: "regex pattern"  # Optional: regex for detection
    reference:
      url: "https://example.com/doc"
      document: "Document Name"
      section: "Section 3.2"
    suggestion: "How to fix the violation"
    examples:
      bad: "Example of violation"
      good: "Example of correct code"
    tags:
      - "owasp"
      - "critical"

Integration with CI/CD

# .github/workflows/standards.yml
name: Standards Check
on: [pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run Standards Check
        run: |
          codegraph standards import standards.yaml
          codegraph standards check src/ --severity error --exit-code