Skip to main content

Production monitoring

This guide connects CodeGraph to an external Prometheus and Alertmanager stack. It deliberately references maintained artifacts instead of copying the.

User Guides

This guide connects CodeGraph to an external Prometheus and Alertmanager stack. It deliberately references maintained artifacts instead of copying the metric catalog or alert expressions into documentation.

Two metrics surfaces

CodeGraph exposes two different Prometheus responses:

Two metrics surfaces
Request Purpose Access
GET /metrics Process and HTTP runtime metrics from the API application. Exposed without API user authentication so a protected network scraper can collect it.
GET /api/v1/traceability/metrics Materializes and exports project/portfolio dashboard metrics for the authenticated scope. Requires API authentication and project context.

Treat the first route as network-sensitive even though it does not use user authentication. Restrict it with the deployment network policy, ingress, or scraper allowlist. Do not publish it on an unrestricted internet ingress.

The metric names and labels are defined in src/monitoring/metrics.py. Do not pin an expected metric count in an acceptance check: optional components and new collectors can change the exported set.

Scrape verification

From the monitoring network:

curl.exe -fsS http://<api-host>:8000/metrics

For dashboard metrics, use an approved API credential and project context:

curl.exe -fsS -H "Authorization: Bearer <token>" -H "X-Project-Id: <project-id>" http://<api-host>:8000/api/v1/traceability/metrics

Verify the HTTP status, Prometheus content type, a current scrape timestamp in Prometheus, and the expected target labels. This set of checks confirms that Prometheus ingested the series.

Shipped configuration artifacts

Shipped configuration artifacts
Artifact Operator use
monitoring/prometheus.yml Reference scrape jobs and rule-file loading. Adapt target names to the deployment topology.
monitoring/rules/alerts.yml Runtime, API, host, queue, storage, and integration alerts.
monitoring/rules/dashboard_alerts.yml Project health, compliance, release, and SCA alert rules.
monitoring/alertmanager.customer-baremetal.yml Customer bare-metal baseline with a local receiver and no external destination.
integrations/grafana/dashboard_ciso.json Importable CISO/CTO Grafana dashboard.
monitoring/yandex/dashboard.json Yandex Monitoring dashboard definition.

The repository Helm chart creates the application workload, service, ingress, storage, and configuration. It does not create Prometheus Operator custom resources. Configure scraping in the customer’s monitoring platform or add an approved deployment overlay outside the chart’s documented baseline.

Alert evaluation

Load both rule files and validate them with the Prometheus version used by the customer. Dashboard defaults currently include:

  • DashboardProjectRiskCritical: health below 40 for 5 minutes;
  • DashboardProjectRiskHigh: health from 40 to below 55 for 10 minutes;
  • DashboardComplianceGap: compliance below 60 for 1 hour;
  • DashboardReleaseGateFail: a failing release gate;
  • DashboardScaCriticalVuln: at least one critical SCA vulnerability.

The YAML files remain authoritative if these values change. Route warning and critical events according to the customer’s approved escalation policy.

Receiver safety

monitoring/alertmanager.customer-baremetal.yml intentionally contains no external webhook, mail, or messenger receiver. Before adding one:

  1. approve the destination and data classification;
  2. store credentials in the customer’s secret system;
  3. decide whether finding details may leave the deployment boundary;
  4. send a synthetic alert;
  5. verify a receiver-side delivery-success event;
  6. verify resolution delivery and inhibition behavior.

The generic monitoring/alertmanager.yml can describe development or managed integrations and must not be copied into a customer environment without this review.

Dashboard import and validation

Import integrations/grafana/dashboard_ciso.json into the customer’s Grafana and bind its Prometheus data source. For Yandex Monitoring, import or adapt monitoring/yandex/dashboard.json.

After deployment, verify:

  1. the API target is up and scraping GET /metrics;
  2. dashboard metrics are collected for an authorized project scope;
  3. both rule groups load without errors;
  4. dashboards return current series rather than empty panels;
  5. one warning and one critical synthetic alert reach the approved receiver;
  6. receiver secrets and private labels are absent from logs and screenshots.

Record the deployed image or commit, Prometheus config revision, rule digest, dashboard revision, test-alert identifiers, and receiver delivery evidence.

Troubleshooting order

  1. Query the CodeGraph endpoint from the scraper network.
  2. Inspect Prometheus target status and last error.
  3. Confirm target labels and project scope.
  4. Check rule loading and evaluation errors.
  5. Check Alertmanager routing and inhibition.
  6. Check the destination’s delivery evidence.

Keep dashboard data freshness separate from scrape health. A healthy Prometheus target can still export stale project evidence; use Dashboard operations for that recovery path.

Sources of truth

  • src/api/main.py — public process metrics route and middleware wiring.
  • src/api/routers/dashboard_core/dashboard_v2_metrics.py — authenticated dashboard metrics route.
  • src/monitoring/metrics.py — metric names, labels, and update helpers.
  • the shipped files listed in the artifact table — scrape, rule, receiver, and dashboard configuration.
Report a documentation issue