Skip to main content

How GoCPG builds a CPG and how the approach differs from Joern

GoCPG's CPG pipeline, Python and JavaScript type recovery, structural analysis, and rules for a defensible Joern comparison.

Enterprise

A Code Property Graph is not a file map or the output of a single parser. It is a unified program model that connects syntax to control flow, calls, types, data dependencies, and source locations. CodeGraph uses these relationships for defect detection, architecture enforcement, and audit metrics.

This article answers two questions:

  1. how the current GoCPG pipeline builds and enriches the graph;
  2. what can be compared fairly with Joern without unsupported performance or accuracy claims.

The formulas that turn graph findings and metrics into Q1–Q12 and the Health Score are documented in the Dashboard methodology.

The GoCPG boundary

GoCPG builds and stores the project CPG. A language frontend creates the base nodes and relationships; ordered passes add semantic layers. The result is persisted in DuckDB, and CodeGraph consumes typed results through service contracts. The application layer does not construct a second canonical graph.

Pipeline composition comes from the current code and selected profile. This document therefore does not advertise a fixed pass count: domain annotations, VCS enrichment, and profile-specific analyses can extend the standard set.

1. From source code to a base graph

Frontend selection

GoCPG registers frontends for C, C++, Go, Python, JavaScript, TypeScript, Java, Kotlin, C#, PHP, and 1C. Most use tree-sitter. Go uses go/parser and go/ast; 1C uses a custom lexer/parser. Some languages provide a reduced fallback path when full parser support is unavailable.

A frontend extracts facts that are directly visible in source:

  • files, namespaces/modules, types, methods, parameters, and locals;
  • calls, literals, identifiers, blocks, and control structures;
  • imports, annotations, comments, and source locations;
  • AST, argument, receiver, and source-file relationships.

Parser success is not the same as complete analysis. A frontend can identify the syntax of a dynamic call while its target remains unknown until type recovery and import resolution run.

Materialization

A full build follows this path:

  1. GoCPG discovers files and applies include/exclude rules plus the selected frontend’s extensions.
  2. The frontend parses files in parallel and produces a DiffGraph. Per-file failures remain visible.
  3. Frontend IDs are synchronized with the global generator so pass-created nodes cannot collide with parsed nodes.
  4. The base diff is written to DuckDB and becomes an in-memory CPGGraph for downstream passes.
  5. Semantic passes read established layers, add nodes and edges, and persist their mutations.

A partial parse, skipped file, or unavailable frontend limits the evidence basis; it is not a clean result.

2. The semantic pipeline

Ordering matters: a call graph cannot reliably precede import and type recovery, and a PDG cannot precede control- and data-flow layers.

2. The semantic pipeline
Stage Added facts Primary use
Base topology metadata, files, namespaces, type nodes/declarations, inheritance, parameter links Stable entity and type identity.
Type recovery identifier and call types, plus origin/confidence for call nodes Better FQNs and call targets in dynamic languages.
Control flow CFG, dominators, post-dominators, CDG Reachability, branching, and control dependence.
Imports and calls import maps, Call FQN propagation, call resolution, dynamic dispatch, FFI edges Cross-file and cross-language call relationships.
Data flow references, alias analysis, reaching definitions, interprocedural propagation Value origin, side effects, and paths across methods.
Risk analyses uninitialized variables, ranges, resource leaks, null dereference, concurrency, taint, integer/buffer overflow Findings tied to graph paths and source.
Program dependencies EVAL_TYPE, return type propagation, PDG, and DDG Combined control- and data-dependency analysis.
Object model bindings, vtables, dynamic call linking Interface, virtual-dispatch, and constructor-to-initializer relationships.
Enrichment method metrics, comments, containment, findings, structural pattern matching Audit metrics, pattern search, and explainable results.
Optional layers domain annotations, use-case propagation, VCS tags Domain context, entry points, authorship, and change frequency.

Every pass declares dependencies and incremental support. Independent DAG branches may run concurrently, but concurrency does not bypass semantic order.

3. Type recovery for Python and JavaScript

In a compiled project, a declaration or toolchain often supplies a type. In Python and JavaScript, an identifier’s type may exist only at runtime. GoCPG does not pretend that this uncertainty disappears: it collects static evidence, applies it iteratively, and records origin and confidence where the schema supports them.

Sources of type facts

The first, intraprocedural stage uses:

  • annotations and already extracted type declarations;
  • assignments such as x = ClassName(...);
  • constructor calls;
  • locals and same-name identifiers;
  • self/this fields and member assignments;
  • Python isinstance guards;
  • builtin and method return types from a language-specific Type Knowledge Base.

Python and JavaScript have separate builtin/type-stub sets. They are not a full runtime environment: user metaprogramming, monkey patching, eval, dynamic imports, and arbitrary object-shape changes can remain unresolved.

Iteration and convergence

Later iterations propagate:

  • a resolved method’s return type to its call site;
  • a call result type into assignment targets;
  • types from return statements and properties;
  • dynamic-dispatch facts from the Type Knowledge Base;
  • a temporary <returnValue> type when the call target is known but its exact return type is not.

Updates are collected concurrently and then applied deterministically. The default configuration preserves three base iterations, while the adaptive loop may continue for up to ten. After the third iteration it stops when the ratio of newly applied updates to the previous iteration is below 1%, or earlier when there are no updates.

Call nodes persist type_origin and type_confidence. Example origins include builtin, tkb, constructor, propagated, and inferred. Constructor and builtin facts carry higher confidence; a temporary <returnValue> carries less. Downstream analysis can distinguish a direct fact from an approximation.

How recovered types improve the call graph

After type recovery, a dedicated pass rewrites a call’s method_full_name from the recovered receiver type and import map. Call resolution then links the call node to the best available method node, and the dynamic linker adds virtual-dispatch and constructor-to-__init__ relationships.

If evidence is insufficient, the call remains unresolved or approximate. A call edge does not prove runtime reachability, and a missing edge in highly dynamic code does not prove the call is impossible. Security and architecture rules must consider confidence and completeness.

4. Control and data flow

GoCPG materializes several connected representations:

  • CFG captures possible execution order within a method;
  • dominator and post-dominator trees support mandatory-path analysis;
  • CDG connects a node to the condition that controls its execution;
  • REF and reaching-definition edges connect a use to its value;
  • alias analysis refines references that may denote the same object;
  • interprocedural reaching definitions propagate facts through call sites;
  • DDG materializes data dependencies;
  • PDG combines data and control dependencies.

Taint, resource-leak, null-dereference, and related analyzers operate on these layers. A finding should therefore retain a message, source location, related node, and an evidence path where applicable.

Different CPG implementations can materialize different helper nodes and edges. Raw node, CALL-edge, or CDG-edge counts are not accuracy metrics without labeled expected facts.

5. Architecture v1 structural analysis

The current structural analyzer goes beyond import search. It classifies CPG entities into components and evaluates which relationships are allowed between them.

Model and rules

  • Every eligible entity receives a stable component UID. Display id is mutable; uid participates in identity and finding fingerprints.
  • Parent components group leaves but do not classify entities themselves.
  • Scope, layer, type, platform, and runtime are independent dimensions. Enforcement requires exhaustive, unambiguous classification.
  • Architecture rules use spec_version: "2.0", kind: architecture, a stable rule ID, and all, any, and not selectors.
  • Preset and rule references are version-pinned as id@version. latest and implicit inheritance from a neighboring language are rejected.

Architecture v1 contains three generic templates and nine language presets: C, C++, C#, Go, JavaScript, Kotlin, PHP, Python, and TypeScript. JavaScript and TypeScript have independent capability manifests and acceptance status.

Completeness, baselines, and waivers

The capability registry declares the fact families supplied by each language adapter. A missing required capability cannot be averaged away: enforcement returns BLOCKED_INCOMPLETE.

An incremental run is authoritative only as incremental_complete when its fingerprints, baseline/waiver states, and semantic digest equal the corresponding full run. Public API, model, rule, baseline, waiver, edge, or SCC changes expand invalidation. A partial result triggers one synchronous full retry; only a fresh full result may replace it.

A baseline binds accepted debt to an exact rule, fingerprint version, finding fingerprint, and semantic hash. A file in the analyzed repository cannot suppress a finding by itself. A waiver is time-bounded, requires distinct requester and approver identities, and signs the project, repository, commit, branch, hashes, and revision. Trust keys and revocation state come from the control plane or CI trust configuration, not the analyzed repository.

Results and release gates

Canonical JSON contains the revision-bound run, completeness evidence, an outcome for every active rule, findings, coverage, and metrics. SARIF, Mermaid, and PlantUML are deterministic projections of that result and do not change fingerprints. A read-only gRPC QueryService supports list/export/validate/describe/explain operations for persisted runs.

discovery and advisory profiles do not authorize a release. enforce_new, enforce_all, and strict may return PASS or FAIL_VIOLATIONS, but only with fresh, complete facts and sufficient capability coverage. Missing, stale, partial, failed, or zero-denominator evidence yields BLOCKED_INCOMPLETE.

These results primarily inform audit sections Q2, Q4, Q6, and Q11; they do not replace other audit scenarios.

6. GoCPG and Joern: architecture comparison

Joern also builds CPGs through language frontends, adds semantic layers through passes, and exposes a Scala-based CPG Query Language. Its official documentation describes an interactive shell, scripts, server mode, extensible passes, and plugins. Frontends and overlays vary by release and must be pinned for an evaluation.

6. GoCPG and Joern: architecture comparison
Dimension GoCPG in CodeGraph Joern
Primary workflow Project import, automated analyzers, Dashboard, CLI/MCP/REST, and governed evidence. Interactive and query-driven vulnerability research, scripts, scan, and server workflows.
Storage Project-scoped DuckDB, atomic writes, and typed service contracts. A custom in-memory graph database and Joern project model.
Query surface Go services, gRPC, and controlled project interfaces; SQL remains inside the storage boundary. Scala-based CPGQL, REPL, scripts, and an HTTP server.
Extension model Language frontends, dependency-aware Go passes, pattern/architecture rules, domain and VCS layers. Frontends, CPG passes, query extensions, and JVM plugins.
Dynamic types Iterative shared pass, language-specific type stubs, origin/confidence, and FQN propagation. Behavior depends on the pinned frontend and overlays.
Structural enforcement Versioned model/presets, capability registry, signed governance, full/incremental equivalence, and a fail-closed gate. Architecture queries and extensions can be built with CPGQL, passes, and plugins; the policy workflow must be evaluated separately.

There is no universal winner. Joern fits teams that perform interactive analysis and have invested in CPGQL. GoCPG is integrated into CodeGraph’s automated lifecycle, where project-scoped persistence, reproducible findings, freshness, governance, and release evidence matter.

Official Joern sources:

7. Why the old performance numbers are not repeated

An earlier revision of this article contained exact timings, node/edge counts, and conclusions about Python call graphs. Those measurements belonged to old GoCPG and Joern versions, two specific checkouts, and a known Python path normalization issue. Without the original manifest, raw logs, corpus digest, and a fresh run, the values cannot describe the current product.

The useful technical structure of that article has been restored. Its numerical conclusions have deliberately not been promoted as current facts. This is an evidence boundary, not a refusal to compare.

8. Reproducible benchmark protocol

Every numerical conclusion needs an immutable benchmark manifest:

  • GoCPG/CodeGraph commit, exact Joern release, and artifact digests;
  • corpus commit, languages, include/exclude rules, and labeled expected facts;
  • hardware, OS, runtime limits, cache state, and concurrent load;
  • exact import, query, export, and cleanup commands;
  • cold/warm policy, repetitions, timeouts, failures, and outlier treatment;
  • wall time, CPU, peak memory, storage, and result completeness;
  • raw logs, machine-readable output, digests, and an independent reviewer.

Compare more than import time and raw graph size:

  1. parser/import completeness;
  2. correctness and completeness against labeled facts;
  3. post-import query latency;
  4. end-to-end time to an actionable result;
  5. unresolved calls and unsupported constructs;
  6. memory, storage, and incremental-update behavior;
  7. operator effort, deployment, security, and upgrade/rollback fit.

Run each tool in a clean workspace suited to that tool. Failed imports, partial graphs, and manual tuning remain part of the result. Without a manifest, only verified architecture and workflow conclusions are defensible.

GoCPG sources of truth

  • gocpg/pkg/frontend and gocpg/docs/frontend-guide.md — frontends and extracted facts.
  • gocpg/pkg/cpg/schema — nodes, edges, and properties.
  • gocpg/pkg/passes/pipeline.go — current semantic pass ordering.
  • gocpg/pkg/passes/types/type_recovery*.go and gocpg/pkg/typestubs — type recovery and the Type Knowledge Base.
  • gocpg/pkg/passes/controlflow, dataflow, and callgraph — CFG/CDG, data flow, and call graph construction.
  • gocpg/pkg/architecture and gocpg/docs/architecture/v1/ — structural analysis, governance, and result formats.
  • gocpg/pkg/storage/duckdb and gocpg/api/proto/gocpg/v1 — persistence and typed integration.