dynaxx on the hidden graph: interpretability via circuit-level abstraction

If you have ever stared at a model's attention patterns and felt no closer to understanding why it made a specific prediction, you are not alone. Raw activations are noise; the signal lives in the hidden graph of functional circuits. Circuit-level abstraction—isolating the minimal subgraph that drives a behavior—is the most reliable method we have for turning interpretability from a visualization exercise into an engineering tool. But the choice between manual, automated, and hybrid approaches is not obvious, and the wrong choice wastes weeks. This guide is for teams that already know what attention heads and MLP neurons are and need a decision framework for choosing an abstraction strategy that actually ships.

Why circuit-level abstraction matters now

Large language models are not monolithic reasoners; they are compositions of specialized circuits that handle syntax, factual recall, multi-step reasoning, and output formatting. When a model answers a question correctly, it is because a specific set of attention heads and MLP layers activated in a particular sequence. When it fails, it is because a circuit misfired or a necessary subcircuit was absent. Understanding these circuits is the difference between patching a bug and guessing at hyperparameters.

Circuit-level abstraction is the practice of identifying the minimal subgraph of the model's computational graph that is necessary and sufficient for a given behavior. This subgraph—often called a "circuit"—can be analyzed, edited, or compared across models. The goal is not to explain every neuron but to isolate the functional backbone that drives the behavior you care about.

For a production team, this matters because it enables targeted interventions. If you know that a factual recall failure is caused by a specific attention head in layer 7 not attending to the correct token, you can investigate that head's training data, adjust its position encoding, or even patch in a head from a fine-tuned model. Without circuit-level abstraction, you are debugging in the dark.

The catch is that circuit discovery is still a young field, and the tools are uneven. The three main approaches—manual, automated, and hybrid—each carry different trade-offs in scalability, fidelity, and interpretability of the result. Choosing between them depends on your model size, your tolerance for false positives, and whether you need to act on the circuit or just understand it.

Three approaches to circuit discovery

Manual circuit discovery

Manual discovery is the original method: you start with a behavior, hypothesize which components matter, and test by ablating or patching activations. This is the approach used in early mechanistic interpretability papers on small transformers. It yields high-fidelity circuits because every edge is verified by intervention, but it does not scale. For a single behavior on a 7B model, a skilled practitioner might need several days to trace a circuit of 20–30 nodes. The process is labor-intensive and hard to parallelize.

Automated circuit extraction

Automated methods use gradient-based attribution or activation patching to propose circuits without human guesswork. Tools like automatic circuit discovery (ACDC) or edge attribution patching (EAP) score every edge in the computational graph and return the top-k edges that most affect the output. These methods are fast—they can process a 7B model in hours—but they suffer from high false-positive rates. Edges that are correlated with the behavior but not causally necessary often appear in the top-k list. The resulting circuit is a superset of the true circuit, and cleaning it requires manual verification.

Hybrid analysis

Hybrid approaches combine automated attribution to narrow the search space with manual intervention to verify. A typical workflow: run EAP to get a candidate graph of 100–200 edges, then manually ablate subsets to confirm necessity. This balances speed and fidelity. The automated step reduces the search space by two orders of magnitude, and the manual step filters out the false positives. For most production use cases, hybrid analysis offers the best return on time invested.

Criteria for choosing your approach

Not every team needs the same level of circuit fidelity. The decision depends on three factors: model size, behavior complexity, and the actionability of the result.

Model size is the most obvious constraint. Manual discovery on a 70B model is impractical for all but the most critical behaviors. If your model is under 1B parameters, manual methods are feasible and give the cleanest results. For models between 1B and 10B, hybrid is the sweet spot. Above 10B, automated methods are your only option unless you have a dedicated research team.

Behavior complexity matters because simple behaviors (e.g., "does the model output the capital of France?") have small circuits that manual discovery can handle. Complex behaviors (e.g., multi-hop reasoning or instruction following) involve dozens of interacting subcircuits. Automated methods are better at surfacing the full graph, but they require more careful validation.

Actionability is the hardest criterion. If you need to edit the circuit—for example, to patch in a fix or to remove a bias—you need high-fidelity causal evidence. Manual verification of each edge is non-negotiable. If you only need to understand why a model failed on a specific input, a noisy automated circuit might be sufficient to guide your next training data collection.

We recommend scoring each candidate behavior on a 1–5 scale for these three factors and using a simple weighted sum to decide the approach. For example, if model size is 4 (large) and behavior complexity is 3 (moderate) but actionability is 5 (must edit), the hybrid approach is the only viable choice.

Trade-offs in practice: a structured comparison

The table below summarizes the key trade-offs across the three approaches. We have included typical values based on common practitioner reports; your mileage will vary with model architecture and tooling maturity.

The hybrid approach occupies a middle ground that is often the most practical for teams that need both speed and reliability. The main drawback is the lack of standardized tooling—you will likely need to build your own pipeline that connects an automated attribution step with a manual verification interface.

One common mistake is to assume that automated circuits are ready to use without verification. In one typical project, a team used EAP to extract a circuit for a safety behavior and found that over 40% of the top-50 edges were not causally necessary when tested with activation patching. The automated circuit was a useful starting point, but acting on it directly would have led to incorrect conclusions about which components to modify.

Implementation path after choosing your approach

Once you have selected an approach, the implementation follows a consistent pattern regardless of method. We outline the steps below, noting where each approach diverges.

Step 1: Define the behavior precisely

Circuit discovery requires a clean specification of the behavior. This means a set of input-output pairs where the model succeeds and a set where it fails. The behavior should be narrow: "answering questions about US presidents" is too broad; "outputting the birth year of a US president given the name" is better. For each behavior, create at least 50–100 examples to use for attribution.

Step 2: Run attribution (automated or manual)

For manual discovery, this step involves hypothesizing which layers and heads matter based on existing knowledge or probing. For automated methods, run your chosen attribution algorithm on the behavior examples. For hybrid, run the automated method first, then filter the top-k edges (we recommend k=200 for a 7B model).

Step 3: Verify with causal intervention

Every edge in your candidate circuit must be tested by ablation or activation patching. Ablate the edge (set its activation to zero) and check if the behavior degrades. If it does not, the edge is not necessary. For manual and hybrid approaches, this step is built in. For automated approaches, you must budget time for verification—expect to spend 80% of your total effort here.

Step 4: Minimize the circuit

Once you have a set of verified edges, attempt to remove edges while maintaining the behavior. This is the "minimality" step. Often, the circuit can be reduced by 30–50% without affecting performance. The minimal circuit is easier to analyze and more likely to generalize to similar inputs.

Step 5: Document and share

Circuit diagrams are useful, but they rot quickly if not documented. For each circuit, record the behavior definition, the list of edges (with layer and head indices), the verification method, and any failure cases where the circuit does not explain the output. Share this with your team in a format that can be queried (e.g., a JSON file with metadata).

Risks if you choose wrong or skip steps

The most common failure mode is overconfidence in automated circuits. Teams that skip manual verification often end up with a circuit that looks plausible but fails to predict intervention outcomes. This leads to wasted engineering effort on modifying the wrong components and, worse, a false sense of understanding.

Another risk is focusing on circuits that do not generalize. A circuit discovered on a narrow set of inputs may not activate on slightly different prompts. For example, a circuit that handles "What is the capital of France?" might fail on "What is the capital of France, and what is its population?" because the model uses a different subcircuit for multi-query prompts. Always test your circuit on held-out inputs that vary in syntax, length, and context.

Ignoring interaction effects is a third pitfall. Circuits are not independent; the effect of ablating one edge can be compensated by other edges. If you test edges one at a time, you may miss synergistic dependencies. We recommend testing edges in combination, especially for pairs that are close in the computational graph.

Finally, there is the risk of conflating correlation with causation. Attribution methods like gradient-based scores measure how much the output changes when an edge is perturbed, but they do not prove that the edge is part of a functional circuit. An edge might be important because it carries a signal that is later overwritten by another edge. The only way to establish causation is through intervention—ablation or patching—and even then, the results are conditional on the specific input distribution.

If your team is new to circuit analysis, start with a simple behavior on a small model. Do not attempt to interpret a 70B model's safety behavior on your first try. The learning curve is steep, and the cost of mistakes is high. Budget at least two weeks for the first circuit, including time for false starts and tooling setup.

Mini-FAQ on circuit-level abstraction

Do circuits transfer across model architectures?

Sometimes, but not reliably. Circuits that rely on positional encoding patterns (e.g., attending to the previous token) often transfer between models with the same architecture family. Circuits that use specific factual knowledge (e.g., a head that stores the capital of France) are unlikely to transfer because the knowledge is stored in different weights. In general, assume a circuit is architecture-specific unless you have evidence otherwise.

How do I validate that a discovered circuit is correct?

The gold standard is to use the circuit to predict the model's output on unseen inputs. If you can ablate the circuit and cause the model to fail on a held-out set, and if you can restore the circuit (via patching) and recover the behavior, you have strong evidence. A weaker validation is to check that the circuit's edges have high attribution scores on a separate test set.

When should I skip circuit-level abstraction entirely?

If you only need to know whether a model is biased on a specific dimension, a behavioral test (e.g., measuring accuracy on different demographic groups) is faster and more reliable. Circuit analysis is worth the effort only when you need to understand the mechanism—for example, to fix a specific failure mode or to ensure that a safety intervention does not break unrelated capabilities.

What tools are available for automated circuit discovery?

Several open-source libraries exist, including TransformerLens (for activation patching and attribution) and the ACDC implementation from the Mechanistic Interpretability group. These tools work best on models with standard architectures (GPT-2, Pythia, Llama). For custom architectures, you may need to implement the attribution methods yourself.

Can I use circuit analysis on vision models?

Yes, the same principles apply to convolutional networks and vision transformers. The main difference is that the computational graph includes spatial dimensions, so circuits are often larger and harder to visualize. Most of the tooling is language-model-focused, but the concepts transfer.

Recommendation recap without hype

Circuit-level abstraction is a powerful tool, but it is not a silver bullet. For most production teams, we recommend starting with a hybrid approach: use automated attribution to generate candidate circuits, then invest the time to verify the top edges manually. This gives you the best balance of speed and reliability.

If you are working with models under 1B parameters and have a clear behavior, manual discovery is feasible and yields the highest fidelity. For models above 10B, automated methods are your only option, but you must budget significant time for verification—do not trust the raw output.

Here are three specific next moves you can make this week:

1. Pick one behavior that your model currently fails on and define it precisely with 50–100 input-output pairs. Do not choose a safety-critical behavior for your first attempt; choose something simple like a factual recall task.
2. Run an automated attribution method (e.g., edge attribution patching) on that behavior using an existing library. Examine the top-20 edges and see if they form a plausible circuit. This will take a few hours and will teach you the practical challenges of the method.
3. Manually verify three edges from the automated output using activation patching. If the automated circuit holds up, you can scale to more edges. If it does not, you have learned the importance of verification without wasting weeks on a false lead.

Circuit-level abstraction is not the only interpretability technique, but it is the one that gives you causal leverage. Use it when you need to act, not just to understand. And always remember: the hidden graph is a tool, not a truth—it is a model of the model, and like all models, it is wrong in ways you will discover only when you test it.