Concept-Direction Steering Demo (local Neuronpedia)#

Demonstrates interpretune’s concept-direction-mediated, sign-aware, multi-feature steering on a trivial example, orange color-vs-fruit sense disambiguation to familiarize the user with some of interpretune’s intervention mechanisms:

Feature-mediated path: concept direction -> attribution graph -> sign-aware FeatureSelectionSpec top-feature selection -> feature_intervention_forward (circuit-tracer feature interventions with sign-aware, influence-normalized scaling).
Direct-hook path: the same concept direction applied via model_fwd_intervention (hook-tensor add/project interventions at canonical hook points).

Both paths derive the concept direction from the token-embedding basis (paired_rejection over the concept groups).

The BACKEND parameter selects the circuit-tracer backend for all steps: NNsight (default) or TransformerLens — both are validated by the parameterized notebook tests. (The TransformerLens backend uses the legacy HookedTransformer path; circuit-tracer does not yet support TransformerBridge — see the tracking notes in docs/circuit_tracer_backend_support.md.)

This is the local-Neuronpedia variant: it runs the instruction-tuned gemma-3-1b-it substrate against a local Neuronpedia dev webapp + database, and resolves feature explanations from that local database (optionally generating any that are missing). It exercises the full local pipeline — dashboards -> explanations -> selection -> steering.

For the zero-setup version that uses the public neuronpedia.org dashboards and needs no local services, see ct_concept_steering_demo.ipynb.

Prerequisites: a GPU with bf16 support, the model/transcoder weights, and a running local Neuronpedia webapp + Postgres serving the gemmascope-2-transcoder-16k dashboards for gemma-3-1b-it (see docs/neuronpedia_dashboard_pipeline.md). Explanation generation is optional and additionally needs the explanation CLI + an API key — see the next cell.

Expected result — the three steering paths are not equally strong. The attribution-graph feature-mediated path (step 2) is the one selecting features for their causal effect on the target logit difference, and it should produce the largest target-gap shift. Direct-hook steering (step 4) applies a concept direction at a hook point without that per-feature attribution, and the optional user-curated path (step 5) picks features by human dashboard inspection — features chosen for what they fire on rather than what they write to. Both are expected to be weaker than the attribution-mediated result. That gap is a finding, not a defect: step 6’s decoupling analysis exists to explain it, showing curated picks typically carry high input-concept share but the smallest |output projections|.

Credentials for local explanation generation (optional)#

Only needed when GENERATE_MISSING_LOCAL_EXPLANATIONS=True (or REGENERATE_LOCAL_EXPLANATIONS=True). Everything else in this notebook runs without it — explanations already present in the local database are read directly.

Generation shells out to a conforming CLI (GitHub Copilot by default). Set these before starting the kernel, e.g. in the repo .env or your shell:

export IT_EXPLANATION_PROVIDER_API_KEY=sk-or-v1-...            # an OpenRouter key works here
export IT_EXPLANATION_CLI_MODEL=nvidia/nemotron-3-ultra-550b-a55b:free

Key precedence, highest first: IT_EXPLANATION_PROVIDER_API_KEY -> COPILOT_PROVIDER_API_KEY -> OPENROUTER_API_KEY. The endpoint follows the key, not the variable holding it, so an OpenRouter key (sk-or- prefix) routes to OpenRouter from any of them. Always set IT_EXPLANATION_CLI_MODEL with an OpenRouter key – the default model id belongs to another provider and will not resolve.

Set REGENERATE_LOCAL_EXPLANATIONS=True to re-generate features that already have an explanation. On an already-populated database every feature is otherwise skipped, so the run reports full coverage without the CLI being called once. Nothing is deleted.

# Parameters - These will be injected by papermill during parameterized test runs
BACKEND = "nnsight"  # circuit-tracer backend for all steps: "nnsight" or "transformerlens"
CONCEPT_PROMPT = "Is orange a color or a fruit? Answer with one word: Color or Fruit. orange ->"
CONCEPT_TARGET_TOKENS = ["Fruit", "Color"]
FEATURE_SELECTION_TOP_N = 5
FEATURE_SELECTION_MIN_LAYER = 10  # fs_l10_n5 lineage: layers >= 10
FEATURE_SELECTION_SCORE_SIGN = "any"  # any | positive | negative
INTERVENTION_SCALE_FACTOR = 20.0  # validated s5_any demo scale
EMBED_INTERVENTION_MODE = "add"  # model_fwd_intervention mode for the embed step
EMBED_INTERVENTION_HOOK = "unembed.hook_in"

# -- Model / dashboard substrate: gemma-3-1b-it + a LOCAL Neuronpedia dev webapp ----------------
# Dashboard/runtime width must match: the served source set has to correspond to the runtime
# transcoder_set width. Feature indices are only meaningful within one feature space.
REGISTRY_KEY = "gemma3.rte_demo.circuit_tracer_w_neuronpedia"
MODEL_NAME = "gemma-3-1b-it"
TRANSCODER_SET = None  # keep the registry default (gemma-scope-2, 16k width)
NEURONPEDIA_MODEL_ID = "gemma-3-1b-it"
NEURONPEDIA_SOURCE_SET = "gemmascope-2-transcoder-16k"  # matches the registry transcoder width (16k)
CHAT_FORMAT_PROMPT = True  # gemma-3-1b-it is instruction-tuned
LOCAL_WEBAPP_URL = "http://localhost:3000"
LOCAL_DB_URL = "postgres://postgres:postgres@127.0.0.1:5433/postgres"

# -- Local explanation generation (OPTIONAL) ----------------------------------------------------
# Everything above runs without these. Generation shells out to a conforming CLI and needs an API
# key -- see the credentials cell below.
GENERATE_MISSING_LOCAL_EXPLANATIONS = False  # backfill features that have no local explanation yet
REGENERATE_LOCAL_EXPLANATIONS = False  # also re-generate features that already have one

# -- Step 5 (user-curated feature steering) -- DISABLED BY DEFAULT ------------------------------
# Curated steering pins specific (layer, feature) ids that were found by manual dashboard search
# against one dashboard generation. Curated ids are tied to a specific (model, source set) pair --
# they index the transcoder's feature space, so they stay valid across regenerations of the SAME
# model/source set, but they do not carry across models, widths or source sets. The *explanations*
# that justified picking them are weaker: those describe the dashboard evidence, which does change
# when the corpus changes. Sharing curated picks reliably needs a shared dashboard artifact, which is
# not available yet (priority item on the IT roadmap).
#
# So it is OFF unless you supply ids for YOUR dashboards. To enable, fill in the two lists below with
# features you located in your own local webapp.
#
# This becomes a safe default once the generated dashboard artifacts are downloadable from the Hub,
# because then everyone can share one substrate with stable ids -- tracked for the IT MVP milestone.
CURATED_FEATURES_SOURCE_SET = "gemmascope-2-transcoder-16k"
CURATED_POSITIVE_FEATURES: list[tuple[int, int]] = []  # amplify; e.g. [(19, 11234)] on our 24,576-prompt 16k set
CURATED_NEGATIVE_FEATURES: list[tuple[int, int]] = []  # suppress; e.g. [(16, 199), (16, 13701), (17, 6499)]
CURATED_OVERRIDE_MAGNITUDE = 4.0  # fixed |activation| override per curated feature

# @title Imports { display-mode: "form" }
import torch  # noqa: F401

import interpretune.analysis  # noqa: F401  # ensure op wrappers are registered
from interpretune.analysis.backends import FeatureSelectionSpec  # noqa: F401
from interpretune.utils import ensure_local_feature_explanations, feature_tuples_to_feature_refs  # noqa: F401
from it_examples.utils.nb_ui_utils import (  # noqa: F401
    best_variant_token_ids,
    display_steering_results,
    display_target_gap,
    display_top_features_comparison,
    resolve_feature_explanations,
)

1. Session setup#

Single-backend circuit-tracer session built from the REGISTRY_KEY example-registry entry.

# @title 1: Session construction { display-mode: "form" }
from pathlib import Path

from dotenv import load_dotenv

import interpretune as it
from it_examples import _ACTIVE_PATCHES  # noqa: F401  # runtime analysis-hook patches
from it_examples.example_module_registry import MODULE_EXAMPLE_REGISTRY
from interpretune import ITSession, ITSessionConfig

# load HF credentials before session init (model + transcoder downloads)
for _env_candidate in (Path.cwd() / ".env", Path.home() / "repos" / "interpretune" / ".env"):
    if _env_candidate.exists():
        load_dotenv(_env_candidate)
        break

base_itdm_cfg, base_it_cfg, dm_cls, m_cls = MODULE_EXAMPLE_REGISTRY.get(REGISTRY_KEY)
# single circuit-tracer backend for all phases (BACKEND selects the replacement-model implementation)
base_it_cfg.circuit_tracer_cfg.backend = BACKEND
if TRANSCODER_SET:
    base_it_cfg.circuit_tracer_cfg.transcoder_set = TRANSCODER_SET
if BACKEND == "nnsight":
    adapter_ctx = (it.Adapter.core, it.Adapter.nnsight, it.Adapter.circuit_tracer)
else:
    # the TL circuit-tracer backend needs the transformer_lens adapter in the composition
    # (it provides the replacement-model init path; see docs/circuit_tracer_backend_support.md)
    adapter_ctx = (it.Adapter.core, it.Adapter.transformer_lens, it.Adapter.circuit_tracer)
session_cfg = ITSessionConfig(
    adapter_ctx=adapter_ctx,
    datamodule_cfg=base_itdm_cfg,
    module_cfg=base_it_cfg,
    datamodule_cls=dm_cls,
    module_cls=m_cls,
)
it_session = ITSession(session_cfg)
it.it_init(**it_session)
module = it_session.module
tokenizer = module.replacement_model.tokenizer
print(f"session ready: {type(module).__name__} ({MODEL_NAME} + circuit-tracer {BACKEND} backend)")

[INFO] interpretune.extensions.neuronpedia: Neuronpedia package available
INFO:interpretune.extensions.neuronpedia:Neuronpedia package available
[INFO] interpretune.utils.logging: Loading ReplacementModel with backend: nnsight
INFO:interpretune.utils.logging:Loading ReplacementModel with backend: nnsight
[INFO] interpretune.utils.logging: NNsight ReplacementModel initialized for Circuit Tracer
INFO:interpretune.utils.logging:NNsight ReplacementModel initialized for Circuit Tracer
[INFO] interpretune.utils.logging: Attempted to clean a key that was not present, continuing without cleaning that key: 'Gemma3TextConfig' object has no attribute 'quantization_config'
INFO:interpretune.utils.logging:Attempted to clean a key that was not present, continuing without cleaning that key: 'Gemma3TextConfig' object has no attribute 'quantization_config'
[INFO] interpretune.utils.logging: Attempted to clean a key that was not present, continuing without cleaning that key: 'Gemma3TextConfig' object has no attribute '_pre_quantization_dtype'
INFO:interpretune.utils.logging:Attempted to clean a key that was not present, continuing without cleaning that key: 'Gemma3TextConfig' object has no attribute '_pre_quantization_dtype'
[INFO] interpretune.utils.logging: Preparing data: InterpretunableDataModule
INFO:interpretune.utils.logging:Preparing data: InterpretunableDataModule
[INFO] interpretune.utils.logging: The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
INFO:interpretune.utils.logging:The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
[INFO] interpretune.utils.logging: The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
INFO:interpretune.utils.logging:The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
[INFO] interpretune.utils.logging: The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
INFO:interpretune.utils.logging:The following columns  don't have a corresponding argument in `NNSightReplacementModel.forward` and have been ignored: hypothesis, idx, label, premise, sequences. If hypothesis, idx, label, premise, sequences are not expected by `NNSightReplacementModel.forward`, you can safely ignore this message.
[INFO] interpretune.utils.logging: Setting up datamodule: InterpretunableDataModule
INFO:interpretune.utils.logging:Setting up datamodule: InterpretunableDataModule
[INFO] interpretune.utils.logging: Setting up model: InterpretunableModule
INFO:interpretune.utils.logging:Setting up model: InterpretunableModule
[INFO] interpretune.utils.logging: initializing optimizers and schedulers: InterpretunableModule
INFO:interpretune.utils.logging:initializing optimizers and schedulers: InterpretunableModule
[INFO] interpretune.utils.logging: Input gradient requirements handled by circuit tracer internally.
INFO:interpretune.utils.logging:Input gradient requirements handled by circuit tracer internally.

session ready: InterpretunableModule (gemma-3-1b-it + circuit-tracer nnsight backend)

2. Feature-mediated path: concept direction -> attribution -> sign-aware selection -> feature steering#

Runs the registered composite it.intervention_from_concept(...) (concept_direction -> compute_attribution_graph -> graph_node_influence -> extract_top_features -> feature_intervention_forward) with:

FeatureSelectionSpec(layer_slice=(FEATURE_SELECTION_MIN_LAYER, None), score_sign=FEATURE_SELECTION_SCORE_SIGN, score_source="signed_influence")
sign-aware, influence-normalized scaling (intervention_sign_aware_scale=True, intervention_max_influence_norm_scale=True, intervention_value_source="top_feature_activation_values", intervention_scale_factor=INTERVENTION_SCALE_FACTOR)

Expected outcome: post-intervention target gap exceeds the pre-intervention gap and the post-intervention argmax lands in the target-token variant set.

# @title 2: Feature-mediated steering { display-mode: "form" }
from interpretune.analysis.ops.base import AnalysisBatch
from interpretune.config import AnalysisCfg, init_analysis_cfgs

module.analysis_cfg = AnalysisCfg(target_op=it.compute_attribution_graph, ignore_manual=True, save_tokens=False)
init_analysis_cfgs(module, [module.analysis_cfg])

fruits = ["apple", "banana", "grape", "peach"]
colors = ["red", "blue", "green", "yellow"]
if CHAT_FORMAT_PROMPT:
    # instruction-tuned replacement models assert chat-formatted inputs
    from it_examples.example_prompt_configs import GemmaPromptConfig

    prompt = GemmaPromptConfig().apply_chat_template_fn(
        tokenizer, CONCEPT_PROMPT, tokenize=False, add_generation_prompt=True
    )
else:
    prompt = CONCEPT_PROMPT

# sign-aware, influence-normalized scaling (the validated s5_any lineage)
ct_cfg = module.it_cfg.circuit_tracer_cfg
ct_cfg.intervention_sign_aware_scale = True
ct_cfg.intervention_max_influence_norm_scale = True
ct_cfg.intervention_value_source = "top_feature_activation_values"

selection_spec = FeatureSelectionSpec(
    layer_slice=slice(FEATURE_SELECTION_MIN_LAYER, None),
    score_source="signed_influence",
    score_sign=FEATURE_SELECTION_SCORE_SIGN,
    rank_by_abs=True,
)
pipeline_results = it.intervention_from_concept(
    module,
    AnalysisBatch(
        concept_group_a=fruits,
        concept_group_b=colors,
        concept_label="Concept: Fruit - Color",
        concept_direction_mode="paired_rejection",
        prompts=[prompt],
    ),
    None,
    0,
    top_n=FEATURE_SELECTION_TOP_N,
    intervention_scale_factor=INTERVENTION_SCALE_FACTOR,
    feature_selection=selection_spec,
)
# one call renders the linked/signed features table + the consolidated target-gap table and
# returns everything later phases need (features, direction, target ids, gaps)
DASHBOARD_BASE_URL = LOCAL_WEBAPP_URL
steering_base_url = DASHBOARD_BASE_URL
steering = display_steering_results(
    pipeline_results,
    tokenizer,
    CONCEPT_TARGET_TOKENS,
    neuronpedia_model=NEURONPEDIA_MODEL_ID,
    neuronpedia_set=NEURONPEDIA_SOURCE_SET,
    neuronpedia_base_url=steering_base_url,
    min_layer=FEATURE_SELECTION_MIN_LAYER,
)
steered_features = steering.steered_features
pipeline_direction = steering.direction
target_a_id, target_b_id = steering.target_ids
fm_pre_gap, fm_post_gap = steering.pre_gap, steering.post_gap
assert fm_post_gap > fm_pre_gap, "feature-mediated steering should push the gap toward the target concept"

Phase 0: Precomputing activations and vectors
Precomputation completed in 1.07s
Found 11630 active features
Phase 1: Running forward pass
/mnt/cache/speediedan/.venvs/it_latest/lib/python3.13/site-packages/circuit_tracer/transcoder/single_layer_transcoder.py:139: UserWarning: Attempting to run cuBLAS, but there was no current CUDA context! Attempting to set the primary context... (Triggered internally at /__w/pytorch/pytorch/aten/src/ATen/cuda/CublasHandlePool.cpp:408.)
  return input_acts @ self.W_skip.T
Forward pass completed in 0.70s
Phase 2: Building input vectors
Using 1 custom attribution targets with total weight 0.0000
Will include 8192 of 11630 feature nodes
Input vectors built in 0.44s
Phase 3: Computing logit attributions
1 logit attribution(s) completed in 0.11s
Phase 4: Computing feature attributions
Feature influence computation:   0%|          | 0/8192 [00:00<?, ?it/s]
Feature influence computation:   3%|▎         | 256/8192 [00:00<00:07, 1132.24it/s]
Feature influence computation:   9%|▉         | 768/8192 [00:00<00:03, 2183.49it/s]
Feature influence computation:  16%|█▌        | 1280/8192 [00:00<00:02, 2458.82it/s]
Feature influence computation:  22%|██▏       | 1792/8192 [00:00<00:02, 2666.47it/s]
Feature influence computation:  28%|██▊       | 2304/8192 [00:00<00:02, 2820.03it/s]
Feature influence computation:  34%|███▍      | 2816/8192 [00:01<00:01, 2914.13it/s]
Feature influence computation:  41%|████      | 3328/8192 [00:01<00:01, 2927.18it/s]
Feature influence computation:  47%|████▋     | 3840/8192 [00:01<00:01, 2957.87it/s]
Feature influence computation:  53%|█████▎    | 4352/8192 [00:01<00:01, 2838.65it/s]
Feature influence computation:  59%|█████▉    | 4864/8192 [00:01<00:01, 2922.40it/s]
Feature influence computation:  66%|██████▌   | 5376/8192 [00:01<00:01, 2779.26it/s]
Feature influence computation:  72%|███████▏  | 5888/8192 [00:02<00:00, 2881.31it/s]
Feature influence computation:  78%|███████▊  | 6400/8192 [00:02<00:00, 2702.29it/s]
Feature influence computation:  84%|████████▍ | 6912/8192 [00:02<00:00, 2815.04it/s]
Feature influence computation:  91%|█████████ | 7424/8192 [00:02<00:00, 2671.61it/s]
Feature influence computation:  97%|█████████▋| 7936/8192 [00:02<00:00, 2788.91it/s]
Feature influence computation: 100%|██████████| 8192/8192 [00:02<00:00, 2748.10it/s]

Feature attributions completed in 2.98s
Attribution completed in 6.39s

Steered Features (signed influence)

#	Node	Sign	\|Score\|
1	(25, 27, 1316)	−	2.55e-08
2	(25, 27, 765)	−	1.74e-08
3	(16, 27, 155)	+	1.04e-08
4	(23, 27, 725)	−	6.98e-09
5	(25, 27, 662)	−	6.41e-09

Feature-mediated steering — target gap

Token	Pre prob	Post prob	Pre logit	Post logit	Δ
Fruit	4.31e-04	2.26e-06	41.5000	79.5000	+38.0000
Color	99.957%	5.82e-24	49.2500	39.0000	-10.2500
Gap (Fruit − Color)			-7.7500	+40.5000	+48.2500

3. Feature semantics: top-features table + local explanation coverage#

The steered features render as a table with the (layer, pos, feature) node tuple linked to its dashboard, the signed influence score (Sign / |Score| columns — the signed_influence selection can steer with negative-signed features, so the sign is colour-coded), and a best-effort Explanation column resolved from the local webapp’s feature API.

ensure_local_feature_explanations first reports local explanation coverage and, when GENERATE_MISSING_LOCAL_EXPLANATIONS=True, backfills missing explanations through the conforming explanation CLI (see docs/neuronpedia_dashboard_pipeline.md, “Explanation CLI configuration”).

# @title 3: Top-features table + local explanation coverage { display-mode: "form" }
# top_feature_ids are (layer, position, feature) tuples; dashboards/explanations are per
# (layer, feature), so collapse positions while preserving selection order
steered_layer_feature_pairs = list(dict.fromkeys((f[0], f[-1]) for f in steered_features))

feature_refs = feature_tuples_to_feature_refs(
    model_id=NEURONPEDIA_MODEL_ID,
    source_set=NEURONPEDIA_SOURCE_SET,
    feature_tuples=steered_layer_feature_pairs,
    base_url=DASHBOARD_BASE_URL,
)
coverage = ensure_local_feature_explanations(
    feature_refs,
    generate_missing=GENERATE_MISSING_LOCAL_EXPLANATIONS,
    regenerate_existing=REGENERATE_LOCAL_EXPLANATIONS,
    local_db_url=LOCAL_DB_URL,
)
covered = len(coverage.statuses) - len(coverage.missing_feature_refs)
print(f"local explanation coverage: {covered}/{len(coverage.statuses)}")
if coverage.generated_artifacts:
    print(f"explanations generated this run: {len(coverage.generated_artifacts)}")
if coverage.generation_failures:
    print("generation failures:", [f.error for f in coverage.generation_failures])

# Best-effort explanation text from the local webapp's feature API; unmapped features simply
# render an empty Explanation cell
feature_explanations = resolve_feature_explanations(
    model_id=NEURONPEDIA_MODEL_ID,
    source_set=NEURONPEDIA_SOURCE_SET,
    feature_tuples=steered_layer_feature_pairs,
    base_url=DASHBOARD_BASE_URL,
)

display_top_features_comparison(
    {"Steered Features (signed influence)": steered_features},
    {"Steered Features (signed influence)": pipeline_results.top_feature_scores.tolist()},
    neuronpedia_model=NEURONPEDIA_MODEL_ID,
    neuronpedia_set=NEURONPEDIA_SOURCE_SET,
    neuronpedia_base_url=DASHBOARD_BASE_URL,
    show_score_sign=True,
    feature_explanations=feature_explanations,
)

local explanation coverage: 5/5
explanations generated this run: 5

Steered Features (signed influence)

#	Node	Sign	\|Score\|	Explanation
1	(25, 27, 1316)	−	2.55e-08	fruits
2	(25, 27, 765)	−	1.74e-08	code
3	(16, 27, 155)	+	1.04e-08	fruit
4	(23, 27, 725)	−	6.98e-09	proper names
5	(25, 27, 662)	−	6.41e-09	syntax delimiters

4. Direct-hook path: concept direction -> hook-tensor steering#

Recomputes the embed-basis concept direction for the same concept pair (a consistency check against the step 2 pipeline’s direction — cosine should be ~1.0 since both derive from the same embedding-basis paired_rejection) and applies it.model_fwd_intervention(...) at EMBED_INTERVENTION_HOOK in EMBED_INTERVENTION_MODE mode, comparing pre/post_intervention_logits and the target-token gap against the feature-mediated result. The same direction steered through selected transcoder features vs added directly at the hook point produces different effect sizes — the feature-mediated path is typically stronger per unit scale.

# @title 4: Direct-hook steering { display-mode: "form" }
# Recompute the embed-basis concept direction for the same concept pair (no store rows -> embed basis)
direct_result = it.concept_direction(
    module,
    AnalysisBatch(
        concept_group_a=fruits,
        concept_group_b=colors,
        concept_label="Concept: Fruit - Color (direct)",
        concept_direction_mode="paired_rejection",
    ),
    None,
    0,
)
direct_direction = direct_result.concept_direction.detach()
cosine = torch.nn.functional.cosine_similarity(
    pipeline_direction, direct_direction.float().cpu().reshape(-1), dim=0
).item()
print(f"pipeline-vs-direct direction cosine: {cosine:+.4f} (~1.0 expected — same embed-basis construction)")

# Direct hook-tensor intervention at the canonical hook point
module.analysis_cfg = AnalysisCfg(target_op=it.model_fwd_intervention, ignore_manual=True, save_tokens=False)
init_analysis_cfgs(module, [module.analysis_cfg])

# chat-rendered prompts already carry their special tokens; plain completion prompts need them added
enc = tokenizer(prompt, return_tensors="pt", padding=False, add_special_tokens=not CHAT_FORMAT_PROMPT)
device = next(module.model.parameters()).device
if BACKEND == "transformerlens":
    # HookedTransformer.forward takes `input`, not the HF-style `input_ids`/`attention_mask` keys
    batch = {"input": enc["input_ids"].to(device)}
else:
    batch = {k: (v.to(device) if isinstance(v, torch.Tensor) else v) for k, v in dict(enc).items()}

# legacy HookedTransformer models (the CT TransformerLens backend) expose no `unembed.hook_in`;
# their pre-unembed equivalent is `ln_final.hook_normalized` (alias-map expansion tracked in
# interpretune#223)
intervention_hook = EMBED_INTERVENTION_HOOK
if BACKEND == "transformerlens" and EMBED_INTERVENTION_HOOK == "unembed.hook_in":
    intervention_hook = "ln_final.hook_normalized"

direct_batch = AnalysisBatch(
    prompts=[prompt],
    concept_direction=direct_direction,
    logit_target_ids=torch.tensor([target_a_id], dtype=torch.long),
    concept_group_a_token_ids=[target_a_id],
    concept_group_b_token_ids=[target_b_id],
    concept_cache_key=intervention_hook,
    intervention_hook_pattern=intervention_hook,
    intervention_mode=EMBED_INTERVENTION_MODE,
    direction_scale_factor=INTERVENTION_SCALE_FACTOR,
)
direct_out = it.model_fwd_intervention(module, direct_batch, batch, 0)

direct_pre_gap, direct_post_gap = display_target_gap(
    direct_out.pre_intervention_logits.float().cpu().reshape(-1),
    direct_out.post_intervention_logits.float().cpu().reshape(-1),
    (CONCEPT_TARGET_TOKENS[0], target_a_id),
    (CONCEPT_TARGET_TOKENS[1], target_b_id),
    title="Direct-hook steering — target gap",
)
print(
    f"feature-mediated delta {fm_post_gap - fm_pre_gap:+.3f} vs direct-hook delta "
    f"{direct_post_gap - direct_pre_gap:+.3f}"
)
assert direct_post_gap > direct_pre_gap, "direct-hook steering should push the gap toward the target concept"

pipeline-vs-direct direction cosine: +1.0000 (~1.0 expected — same embed-basis construction)
feature-mediated delta +48.250 vs direct-hook delta +5.250

Direct-hook steering — target gap

Token	Pre prob	Post prob	Pre logit	Post logit	Δ
Fruit	4.31e-04	7.586%	41.5000	48.7500	+7.2500
Color	99.957%	92.414%	49.2500	51.2500	+2.0000
Gap (Fruit − Color)			-7.7500	-2.5000	+5.2500

5. User-curated feature steering (OPTIONAL — off by default)#

This step is skipped unless you supply your own feature ids. Curated ids index the transcoder’s feature space, so they are tied to a specific (model, source set) pair — valid across regenerations of that same pair, but not transferable to another model, width or source set. What does not carry over is the reasoning for a pick: the explanation that made a feature look like “a clear fruit feature” describes dashboard evidence, and that evidence shifts when the prompt corpus changes. Curated picks therefore only travel reliably alongside a shared dashboard artifact, which is not available yet (priority item on the IT roadmap). Until then the lists in the parameters cell start empty and this step reports that it is skipping.

To run it, populate CURATED_POSITIVE_FEATURES / CURATED_NEGATIVE_FEATURES with features you located in your own local webapp. What it then demonstrates: instead of letting the attribution graph select features, it steers with features found by manual dashboard/inference search — positively firing on the target concept, or writing against the competing concept. Sign control is direct — FeatureSelectionSpec.activation_overrides pins each curated feature’s intervention activation (+magnitude to amplify, −magnitude to suppress), with sign-aware/influence-normalized scaling disabled so the overrides apply as-is. Comparing that target gap against the attribution-selected step 2 result probes how well graph-derived feature sets capture the concept-relevant circuitry a human finds by direct inspection.

This becomes a sensible default once generated dashboards are downloadable from the Hub and everyone shares one substrate with stable ids.

# @title 5: User-curated feature steering { display-mode: "form" }
if not (CURATED_POSITIVE_FEATURES or CURATED_NEGATIVE_FEATURES):
    print(
        "Skipping curated-feature steering: no curated features configured (disabled by default).\n"
        "Curated ids are tied to a specific (model, source set) pair, so this step needs ids you "
        "located in YOUR local webapp -- see the parameters cell."
    )
elif NEURONPEDIA_SOURCE_SET != CURATED_FEATURES_SOURCE_SET:
    print(
        f"Skipping curated-feature steering: the curated features target {CURATED_FEATURES_SOURCE_SET!r} "
        f"but this run uses {NEURONPEDIA_SOURCE_SET!r}"
    )
else:
    module.analysis_cfg = AnalysisCfg(target_op=it.compute_attribution_graph, ignore_manual=True, save_tokens=False)
    init_analysis_cfgs(module, [module.analysis_cfg])

    # direct sign control via activation overrides (positive = amplify, negative = suppress);
    # disable sign-aware/influence-normalized scaling so the overrides are applied as-is
    ct_cfg.intervention_sign_aware_scale = False
    ct_cfg.intervention_max_influence_norm_scale = False
    curated_pairs = [tuple(p) for p in (*CURATED_POSITIVE_FEATURES, *CURATED_NEGATIVE_FEATURES)]
    curated_overrides = {tuple(p): float(CURATED_OVERRIDE_MAGNITUDE) for p in CURATED_POSITIVE_FEATURES}
    curated_overrides.update({tuple(p): -float(CURATED_OVERRIDE_MAGNITUDE) for p in CURATED_NEGATIVE_FEATURES})
    curated_spec = FeatureSelectionSpec(
        layer_feature_pairs=curated_pairs,
        activation_overrides=curated_overrides,
    )
    curated_results = it.intervention_from_concept(
        module,
        AnalysisBatch(
            concept_group_a=fruits,
            concept_group_b=colors,
            concept_label="Concept: Fruit - Color (curated)",
            concept_direction_mode="paired_rejection",
            prompts=[prompt],
        ),
        None,
        0,
        top_n=len(curated_pairs),
        intervention_scale_factor=INTERVENTION_SCALE_FACTOR,
        feature_selection=curated_spec,
    )
    curated_feature_pairs = [(int(f[0]), int(f[-1])) for f in curated_results.top_feature_ids]
    curated_explanations = resolve_feature_explanations(
        model_id=NEURONPEDIA_MODEL_ID,
        source_set=NEURONPEDIA_SOURCE_SET,
        feature_tuples=curated_feature_pairs,
        base_url=DASHBOARD_BASE_URL,
    )
    curated = display_steering_results(
        curated_results,
        tokenizer,
        CONCEPT_TARGET_TOKENS,
        neuronpedia_model=NEURONPEDIA_MODEL_ID,
        neuronpedia_set=NEURONPEDIA_SOURCE_SET,
        neuronpedia_base_url=DASHBOARD_BASE_URL,
        feature_explanations=curated_explanations,
        features_label="Curated Features (manual search)",
        gap_title="Curated-feature steering — target gap",
    )
    print(
        f"attribution-selected delta {fm_post_gap - fm_pre_gap:+.3f} vs "
        f"curated-feature delta {curated.post_gap - curated.pre_gap:+.3f}"
    )
    # restore the sign-aware defaults for any later cells / re-runs
    ct_cfg.intervention_sign_aware_scale = True
    ct_cfg.intervention_max_influence_norm_scale = True

Skipping curated-feature steering: no curated features configured (disabled by default).
Curated ids are tied to a specific (model, source set) pair, so this step needs ids you located in YOUR local webapp -- see the parameters cell.

6. Input/output decoupling analysis (graph hydration + UMAP)#

Attribution-selected steering features are chosen for their output effect (decoder projection onto the target logit difference), while dashboard explanations describe their input behavior (the contexts they fire on) — the two can decouple sharply. A recurring special case is the suppressor-motif exemplar: a feature that fires on concept contexts yet projects against the concept token (redundancy suppression under next-token training) — causally ideal for sign-aware steering, semantically confusing on its dashboard. This step demonstrates those mechanics directly, and in doing so demos the framework’s graph hydration capability (analysis_backend.hydrate_graph_from_batch) for detailed post-hoc analysis of a persisted attribution result:

hydrate the step-2 attribution graph; highlight the prompt’s concept-token positions;
compute each analyzed feature’s input profile (activation mass at concept positions) and output profile (signed decoder projection onto the unit target-token unembed difference) via the shared feature_io_profiles helper;
render the decoupling table (signature column flags decoupled and suppressor-motif rows);
project decoder vectors to 2D (UMAP, PCA fallback) with hover details per analyzed feature. Axis tick numbers are intentionally hidden of course since UMAP coordinates are non-metric (arbitrary rotation/scale; only local neighborhood structure is meaningful).

Expect roughly 2 of the 5 attribution picks to be input-aligned (“fruits”, “ripe fruit”), including a suppressor-motif exemplar (“fruits” firing on concept while projecting against Fruit), with the rest decoupled (zero input concept share). If you enabled the optional step 5, its curated features also join this analysis — typically with high input shares but the smallest |output projections|, which is the measured explanation for why curated steering tends to be weaker.

# @title 6: Decoupling analysis via hydrated graph + UMAP { display-mode: "form" }
import numpy as np

from interpretune.analysis.backends import require_analysis_backend
from it_examples.utils.example_helpers import concept_token_positions, feature_io_profiles
from it_examples.utils.nb_ui_utils import (
    display_concept_positions,
    display_feature_decoupling_table,
    plot_decoder_projection_map,
)

analysis_backend = require_analysis_backend(module)
graph = analysis_backend.hydrate_graph_from_batch(pipeline_results)

# concept-token positions derived from the demo's own concept groups + probe/target tokens
concept_words = {w.lower() for w in (*fruits, *colors, *CONCEPT_TARGET_TOKENS, "orange")}
prompt_token_ids = [int(t) for t in graph.input_tokens]
concept_positions = concept_token_positions(tokenizer, prompt_token_ids, sorted(concept_words))
display_concept_positions(tokenizer, prompt_token_ids, concept_positions)

embed_weight = analysis_backend.get_embedding_weight(module).detach().float().cpu()
target_diff = embed_weight[target_a_id] - embed_weight[target_b_id]
target_diff = target_diff / target_diff.norm()
target_label = f"{CONCEPT_TARGET_TOKENS[0]}-{CONCEPT_TARGET_TOKENS[1]}"

analyzed_pairs = list(dict.fromkeys((int(f[0]), int(f[-1])) for f in steered_features))
# Merge in the step-5 curated features so the decoupling table can show why curated steering is
# weaker (high input share, smallest |output projection|). The guard tests the name actually bound
# by step 5: it previously tested "curated", which is never defined, so the curated features were
# silently never analyzed and the discussion below described rows that were not in the table.
if "curated_feature_pairs" in globals():
    analyzed_pairs += [pair for pair in curated_feature_pairs if pair not in analyzed_pairs]
all_explanations = dict(feature_explanations)
if "curated_explanations" in globals():
    all_explanations.update(curated_explanations)

transcoder_set = getattr(module.replacement_model.transcoders, "_module", module.replacement_model.transcoders)
profiles = feature_io_profiles(graph, analyzed_pairs, target_diff, transcoder_set, concept_positions)
display_feature_decoupling_table(profiles, all_explanations, target_label=target_label)

# decoder vectors for the interactive projection map (analyzed + random active-feature background)
rng = np.random.default_rng(17)
active_rows = graph.active_features.cpu()
background_pool = sorted({(int(r[0]), int(r[2])) for r in active_rows} - set(analyzed_pairs))
background_idx = rng.choice(len(background_pool), size=min(300, len(background_pool)), replace=False)
background_pairs = [background_pool[i] for i in background_idx]


def _decoder_rows(pairs):
    return torch.stack(
        [transcoder_set._get_decoder_vectors(lyr, torch.tensor([ft]))[0].detach().float().cpu() for lyr, ft in pairs]
    )


plot_decoder_projection_map(
    profiles,
    _decoder_rows(analyzed_pairs),
    _decoder_rows(background_pairs),
    feature_explanations=all_explanations,
    target_label=target_label,
    title="Steering-feature decoder map",
)

Prompt tokens — concept positions highlighted

<bos><start_of_turn>user Is orange a color or a fruit? Answer with one word: Color or Fruit. orange -><end_of_turn> <start_of_turn>model

concept-token positions: [5, 7, 10, 17, 19, 21]

Feature input/output decoupling

Feature	Input concept share	Act mass	Output proj (Fruit-Color)	Signature	Explanation
L25/1316	0.455	1918.25	-0.2458	suppressor-motif	fruits
L16/155	0.841	853.25	+0.1884		fruit
L25/662	0.000	242.00	-0.0349	decoupled	syntax delimiters
L25/765	0.000	506.00	+0.0348	decoupled	code
L23/725	0.000	669.00	+0.0167		proper names

decoupled = large |output proj| with ~zero input concept share; suppressor-motif = fires on concept contexts yet projects against the concept token.

/mnt/cache/speediedan/.venvs/it_latest/lib/python3.13/site-packages/umap/umap_.py:1952: UserWarning: n_jobs value 1 overridden to 1 by setting random_state. Use no seed for parallelism.
  warn(

Summary#

Feature-mediated and direct-hook steering paths (plus the optional user-curated path) on one proven example, one backend per run — all driven by the same embed-basis concept direction.
Semantic grounding via a local Neuronpedia dev webapp, including optional local explanation generation — the full local pipeline: dashboards -> explanations -> selection -> steering.
Input/output decoupling mechanics demonstrated via graph hydration + decoder-space UMAP, doubling as a demo of persisted-graph post-hoc analysis.
The zero-setup public-dashboard version: ct_concept_steering_demo.ipynb.
Deeper coverage of the underlying op pipeline (per-op invocation, native + hub composition): see ct_analysis_backend_demo.ipynb. Capability map: tests/nb_experiments/intervention_capabilities_overview.md.