Example Hub and Local Operation Collections

This notebook demonstrates the complete workflow for uploading and downloading operations collections using the HubAnalysisOpManager and loading local operations via IT_ANALYSIS_OP_PATHS. The workflow includes:

1. Setting up local op collection path via IT_ANALYSIS_OP_PATHS

2. Copying the current hub_op_collection folder to /tmp/

3. Uploading operations to HuggingFace Hub as a private repository

4. Downloading the uploaded collection to the default cache

5. Re-importing interpretune to verify both hub and local operations are available

6. Testing the loaded operations

7. Cleaning up downloaded operations and re-importing

8. Verifying only local operations remain available

9. Final cleanup of the local operations collection

**Note**: This example requires HuggingFace Hub authentication and will create a private repository.

Setup and Imports

import os
from pathlib import Path

# Import interpretune components
import interpretune
from interpretune.analysis.ops.hub_manager import HubAnalysisOpManager
from interpretune.analysis import IT_ANALYSIS_CACHE, IT_ANALYSIS_HUB_CACHE, IT_ANALYSIS_OP_PATHS, IT_MODULES_CACHE
from interpretune.base.components.cli import IT_BASE

# Import utility functions for op collection demo setup/cleanup
import it_examples.notebooks.publish.example_op_collections.op_collection_demo_utils as op_demo_utils

example_op_collections_dir = Path(IT_BASE / "notebooks" / "publish" / "example_op_collections")
example_hub_op_collection_dir = Path(example_op_collections_dir / "hub_op_collection")
example_local_op_collection_dir = Path(example_op_collections_dir / "local_op_collection")

# Print environment summary
op_demo_utils.print_env_summary(
    interpretune.version,
    IT_ANALYSIS_CACHE,
    IT_MODULES_CACHE,
    IT_ANALYSIS_HUB_CACHE,
    IT_ANALYSIS_OP_PATHS,
    example_hub_op_collection_dir,
    example_local_op_collection_dir,
)

Interpretune version: <function version at 0x7f93c6a1ab60>
Current analysis cache location: /mnt/cache_extended/speediedan/.cache/huggingface/interpretune
Current modules cache location: /mnt/cache_extended/speediedan/.cache/huggingface/interpretune/modules
Current hub cache location: /mnt/cache_extended/speediedan/.cache/huggingface/hub/interpretune_ops
Current IT analysis op paths: []
This notebook's example hub op collection directory: /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/example_op_collections/hub_op_collection
This notebook's example local op collection directory: /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/example_op_collections/local_op_collection

Step 1: Stage example local op collections to a temporary directory

Copy the local_op_collection to /tmp/ and add it to IT_ANALYSIS_OP_PATHS so local operations are loaded.

# Define source and destination paths for local ops
source_local_op_collection = example_local_op_collection_dir
tmp_local_op_collection = Path("/tmp/local_op_collection")

# copy our local op collection to `tmp_local_op_collection` and that path to our IT_ANALYSIS_OP_PATHS env var
original_op_paths_env, new_op_paths = op_demo_utils.setup_local_op_collection(
    source_local_op_collection=source_local_op_collection, tmp_local_op_collection=tmp_local_op_collection
)

Source local op_collection: /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/example_op_collections/local_op_collection
Destination: /tmp/local_op_collection
✓ Successfully copied local op_collection to /tmp/local_op_collection
Original IT_ANALYSIS_OP_PATHS environment variable: ''
✓ Set IT_ANALYSIS_OP_PATHS environment variable to: '/tmp/local_op_collection'
✓ Also added /tmp/local_op_collection to imported IT_ANALYSIS_OP_PATHS list

Updated IT_ANALYSIS_OP_PATHS list: ['/tmp/local_op_collection']
Current IT_ANALYSIS_OP_PATHS env var: '/tmp/local_op_collection'

Contents of copied local op_collection:
  - local_op_definitions.py
  - local_op_collection.yaml

Step 2: Copy hub op_collection to /tmp/

Copy the hub op_collection folder to /tmp/ for upload to the hub.

# Define source and destination paths for hub ops
source_op_collection = example_hub_op_collection_dir
tmp_op_collection = Path("/tmp/hub_op_collection")

# Stage a hub op collection using utility function
op_demo_utils.setup_hub_op_collection(source_op_collection=source_op_collection, tmp_op_collection=tmp_op_collection)

Source hub op_collection: /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/example_op_collections/hub_op_collection
Destination: /tmp/hub_op_collection
✓ Successfully copied hub op_collection to /tmp/hub_op_collection

Contents of copied hub op_collection:
  - hub_op_collection.yaml
  - hub_op_definitions.py

Step 3: Upload operations to HuggingFace Hub

Upload the hub op_collection to HuggingFace Hub as a private repository named “trivial_op_repo”.

from huggingface_hub import whoami

# Resolve HF token: try dedicated key first, then standard HF_TOKEN, then interactive login.
hub_token = os.environ.get("HF_TRIVIAL_OP_REPO_EXAMPLE_AUTH_KEY") or os.environ.get("HF_TOKEN")
if not hub_token:
    from huggingface_hub import notebook_login

    notebook_login()
    hub_token = os.environ.get("HF_TOKEN")  # notebook_login sets HF_TOKEN

current_user = whoami(token=hub_token)["name"]

# Initialize the hub manager with the resolved token
hub_manager = HubAnalysisOpManager(token=hub_token)

# Repository configuration
repo_name = "trivial_op_repo"
private = True

print("Uploading op_collection to HuggingFace Hub...")
print(f"Current HF user: {current_user}")
print(f"Repository: {repo_name}")
print(f"Private: {private}")
print(f"Source folder: {tmp_op_collection}")

# Ensure the user is authenticated
repo_id = f"{current_user}/{repo_name}"
try:
    # Upload operations to hub
    # 1. This will create the specified repository if it doesn't exist
    # 2. If the repo exists, it will clean existing operations and upload the new ones in a single commit
    #    - If no files have changed, it will skip the commit and leave the repository unchanged

    upload_result = hub_manager.upload_ops(
        local_dir=tmp_op_collection, repo_id=repo_id, private=private, clean_existing=True
    )

    print(f"\u2713 Successfully uploaded operations (if necessary) to {repo_name}")
    print(f"Upload result (new or latest op repo commit sha): {upload_result}")

except Exception as e:
    print(f"\u274c Error uploading operations: {e}")
    raise

Uploading op_collection to HuggingFace Hub...
Current HF user: speediedan
Repository: trivial_op_repo
Private: True
Source folder: /tmp/hub_op_collection
✓ Successfully uploaded operations (if necessary) to trivial_op_repo
Upload result (new or latest op repo commit sha): 66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934

No files have been modified since last commit. Skipping to prevent empty commit.

Step 4: Download operations to default hub cache

Download the uploaded operations collection to the default IT_ANALYSIS_HUB_CACHE location.

print(f"Downloading operations from {repo_id} to default cache...")
print(f"Cache location: {IT_ANALYSIS_HUB_CACHE}")

# Initialize download_result to None so we can safely check it in cleanup step
download_result = None

try:
    # Download operations from hub to default cache
    download_result = hub_manager.download_ops(repo_id=repo_id)  # no cache_dir default IT_ANALYSIS_HUB_CACHE is used

    print("✓ Successfully downloaded operations to cache")
    print(f"Download result: {download_result}")

    # Check what was downloaded
    cache_path = Path(IT_ANALYSIS_HUB_CACHE)
    if cache_path.exists():
        print("\nContents of hub cache:")
        for item in cache_path.rglob("*"):
            if item.is_file():
                rel_path = item.relative_to(cache_path)
                print(f"  - {rel_path}")

except Exception as e:
    print(f"❌ Error downloading operations: {e}")
    raise

Downloading operations from speediedan/trivial_op_repo to default cache...
Cache location: /mnt/cache_extended/speediedan/.cache/huggingface/hub/interpretune_ops
✓ Successfully downloaded operations to cache
Download result: HubOpCollection(repo_id='speediedan/trivial_op_repo', username='speediedan', repo_name='trivial_op_repo', local_path=PosixPath('/mnt/cache_extended/speediedan/.cache/huggingface/hub/interpretune_ops/models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934'), revision='main')

Contents of hub cache:
  - models--speediedan--trivial_op_repo/refs/main
  - models--speediedan--trivial_op_repo/blobs/d3a1af9f8d6af798898c9c3fb510335840b8892f
  - models--speediedan--trivial_op_repo/blobs/a6344aac8c09253b3b630fb776ae94478aa0275b
  - models--speediedan--trivial_op_repo/blobs/7b95401dc46245ac339fc25059d4a56d90b4cde5
  - models--speediedan--trivial_op_repo/blobs/0c9a339e3604787df9cdb0e59cff48e3481b1370
  - models--speediedan--trivial_op_repo/blobs/a88c06240bbf197b5030361d8601ac94d41273cc
  - models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934/hub_op_definitions.py
  - models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934/.gitattributes
  - models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934/README.md
  - models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934/hub_op_collection.yaml
  - models--speediedan--trivial_op_repo/snapshots/66f9e0cc2c6d9b792bbc5020c5ae20cffb6b8934/__pycache__/hub_op_definitions.cpython-310.pyc
  - .locks/models--speediedan--trivial_op_repo/c68fb334d3f79897fb401c712fe5842b426a090e.lock
  - .locks/models--speediedan--trivial_op_repo/730c649faee83eba503022680e3ba25d1dac1869.lock
  - .locks/models--speediedan--trivial_op_repo/3600a180683366a0958f528646f1807432a29bd7.lock
  - .locks/models--speediedan--trivial_op_repo/68e45efe2690af6547f00c85fdf2b0c3246475a6.lock
  - .locks/models--speediedan--trivial_op_repo/c0df4fea470bb73756130fe1647b61a4cfa103d7.lock
  - .locks/models--speediedan--trivial_op_repo/ac03bc96098cfcb07a9b4ad699c8e61d7abc20b6.lock
  - .locks/models--speediedan--trivial_op_repo/2f07dedfaf6b0931ffce732008d50ef8d618e1f5.lock
  - .locks/models--speediedan--trivial_op_repo/8404f2975e97cf974ba01d02ddc40583e9b1277b.lock
  - .locks/models--speediedan--trivial_op_repo/9f57e62abbbd4cf3cb0659d478526cc1aa1cddc9.lock
  - .locks/models--speediedan--trivial_op_repo/43194e6247d09dba781653b7094e7f059130f540.lock
  - .locks/models--speediedan--trivial_op_repo/814e7a81ede3c895fef998c58d08dac3e1b49b46.lock
  - .locks/models--speediedan--trivial_op_repo/82e04d7a174cd438780c43b7cdd2c2a9de36968f.lock
  - .locks/models--speediedan--trivial_op_repo/81bc1c8d053116a09cbe67552a7774d0e023080b.lock

Step 5: Re-import interpretune and verify hub and local operations

Re-import interpretune to pick up both hub and local operations and verify they are available.

print("Re-importing interpretune to pick up hub and local operations...")
# Remove interpretune modules from sys.modules to force reimport
op_demo_utils.purge_it_modules_from_sys()

# ruff: noqa: E402

# Re-import interpretune
import interpretune as it
from interpretune import DISPATCHER

print("✓ Interpretune re-imported")

# Get operation definitions and generate summary
operation_definitions = DISPATCHER.registered_ops
op_demo_utils.generate_op_summary(operation_definitions)

# Show operations by type
canonical_ops, alias_map, hub_ops, local_ops, composed_ops, builtin_ops = op_demo_utils.categorize_operations(
    operation_definitions
)

# Demo lazy operation instantiation
op_demo_utils.demo_lazy_op_instantiation(it, hub_ops, local_ops)

Re-importing interpretune to pick up hub and local operations...
✓ Interpretune re-imported

📊 Operation Summary:
  Total registered names: 45
  Unique operations: 32
  Hub operations: 1
  Local operations: 10
  Composed operations: 8
  Built-in operations: 13

🌐 Hub operations found:
  - speediedan.trivial_op_repo.trivial_test_op (accessible as: speediedan.trivial_op_repo.trivial_test_op, trivial_test_op)

🏠 Local operations found:
  - extract_concept_latent_state (accessible as: extract_concept_latent_state, concept_latent_state_from_cache) - Extract per-example latent rows from the configured cache key
  - extract_concept_latent_examples (accessible as: extract_concept_latent_examples, concept_latent_examples) - Filter and annotate latent rows for concept-direction aggregation
  - concept_direction (accessible as: concept_direction, semantic_direction) - Aggregate latent concept examples into a normalized concept direction vector
  - compute_attribution_graph (accessible as: compute_attribution_graph, ct_graph) - Generate an attribution graph with circuit-tracer
  - extract_top_features (accessible as: extract_top_features, ct_top_features) - Extract top-N influential features from an attribution graph
  - graph_prune (accessible as: graph_prune, ct_graph_prune) - Prune a circuit-tracer attribution graph
  - graph_node_influence (accessible as: graph_node_influence, ct_node_influence) - Compute node influence scores for an attribution graph
  - feature_intervention_forward (accessible as: feature_intervention_forward, ct_feature_intervention) - Run feature interventions and return pre/post intervention outputs
  - model_fwd_intervention (accessible as: model_fwd_intervention, direction_intervention, direct_concept_direction_intervention) - Apply generalized hook-point interventions and return pre/post intervention logits
  - trivial_local_test_op (accessible as: trivial_local_test_op) - Local test op that transforms a simple orig_labels tensor to a preds tensor

🔧 Testing operation instantiation:
labels_to_ids op reference type: <class 'interpretune.analysis.ops.base.OpWrapper'>
get_answer_indices op reference type: <class 'interpretune.analysis.ops.base.OpWrapper'>
trivial_test_op op reference type: <class 'interpretune.analysis.ops.base.OpWrapper'>
Get non-direct access attribute of labels_to_ids (description of the underlying AnalysisOp): Convert label strings to tensor IDs
Type of labels_to_ids now: <class 'interpretune.analysis.ops.base.AnalysisOp'>
Type of get_answer_indices is still: <class 'interpretune.analysis.ops.base.OpWrapper'> and its instantiated status is False
Non-direct access attribute of get_answer_indices (name of the underlying AnalysisOp): get_answer_indices
Type of get_answer_indices is now: <class 'interpretune.analysis.ops.base.AnalysisOp'>
Type of trivial_test_op is: <class 'interpretune.analysis.ops.base.OpWrapper'> and its instantiated status is False
Non-direct access attribute of trivial_test_op (name of the underlying AnalysisOp): trivial_test_op
Type of trivial_test_op is now: <class 'interpretune.analysis.ops.base.AnalysisOp'> as it has been successfully instantiated
Non-direct access attribute of trivial_local_test_op (name of the underlying AnalysisOp): trivial_local_test_op
Type of trivial_local_test_op is now: <class 'interpretune.analysis.ops.base.AnalysisOp'> as it has been successfully instantiated
speediedan.trivial_op_repo.trivial_test_op op reference type: <class 'interpretune.analysis.ops.base.OpWrapper'>
extract_concept_latent_state op reference type: <class 'interpretune.analysis.ops.base.OpWrapper'>

Overwriting format type 'interpretune' (ITAnalysisFormatter -> ITAnalysisFormatter)
Overwriting format type alias 'itanalysis' (interpretune -> interpretune)
Overwriting format type alias 'it' (interpretune -> interpretune)
Overwriting format type alias 'interpretune' (interpretune -> interpretune)

Step 6: Test executing the loaded operations

Test executing simple hub and local operations both individually executed and as part of a composite operation to ensure loading and execution works correctly.

print("\n🧪 Testing loaded operations with demo data...")

# Import required components
from interpretune import trivial_test_op, trivial_local_test_op, composite_trivial_test_op

NUM_BATCHES = 2  # Number of test batches to generate
VERBOSE_OP_OUTPUTS = False  # Set to True to log operation outputs

# Test the operations
print(f"\n📋 Testing operation pipeline parity of composite vs individual component ops (over {NUM_BATCHES} batches):")
individual_op_output_batches = []
composite_op_output_batches = []

for batch_name, individual_test_batch, composite_test_batch in op_demo_utils.generate_test_batches(NUM_BATCHES):
    print("\nComposite op execution...")
    if VERBOSE_OP_OUTPUTS:
        print(f"\n--- {batch_name} ---")
        print(f"Input batch: {individual_test_batch}")
    composite_output_batch = composite_trivial_test_op(analysis_batch=composite_test_batch)
    op_demo_utils.maybe_print_output(f"Composite op output batch: {composite_output_batch}", VERBOSE_OP_OUTPUTS)
    composite_op_output_batches.append(composite_output_batch)

    print("\nRe-running with individual component ops...")
    local_batch_output = trivial_local_test_op(analysis_batch=individual_test_batch)
    op_demo_utils.maybe_print_output(f"Local op batch output: {local_batch_output}", VERBOSE_OP_OUTPUTS)
    individual_output_batch = trivial_test_op(analysis_batch=local_batch_output)
    op_demo_utils.maybe_print_output(f"Hub output batch: {individual_output_batch}", VERBOSE_OP_OUTPUTS)
    individual_op_output_batches.append(individual_output_batch)

# Compare outputs using utility function
all_match = op_demo_utils.compare_operation_outputs(individual_op_output_batches, composite_op_output_batches)

🧪 Testing loaded operations with demo data...

📋 Testing operation pipeline parity of composite vs individual component ops (over 2 batches):

Composite op execution...
Local op: Converted orig_labels tensor([4, 3, 1, 2]) to preds tensor([5, 4, 2, 3])
Hub op: Calculated pred_sum: 14

Re-running with individual component ops...
Local op: Converted orig_labels tensor([4, 3, 1, 2]) to preds tensor([5, 4, 2, 3])
Hub op: Calculated pred_sum: 14

Composite op execution...
Local op: Converted orig_labels tensor([3, 0, 1, 4]) to preds tensor([4, 1, 2, 5])
Hub op: Calculated pred_sum: 12

Re-running with individual component ops...
Local op: Converted orig_labels tensor([3, 0, 1, 4]) to preds tensor([4, 1, 2, 5])
Hub op: Calculated pred_sum: 12

🔍 Validating that composite and individual component op outputs are identical...
  ✓ Batch 1: Outputs match.
  ✓ Batch 2: Outputs match.

🎉 All batches match: individual and composite operation outputs are identical!

Step 7: Clean up hub operations and re-import

Delete the downloaded hub operations folder and re-import interpretune to verify only local operations remain.

print("Cleaning up downloaded hub operations...")

# Remove only the specific repository we downloaded, not the entire hub cache
op_demo_utils.cleanup_hub_repository(download_result)

# Re-import interpretune again
print("\nRe-importing interpretune after cleanup...")

# Capture stdout and stderr during import to check for the expected warning
stdout_output, stderr_output, DISPATCHER = op_demo_utils.reimport_interpretune_with_capture()

op_demo_utils.inspect_err_for_composite_op_warning(stderr_output)

print("\n ✓ Interpretune re-imported after cleanup")

Cleaning up downloaded hub operations...
✓ Removed specific hub repository cache: /mnt/cache_extended/speediedan/.cache/huggingface/hub/interpretune_ops/models--speediedan--trivial_op_repo

Re-importing interpretune after cleanup...
/home/speediedan/repos/interpretune/src/interpretune/analysis/ops/compiler/schema_compiler.py:306: Failed to compile operation 'composite_trivial_test_op' with composition ['trivial_local_test_op', 'trivial_test_op']: Operation trivial_test_op not found

Note the above "Failed to compile operation 'composite_trivial_test_op'" error on re-import of interpretune after our cleanup.
This is expected: we have removed our hub op definitions (trivial_test_op), but not our local op definitions (trivial_local_test_op, composite_trivial_test_op).
As a result, the locally defined composite operation 'composite_trivial_test_op' could not be constructed since it depended on the now-missing hub op.
All other available operations (local and built-in) should still be present as we will see.

 ✓ Interpretune re-imported after cleanup

Overwriting format type 'interpretune' (ITAnalysisFormatter -> ITAnalysisFormatter)
Overwriting format type alias 'itanalysis' (interpretune -> interpretune)
Overwriting format type alias 'it' (interpretune -> interpretune)
Overwriting format type alias 'interpretune' (interpretune -> interpretune)

Step 8: Verify only local operations remain

Verify that only the local and built-in operations are available after hub cleanup.

print("Verifying operations after cleanup...")

# Get operation definitions after cleanup and verify cleanup status
operation_definitions_after = DISPATCHER.registered_ops
op_demo_utils.verify_cleanup_status(operation_definitions_after)

Verifying operations after cleanup...

📊 Operation Summary After Cleanup:
  Total registered names: 42
  Unique operations: 30
  Hub operations: 0
  Local operations: 10
  Composed operations: 7
  Built-in operations: 13

✅ Success: No hub operations found - cleanup successful!

🏠 Local operations still available:
  - extract_concept_latent_state (accessible as: extract_concept_latent_state, concept_latent_state_from_cache) - Extract per-example latent rows from the configured cache key
  - extract_concept_latent_examples (accessible as: extract_concept_latent_examples, concept_latent_examples) - Filter and annotate latent rows for concept-direction aggregation
  - concept_direction (accessible as: concept_direction, semantic_direction) - Aggregate latent concept examples into a normalized concept direction vector
  - compute_attribution_graph (accessible as: compute_attribution_graph, ct_graph) - Generate an attribution graph with circuit-tracer
  - extract_top_features (accessible as: extract_top_features, ct_top_features) - Extract top-N influential features from an attribution graph
  - graph_prune (accessible as: graph_prune, ct_graph_prune) - Prune a circuit-tracer attribution graph
  - graph_node_influence (accessible as: graph_node_influence, ct_node_influence) - Compute node influence scores for an attribution graph
  - feature_intervention_forward (accessible as: feature_intervention_forward, ct_feature_intervention) - Run feature interventions and return pre/post intervention outputs
  - model_fwd_intervention (accessible as: model_fwd_intervention, direction_intervention, direct_concept_direction_intervention) - Apply generalized hook-point interventions and return pre/post intervention logits
  - trivial_local_test_op (accessible as: trivial_local_test_op) - Local test op that transforms a simple orig_labels tensor to a preds tensor

📋 Detailed breakdown:

  Built-in operations (13):
    - ablation_attribution (accessible as: ablation_attribution)
    - get_alive_latents (accessible as: get_alive_latents)
    - get_answer_indices (accessible as: get_answer_indices)
    - gradient_attribution (accessible as: gradient_attribution)
    - labels_to_ids (accessible as: labels_to_ids)
    - logit_diffs (accessible as: logit_diffs)
    - logit_diffs_cache (accessible as: logit_diffs_cache)
    - model_ablation (accessible as: model_ablation)
    - model_fwd (accessible as: model_fwd, model_forward)
    - model_fwd_w_cache (accessible as: model_fwd_w_cache)
    - model_fwd_w_cache_latent_models (accessible as: model_fwd_w_cache_latent_models)
    - model_gradient (accessible as: model_gradient)
    - sae_correct_acts (accessible as: sae_correct_acts)

  Composed operations (7):
    - attribution_from_concept (accessible as: attribution_from_concept)
    - intervention_from_concept (accessible as: intervention_from_concept)
    - intervention_from_features (accessible as: intervention_from_features)
    - logit_diffs_attr_ablation (accessible as: logit_diffs_attr_ablation, logit_diffs_ablation)
    - logit_diffs_attr_grad (accessible as: logit_diffs_attr_grad)
    - logit_diffs_base (accessible as: logit_diffs_base)
    - logit_diffs_sae (accessible as: logit_diffs_sae)

Cleanup temporary files

Clean up the temporary files created during this example.

# Clean up using utility function
op_demo_utils.cleanup_op_collections(
    tmp_op_collection=tmp_op_collection,
    tmp_local_op_collection=tmp_local_op_collection,
    original_op_paths_env=original_op_paths_env,
)

print("\n🎉 Hub and Local operations workflow example completed successfully!")
print("\nSummary of what was demonstrated:")
print("1. ✓ Setup local op collection path via IT_ANALYSIS_OP_PATHS environment variable")
print("2. ✓ Copied hub op_collection to /tmp/ with overwrite warning")
print("3. ✓ Uploaded operations to HuggingFace Hub as private repo")
print("4. ✓ Downloaded operations to default hub cache")
print("5. ✓ Re-imported interpretune and verified both hub and local operations")
print("6. ✓ Tested operation instantiation and execution with demo data")
print("7. ✓ Cleaned up hub operations and re-imported")
print("8. ✓ Verified only local and built-in operations remain available")
print("9. ✓ Restored original IT_ANALYSIS_OP_PATHS environment variable")

Cleaning up temporary files...
✓ Removed temporary hub op_collection: /tmp/hub_op_collection
✓ Removed temporary local op_collection: /tmp/local_op_collection
✓ Unset IT_ANALYSIS_OP_PATHS environment variable
✓ Removed /tmp/local_op_collection from imported IT_ANALYSIS_OP_PATHS list

Final IT_ANALYSIS_OP_PATHS list: []
Final IT_ANALYSIS_OP_PATHS env var: ''

🎉 Hub and Local operations workflow example completed successfully!

Summary of what was demonstrated:
1. ✓ Setup local op collection path via IT_ANALYSIS_OP_PATHS environment variable
2. ✓ Copied hub op_collection to /tmp/ with overwrite warning
3. ✓ Uploaded operations to HuggingFace Hub as private repo
4. ✓ Downloaded operations to default hub cache
5. ✓ Re-imported interpretune and verified both hub and local operations
6. ✓ Tested operation instantiation and execution with demo data
7. ✓ Cleaned up hub operations and re-imported
8. ✓ Verified only local and built-in operations remain available
9. ✓ Restored original IT_ANALYSIS_OP_PATHS environment variable

Step 9: Final verification after environment cleanup

Re-import interpretune one final time to verify that local operations are no longer available after unsetting IT_ANALYSIS_OP_PATHS.

print("Final verification: Re-importing interpretune after environment cleanup...")

# Remove interpretune modules from sys.modules to force reimport
op_demo_utils.purge_it_modules_from_sys()

# Re-import interpretune one final time
import interpretune
from interpretune import DISPATCHER

print("✓ Interpretune re-imported after environment cleanup")

# Get operation definitions after complete cleanup and generate final summary
operation_definitions_final = DISPATCHER.registered_ops
canonical_ops_final, alias_map_final, hub_ops_final, local_ops_final, composed_ops_final, builtin_ops = (
    op_demo_utils.categorize_operations(operation_definitions_final)
)

print("\n📊 Final Operation Summary (after complete cleanup):")
print(f"  Total registered names: {len(operation_definitions_final)}")
print(f"  Unique operations: {len(canonical_ops_final)}")
print(f"  Hub operations: {len(hub_ops_final)}")
print(f"  Local operations: {len(local_ops_final)}")
print(f"  Composed operations: {len(composed_ops_final)}")
print(f"  Built-in operations: {len(builtin_ops)}")

# Verify complete cleanup
if len(hub_ops_final) == 0 and len(local_ops_final) == 0:
    print("\n🎯 Perfect! Complete cleanup successful - only built-in and composed operations remain!")
elif len(hub_ops_final) == 0:
    print(f"\n⚠️ Hub operations cleaned up, but {len(local_ops_final)} local operations still present:")
    for op_name, op_def in local_ops_final.items():
        aliases = alias_map_final.get(op_name, [])
        all_names = [op_name] + aliases
        print(f"    - {op_name} (accessible as: {', '.join(all_names)})")
elif len(local_ops_final) == 0:
    print(f"\n⚠️ Local operations cleaned up, but {len(hub_ops_final)} hub operations still present:")
    for op_name, op_def in hub_ops_final.items():
        aliases = alias_map_final.get(op_name, [])
        all_names = [op_name] + aliases
        print(f"    - {op_name} (accessible as: {', '.join(all_names)})")
else:
    print(f"\n❌ Cleanup incomplete: {len(hub_ops_final)} hub ops and {len(local_ops_final)} local ops still present")

print("\nEnvironment verification:")
print(f"  Current IT_ANALYSIS_OP_PATHS env var: '{os.environ.get('IT_ANALYSIS_OP_PATHS', 'Not set')}'")

Final verification: Re-importing interpretune after environment cleanup...
✓ Interpretune re-imported after environment cleanup

📊 Final Operation Summary (after complete cleanup):
  Total registered names: 41
  Unique operations: 29
  Hub operations: 0
  Local operations: 9
  Composed operations: 7
  Built-in operations: 13

⚠️ Hub operations cleaned up, but 9 local operations still present:
    - extract_concept_latent_state (accessible as: extract_concept_latent_state, concept_latent_state_from_cache)
    - extract_concept_latent_examples (accessible as: extract_concept_latent_examples, concept_latent_examples)
    - concept_direction (accessible as: concept_direction, semantic_direction)
    - compute_attribution_graph (accessible as: compute_attribution_graph, ct_graph)
    - extract_top_features (accessible as: extract_top_features, ct_top_features)
    - graph_prune (accessible as: graph_prune, ct_graph_prune)
    - graph_node_influence (accessible as: graph_node_influence, ct_node_influence)
    - feature_intervention_forward (accessible as: feature_intervention_forward, ct_feature_intervention)
    - model_fwd_intervention (accessible as: model_fwd_intervention, direction_intervention, direct_concept_direction_intervention)

Environment verification:
  Current IT_ANALYSIS_OP_PATHS env var: 'Not set'

Overwriting format type 'interpretune' (ITAnalysisFormatter -> ITAnalysisFormatter)
Overwriting format type alias 'itanalysis' (interpretune -> interpretune)
Overwriting format type alias 'it' (interpretune -> interpretune)
Overwriting format type alias 'interpretune' (interpretune -> interpretune)

Appendix: All Registered Analysis Ops

The dispatcher also includes built-in native analysis ops (e.g., circuit-tracer attribution and intervention ops). Here is the full set of registered operations:

from interpretune.analysis.ops.dispatcher import DISPATCHER

print(f"Total registered ops: {len(DISPATCHER.registered_ops)}")
for name, op in sorted(DISPATCHER.registered_ops.items()):
    desc = getattr(op, "description", "")
    print(f"  {name}: {desc}")

Total registered ops: 41
  ablation_attribution: Compute attribution values from ablation
  attribution_from_concept: Compiled composition: concept_direction.compute_attribution_graph.graph_node_influence.extract_top_features
  compute_attribution_graph: Generate an attribution graph with circuit-tracer
  concept_direction: Aggregate latent concept examples into a normalized concept direction vector
  concept_latent_examples: Filter and annotate latent rows for concept-direction aggregation
  concept_latent_state_from_cache: Extract per-example latent rows from the configured cache key
  ct_feature_intervention: Run feature interventions and return pre/post intervention outputs
  ct_graph: Generate an attribution graph with circuit-tracer
  ct_graph_prune: Prune a circuit-tracer attribution graph
  ct_node_influence: Compute node influence scores for an attribution graph
  ct_top_features: Extract top-N influential features from an attribution graph
  direct_concept_direction_intervention: Apply generalized hook-point interventions and return pre/post intervention logits
  direction_intervention: Apply generalized hook-point interventions and return pre/post intervention logits
  extract_concept_latent_examples: Filter and annotate latent rows for concept-direction aggregation
  extract_concept_latent_state: Extract per-example latent rows from the configured cache key
  extract_top_features: Extract top-N influential features from an attribution graph
  feature_intervention_forward: Run feature interventions and return pre/post intervention outputs
  get_alive_latents: Extract alive latents from cache
  get_answer_indices: Extract answer indices from batch
  gradient_attribution: Compute attribution values from gradients
  graph_node_influence: Compute node influence scores for an attribution graph
  graph_prune: Prune a circuit-tracer attribution graph
  intervention_from_concept: Compiled composition: concept_direction.compute_attribution_graph.graph_node_influence.extract_top_features.feature_intervention_forward
  intervention_from_features: Compiled composition: feature_intervention_forward
  labels_to_ids: Convert label strings to tensor IDs
  logit_diffs: Clean forward pass for computing logit differences
  logit_diffs_ablation: Compiled composition: labels_to_ids.model_fwd_w_cache_latent_models.logit_diffs_cache.model_ablation.ablation_attribution
  logit_diffs_attr_ablation: Compiled composition: labels_to_ids.model_fwd_w_cache_latent_models.logit_diffs_cache.model_ablation.ablation_attribution
  logit_diffs_attr_grad: Compiled composition: labels_to_ids.model_gradient.gradient_attribution
  logit_diffs_base: Compiled composition: labels_to_ids.model_fwd.logit_diffs
  logit_diffs_cache: Clean forward pass for computing logit differences including cache activations (composition only)
  logit_diffs_sae: Compiled composition: labels_to_ids.model_fwd_w_cache_latent_models.logit_diffs_cache.sae_correct_acts
  model_ablation: Model ablation analysis
  model_forward: Basic model forward pass
  model_fwd: Basic model forward pass
  model_fwd_intervention: Apply generalized hook-point interventions and return pre/post intervention logits
  model_fwd_w_cache: Model forward pass with activation caching (no latent model hooks)
  model_fwd_w_cache_latent_models: Model forward pass with activation caching and latent model (SAE) hooks
  model_gradient: Model gradient-based attribution
  sae_correct_acts: Compute correct activations from SAE cache
  semantic_direction: Aggregate latent concept examples into a normalized concept direction vector