{ "cells": [ { "cell_type": "markdown", "id": "bdd785de", "metadata": { "id": "colab-badge", "papermill": { "duration": 0.017123, "end_time": "2026-07-30T19:06:55.271047+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.253924+00:00", "status": "completed" }, "tags": [] }, "source": [ "\n", " \"Open\n", "" ] }, { "cell_type": "markdown", "id": "b401f97a", "metadata": { "papermill": { "duration": 0.015513, "end_time": "2026-07-30T19:06:55.339975+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.324462+00:00", "status": "completed" }, "tags": [] }, "source": [ "# Attribution Analysis with Interpretune, Circuit Tracer, and Neuronpedia\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/speediedan/interpretune/blob/main/src/it_examples/notebooks/publish/attribution_analysis/attribution_analysis.ipynb)" ] }, { "cell_type": "markdown", "id": "564a573f", "metadata": { "papermill": { "duration": 0.010632, "end_time": "2026-07-30T19:06:55.364228+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.353596+00:00", "status": "completed" }, "tags": [] }, "source": [ "## Analyzing Attribution Graphs with Interpretune, Circuit Tracer and Neuronpedia" ] }, { "cell_type": "markdown", "id": "2f9e565f", "metadata": { "papermill": { "duration": 0.009703, "end_time": "2026-07-30T19:06:55.385806+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.376103+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Intro" ] }, { "cell_type": "markdown", "id": "20897736", "metadata": { "papermill": { "duration": 0.009359, "end_time": "2026-07-30T19:06:55.405009+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.395650+00:00", "status": "completed" }, "tags": [] }, "source": [ "[Interpretune](https://github.com/speediedan/interpretune) is a flexible framework for exploring, analyzing and tuning \n", "llm world models. In this tutorial, we'll walk through a simple example of using Interpretune to pursue interpretability \n", "research with Circuit Tracer. As we'll see, Interpretune handles the required execution context composition, allowing us to \n", "use the same code in a variety of contexts, depending upon the level of abstraction required.\n", "\n", "As a long-time PyTorch and PyTorch Lightning contributor, I've found the PyTorch Lightning framework is the right level \n", "of abstraction for a large variety of ML research contexts, but some contexts benefit from using core PyTorch directly. \n", "Additionally, some users may prefer to use the core PyTorch framework directly for a wide variety of reasons including \n", "maximizing portability. As will be demonstrated here, Interpretune maximizes flexibility and portability by adhering to \n", "a well-defined protocol that allows auto-composition of our research module with the adapters required for execution in \n", "a wide variety of contexts. In this example, we'll be executing the same module with core PyTorch and PyTorch Lightning, \n", "demonstrating the use of `Circuit Tracer` w/ Interpretune for circuit discovery and interpretability research.\n", "\n", "> Note - **this is a WIP**, but this is the core idea. If you have any feedback, please let me know!" ] }, { "cell_type": "markdown", "id": "1834cbde", "metadata": { "papermill": { "duration": 0.009072, "end_time": "2026-07-30T19:06:55.423214+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.414142+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Setup" ] }, { "cell_type": "markdown", "id": "9532c0ed", "metadata": { "papermill": { "duration": 0.009053, "end_time": "2026-07-30T19:06:55.441688+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.432635+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Notebook Parameters\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "6f6e19af", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:06:55.461948Z", "iopub.status.busy": "2026-07-30T19:06:55.461684Z", "iopub.status.idle": "2026-07-30T19:06:55.465775Z", "shell.execute_reply": "2026-07-30T19:06:55.464930Z" }, "papermill": { "duration": 0.015995, "end_time": "2026-07-30T19:06:55.466779+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.450784+00:00", "status": "completed" }, "tags": [ "parameters" ] }, "outputs": [], "source": [ "# Parameters - These will be injected by papermill during parameterized test runs\n", "use_baseline_salient_logits = True # logits computation mode: True->salient logits, False->specific logits\n", "use_baseline_transcoder_arch = True # transcoder architecture: True->SingleLayerTranscoder, False->CrossLayerTranscoder\n", "enable_analysis_injection = True # Toggle analysis injection for analyzing attribution flow\n", "core_log_dir = None # Directory to save analysis logs (if None, a temp directory will be created)\n", "analysis_config_path = \"analysis_injection_config.yaml\" # Base YAML config merged with notebook overrides" ] }, { "cell_type": "markdown", "id": "15b68a20", "metadata": { "papermill": { "duration": 0.00911, "end_time": "2026-07-30T19:06:55.485307+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.476197+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Interpretune Import" ] }, { "cell_type": "code", "execution_count": 3, "id": "cb7efca9", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:06:55.504993Z", "iopub.status.busy": "2026-07-30T19:06:55.504796Z", "iopub.status.idle": "2026-07-30T19:06:59.012923Z", "shell.execute_reply": "2026-07-30T19:06:59.011940Z" }, "papermill": { "duration": 3.519268, "end_time": "2026-07-30T19:06:59.013879+00:00", "exception": false, "start_time": "2026-07-30T19:06:55.494611+00:00", "status": "completed" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "import interpretune as it # registered analysis ops will be available as it. when analysis is imported" ] }, { "cell_type": "code", "execution_count": 4, "id": "468a459c", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:06:59.050461Z", "iopub.status.busy": "2026-07-30T19:06:59.050178Z", "iopub.status.idle": "2026-07-30T19:07:07.453305Z", "shell.execute_reply": "2026-07-30T19:07:07.452385Z" }, "papermill": { "duration": 8.423279, "end_time": "2026-07-30T19:07:07.454447+00:00", "exception": false, "start_time": "2026-07-30T19:06:59.031168+00:00", "status": "completed" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "# Import circuit tracer and required modules\n", "from transformer_lens import ActivationCache # noqa: F401\n", "from pprint import pformat\n", "from datetime import datetime\n", "\n", "from it_examples import _ACTIVE_PATCHES # noqa: F401 # TODO: add note about this unless patched in SL before release\n", "from it_examples.example_module_registry import MODULE_EXAMPLE_REGISTRY # TODO: move to hub once implemented\n", "from interpretune import ITSessionConfig, ITSession\n", "from interpretune.base.call import it_init" ] }, { "cell_type": "markdown", "id": "7cbf92f3", "metadata": { "papermill": { "duration": 0.015485, "end_time": "2026-07-30T19:07:07.485893+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.470408+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Enable Analysis Injection" ] }, { "cell_type": "markdown", "id": "bd810d33", "metadata": { "papermill": { "duration": 0.015542, "end_time": "2026-07-30T19:07:07.517084+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.501542+00:00", "status": "completed" }, "tags": [] }, "source": [ "Interpretune includes a lightweight in-notebook analysis injection utility that can temporarily instrument adapter packages (for this notebook, `circuit_tracer`) with small, configurable hooks named analysis points.\n", "\n", "This tooling is intentionally experimental and most useful for short-lived, exploratory work: it lets you quickly inspect intermediate values, add ad-hoc diagnostics and exposition (as in this notebook), and prototype ideas without changing the adapter's source tree.\n", "\n", "By default, we set `enable_analysis_injection = True` in our parameters cell above to enable the runtime patching. The orchestrator validates hooks, applies them to the target package path, and registers the analysis point functions you provide (whether sourced from files or notebook cells).\n", "\n", "
\n", "More on Analysis Injection\n", "\n", "Key characteristics:\n", "\n", "- Ephemeral: patches are applied in-process for the notebook session only and do not persist to source files.\n", "- Configuration-driven: hooks are registered via YAML configs plus analysis function mappings that can live in this notebook, external files, or any mix of the two.\n", "- Flexible composition: you can load the base YAML and analysis hook functions from external files, then override or extend them inline in subsequent notebook cells.\n", "- Notebook-friendly output: each analysis point's collected data is available via various helper methods on the `orchestrator` (e.g. `get_analysis_data`, `get_output`), allowing later cells to display the captured values without re-running the instrumentation (as is done in the attribution analysis exposition of this notebook). Analysis events are also written to the configured log (e.g. `/tmp/attribution_flow_analysis_.log`) and optionally to the console when enabled.\n", "\n", "The `analysis_injection` utility is a powerful exploratory and expository tool, but for ongoing or production workflows you should upstream a proper hook interface in the relevant adapter package.\n", "
\n", "\n", "
\n", "Guidance on Analysis Injection Usage\n", "\n", "If you or other users find a recurring need to access intermediate analysis variables, the more robust and maintainable solution is to add a small hook interface to the adapter package itself.\n", "\n", "Why prefer adapter-level hooks over regex-based patching:\n", "- Stability: explicit APIs are far less brittle than runtime regex patching.\n", "- Maintainability: adapter maintainers can review, test, and document hook APIs so they remain supported across releases.\n", "- Performance and safety: built-in hooks can be designed to avoid unintended side-effects or excessive overhead.\n", "\n", "If an adapter would benefit from exposing intermediate analysis variables, Interpretune recommends opening an issue or a pull request against the adapter repository proposing a small, well-scoped hook API (describe the use case, example call signatures, and what guarantees callers should expect). This is the recommended path for any capability you expect to use repeatedly or across teams.\n", "\n", "When to use which approach:\n", "- Use `analysis_injection` for short experiments, expository notebooks like this one, ad-hoc debugging, or rapid iteration on ideas.\n", "- Propose adapter hooks (issue/PR) when you want a repeatable, supported, and long-term inspection facility.\n", "\n", "For more details and guidance on safe usage patterns, see the Interpretune project documentation and repo: https://github.com/speediedan/interpretune\n", "\n", "
\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "8e4bc45e", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:07.544943Z", "iopub.status.busy": "2026-07-30T19:07:07.544767Z", "iopub.status.idle": "2026-07-30T19:07:07.572835Z", "shell.execute_reply": "2026-07-30T19:07:07.571967Z" }, "papermill": { "duration": 0.041472, "end_time": "2026-07-30T19:07:07.574000+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.532528+00:00", "status": "completed" }, "tags": [ "hide-cell" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using base analysis injection config at /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/attribution_analysis/analysis_injection_config.yaml\n" ] } ], "source": [ "from pathlib import Path\n", "from typing import Any, Dict\n", "\n", "import torch\n", "\n", "from it_examples.utils.analysis_injection.analysis_hook_patcher import HOOK_REGISTRY\n", "from it_examples.utils.analysis_injection.orchestrator import analysis_log_point, sample_tensor_output\n", "from it_examples.utils.example_helpers import collect_shapes, VarAnnotate\n", "from it_examples.utils.raw_graph_analysis import plot_ridgeline_convergence\n", "from IPython.display import display\n", "\n", "NOTEBOOK_DIR = Path.cwd()\n", "raw_config_path = Path(analysis_config_path)\n", "base_config_path = raw_config_path if raw_config_path.is_absolute() else NOTEBOOK_DIR / raw_config_path\n", "if not base_config_path.exists():\n", " raise FileNotFoundError(f\"Expected config file at {base_config_path}\")\n", "base_config_path = base_config_path.resolve()\n", "\n", "print(f\"Using base analysis injection config at {base_config_path}\")" ] }, { "cell_type": "markdown", "id": "1956195b", "metadata": { "papermill": { "duration": 0.015482, "end_time": "2026-07-30T19:07:07.605404+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.589922+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Define/Customize Analysis Points" ] }, { "cell_type": "code", "execution_count": 6, "id": "d1d8e0e6", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:07.633597Z", "iopub.status.busy": "2026-07-30T19:07:07.633385Z", "iopub.status.idle": "2026-07-30T19:07:07.639330Z", "shell.execute_reply": "2026-07-30T19:07:07.638636Z" }, "papermill": { "duration": 0.018742, "end_time": "2026-07-30T19:07:07.640086+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.621344+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Registered notebook override analysis point: ap_setup_attribution_end\n" ] } ], "source": [ "# Notebook override: add the attribution setup analysis point locally while still\n", "# reusing the module-defined defaults. This demonstrates how to extend the\n", "# external analysis_points module from within the notebook.\n", "def ap_setup_attribution_end(local_vars: Dict[str, Any]) -> None:\n", " data: Dict[str, Any] = {}\n", " # Collect shapes from ctx attributes with descriptions\n", " collect_shapes(\n", " data,\n", " local_vars,\n", " [\n", " VarAnnotate(\"ctx.activation_matrix\", \"n_layers, n_pos, d_transcoder\"),\n", " VarAnnotate(\"ctx.decoder_vecs\", \"num_active_features, d_model\"),\n", " VarAnnotate(\"ctx.encoder_vecs\", \"num_active_features, d_model\"),\n", " VarAnnotate(\"ctx.logits\", \"n_examples (usually 1), n_pos, d_vocab\"),\n", " VarAnnotate(\"ctx.token_vectors\", \"n_pos, d_model\"),\n", " VarAnnotate(\"ctx.error_vectors\", \"n_layers, n_pos, d_model\"),\n", " VarAnnotate(\"ctx.encoder_to_decoder_map\", \"num_active_features\"),\n", " VarAnnotate(\n", " \"ctx.decoder_locations\",\n", " \"dims activation_matrix, num_active_features (sparse indices into activation_matrix)\",\n", " ),\n", " ],\n", " )\n", " ctx = local_vars.get(\"ctx\")\n", " # Add non-shape attributes\n", " data[\"n_layers\"] = getattr(ctx, \"n_layers\", None)\n", " data[\"_row_size\"] = VarAnnotate(\n", " \"ctx._row_size\", ctx._row_size, \"total_active_feats + error_nodes ((n_layers + 1) * n_pos) # + logits later\"\n", " )\n", " analysis_log_point(\"AttributionContext summary after precomputing activations and vectors\", data)\n", "\n", "\n", "NOTEBOOK_ANALYSIS_FUNCTIONS = {\"ap_setup_attribution_end\": ap_setup_attribution_end}\n", "print(\"Registered notebook override analysis point: ap_setup_attribution_end\")" ] }, { "cell_type": "markdown", "id": "d79eb200", "metadata": { "papermill": { "duration": 0.00949, "end_time": "2026-07-30T19:07:07.659215+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.649725+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Define/Customize Analysis Injection Config" ] }, { "cell_type": "code", "execution_count": 7, "id": "35be5969", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:07.679273Z", "iopub.status.busy": "2026-07-30T19:07:07.679072Z", "iopub.status.idle": "2026-07-30T19:07:07.684220Z", "shell.execute_reply": "2026-07-30T19:07:07.683556Z" }, "papermill": { "duration": 0.016449, "end_time": "2026-07-30T19:07:07.684987+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.668538+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Configured log directory: /tmp\n", "Prepared analysis injection config overrides:\n", "\n", "settings:\n", " log_dir: /tmp\n", "\n", "file_hooks:\n", " ap_setup_attribution_end:\n", " file_path: attribution/attribute_transformerlens.py\n", " enable: true\n", " regex_pattern: '^\\s*ctx\\s*=\\s*model\\.setup_attribution'\n", " insert_after: true\n", " description: \"AttributionContext summary at end of phase 0 (added from notebook)\"\n", "\n" ] } ], "source": [ "# We can customize the base config (`analysis_injection_config.yaml` in this case) with overrides to tweak settings and\n", "# manipulate hook definitions declaratively.\n", "import tempfile\n", "from pathlib import Path\n", "\n", "# Ensure the log directory is reachable for this session.\n", "target_log_dir = Path(core_log_dir).expanduser() if core_log_dir else Path(tempfile.gettempdir())\n", "target_log_dir.mkdir(parents=True, exist_ok=True)\n", "\n", "# Demonstrate notebook-based overrides: add (or replace) the\n", "# `ap_setup_attribution_end` hook definition directly via config overrides.\n", "analysis_config_overrides = f\"\"\"\n", "settings:\n", " log_dir: {target_log_dir.as_posix()}\n", "\n", "file_hooks:\n", " ap_setup_attribution_end:\n", " file_path: attribution/attribute_transformerlens.py\n", " enable: true\n", " regex_pattern: '^\\\\s*ctx\\\\s*=\\\\s*model\\\\.setup_attribution'\n", " insert_after: true\n", " description: \"AttributionContext summary at end of phase 0 (added from notebook)\"\n", "\"\"\"\n", "\n", "print(\"Configured log directory:\", target_log_dir.as_posix())\n", "print(\"Prepared analysis injection config overrides:\")\n", "print(analysis_config_overrides)" ] }, { "cell_type": "markdown", "id": "0e02ca41", "metadata": { "papermill": { "duration": 0.009353, "end_time": "2026-07-30T19:07:07.704241+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.694888+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Instantiate Our Analysis Injector" ] }, { "cell_type": "code", "execution_count": 8, "id": "7afebed4", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:07.724689Z", "iopub.status.busy": "2026-07-30T19:07:07.724490Z", "iopub.status.idle": "2026-07-30T19:07:08.077033Z", "shell.execute_reply": "2026-07-30T19:07:08.076270Z" }, "papermill": { "duration": 0.36448, "end_time": "2026-07-30T19:07:08.078161+00:00", "exception": false, "start_time": "2026-07-30T19:07:07.713681+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Setting up analysis injection using base config: /home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/attribution_analysis/analysis_injection_config.yaml\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/speediedan/repos/interpretune/src/it_examples/utils/analysis_injection/analysis_hook_patcher.py:369: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead\n", "/usr/lib/python3.13/inspect.py:304: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead\n", " return isinstance(object, type)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📝 Analysis output will be logged to: /tmp/attribution_flow_analysis_20260730_120707.log\n", "Analysis injection ready. Active patched modules:\n", " - circuit_tracer.attribution.attribute_transformerlens\n", " - circuit_tracer.replacement_model.replacement_model_transformerlens\n", " - circuit_tracer.graph\n", "Hook registry status:\n", " Enabled: True\n", " Registered hooks: 14\n", "\n", "You can inspect collected analysis data via orchestrator.get_analysis_data().\n" ] } ], "source": [ "# Analysis Injection — Setup\n", "# The orchestrator loads the configured analysis point module automatically. We only pass the\n", "# additional notebook-defined hooks via `analysis_functions`.\n", "# Centralized setup: optional env_path (default None), uses the base config from NOTEBOOK_DIR.\n", "from it_examples.utils.example_helpers import required_os_env\n", "\n", "# Optional: the user can set `env_path` to a specific .env file path before running this cell.\n", "# If left as None, load_dotenv() will be called without a path so it can auto-discover the .env file.\n", "env_path: str | None = None # set to '/full/path/to/.env' to override\n", "\n", "if enable_analysis_injection:\n", " # Load environment variables. If env_path is provided, use it; otherwise let load_dotenv auto-discover.\n", " os_env_reqs = None\n", " assert required_os_env(env_path=env_path, env_reqs=os_env_reqs)\n", "\n", " # Import orchestrator from the analysis_injection package\n", " from it_examples.utils.analysis_injection import orchestrator\n", "\n", " print(\"Setting up analysis injection using base config:\", base_config_path)\n", "\n", " # Create the orchestrator which performs the patching. The simplified API handles validation,\n", " # module loading, and registration automatically.\n", " analysis_injector = orchestrator.setup_analysis_injection(\n", " config_path=base_config_path,\n", " target_package=\"circuit_tracer\",\n", " config_overrides=analysis_config_overrides,\n", " analysis_functions=NOTEBOOK_ANALYSIS_FUNCTIONS,\n", " )\n", "\n", " print(\"Analysis injection ready. Active patched modules:\")\n", " if analysis_injector.patched_modules:\n", " for module_name in analysis_injector.patched_modules.keys():\n", " print(f\" - {module_name}\")\n", " else:\n", " print(\" (No modules patched; check configuration)\")\n", "\n", " print(\"Hook registry status:\")\n", " print(f\" Enabled: {orchestrator.HOOK_REGISTRY._enabled}\")\n", " print(f\" Registered hooks: {len(orchestrator.HOOK_REGISTRY._hooks)}\")\n", "\n", " print(\"\\nYou can inspect collected analysis data via orchestrator.get_analysis_data().\")\n", "else:\n", " print(\"Analysis injection disabled via parameters; skipping setup.\")" ] }, { "cell_type": "markdown", "id": "1b1e0db3", "metadata": { "papermill": { "duration": 0.016377, "end_time": "2026-07-30T19:07:08.111178+00:00", "exception": false, "start_time": "2026-07-30T19:07:08.094801+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Configure our IT Session\n" ] }, { "cell_type": "markdown", "id": "0d374d64", "metadata": { "papermill": { "duration": 0.01637, "end_time": "2026-07-30T19:07:08.143919+00:00", "exception": false, "start_time": "2026-07-30T19:07:08.127549+00:00", "status": "completed" }, "tags": [] }, "source": [ "Here we define or customize our session configuration, which includes:\n", "1. Experiment/task module and datamodule (in this case, 'rte' for the RTE task) \n", " * We can customize any module, datamodule, or adapter-specific configuration options we want to use. In this case, we set target `circuit_tracer_cfg` that we want to use for our analysis. We also could customize generation parameters, tokenization, the pretrained/config-based model we want to use (in this case, GPT2) etc.\n", "2. The adapter context we want to use. In this case, `core` PyTorch (vs e.g. Lightning) and `circuit_tracer` (vs e.g. `transformer_lens` or `sae_lens`). \n", "\n", "When an `ITSession` is created, the selected adapter context will trigger composition of the relevant adapters with our experiment/task module and datamodule. The intention of this abstraction is to enable the same experiment/task logic to be used unchanged across a broad variety of PyTorch framework and analytical package contexts.\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "361ce753", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:08.171941Z", "iopub.status.busy": "2026-07-30T19:07:08.171676Z", "iopub.status.idle": "2026-07-30T19:07:08.461200Z", "shell.execute_reply": "2026-07-30T19:07:08.460391Z" }, "papermill": { "duration": 0.301988, "end_time": "2026-07-30T19:07:08.461982+00:00", "exception": false, "start_time": "2026-07-30T19:07:08.159994+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "use_baseline_salient_logits=True: cleared analysis_target_tokens and target_token_ids -> using default compute_salient_logits path\n", "✓ Analysis injection enabled: disabled circuit_tracer verbose logging to avoid duplicate logs\n", "use_baseline_transcoder_arch=True: set transcoder_set='gemma' -> set transcoder_set to HF URL -> using the SingleLayerTranscoder checkpoint\n", "CircuitTracerConfig(backend='transformerlens',\n", " model_name=None,\n", " transcoder_set='gemma',\n", " dtype=torch.bfloat16,\n", " max_n_logits=10,\n", " desired_logit_prob=0.95,\n", " batch_size=256,\n", " max_feature_nodes=8192,\n", " offload='cpu',\n", " lazy_encoder=None,\n", " lazy_decoder=True,\n", " verbose=False,\n", " default_node_threshold=0.8,\n", " default_edge_threshold=0.98,\n", " save_graphs=True,\n", " graph_output_dir=None,\n", " analysis_target_tokens=None,\n", " target_token_ids=None,\n", " use_neuronpedia=False,\n", " intervention_scale_factor=1.0,\n", " intervention_max_influence_norm_scale=False,\n", " intervention_sign_aware_scale=True,\n", " intervention_value=None,\n", " intervention_value_source='top_feature_scores',\n", " intervention_constrained_layers=None,\n", " intervention_freeze_attention=None,\n", " intervention_apply_activation_function=None,\n", " intervention_sparse=False,\n", " intervention_return_activations=False,\n", " nnsight_remote=False,\n", " ndif_api_key=None)\n" ] } ], "source": [ "# Load our demo config (this will be done from the hub once that is available)\n", "base_itdm_cfg, base_it_cfg, dm_cls, m_cls = MODULE_EXAMPLE_REGISTRY.get(\"gemma2.rte_demo.circuit_tracer\")\n", "# Optionally override base_it_cfg.core_log_dir with the notebook parameter if provided\n", "if core_log_dir:\n", " base_it_cfg.core_log_dir = core_log_dir\n", "# If the user requests the baseline salient-logits path, clear the explicit target\n", "# token configuration so the adapter will fall back to the default compute_salient_logits\n", "# implementation. This forces usage of the baseline salient-logits computation.\n", "if use_baseline_salient_logits:\n", " # Clear any explicit token selection so compute_salient_logits() runs its default path\n", " base_it_cfg.circuit_tracer_cfg.analysis_target_tokens = None\n", " base_it_cfg.circuit_tracer_cfg.target_token_ids = None\n", " print(\n", " \"use_baseline_salient_logits=True: cleared analysis_target_tokens and \"\n", " \"target_token_ids -> using default compute_salient_logits path\"\n", " )\n", "else:\n", " print(\"use_baseline_salient_logits=False: keeping configured analysis_target_tokens / target_token_ids (if any)\")\n", "\n", "if enable_analysis_injection:\n", " try:\n", " base_it_cfg.circuit_tracer_cfg.verbose = False\n", " print(\"✓ Analysis injection enabled: disabled circuit_tracer verbose logging to avoid duplicate logs\")\n", " except Exception as e:\n", " print(f\"⚠️ Could not disable circuit_tracer verbose logging: {e}\")\n", "\n", "\n", "# Configure transcoder architecture selection based on the toggle. When True, use the 'gemma'\n", "# CrossLayerTranscoder (demo). When False, point to the HF SingleLayerTranscoder checkpoint URL.\n", "if use_baseline_transcoder_arch:\n", " base_it_cfg.circuit_tracer_cfg.transcoder_set = \"gemma\"\n", " print(\n", " \"use_baseline_transcoder_arch=True: set transcoder_set='gemma' -> \"\n", " \"set transcoder_set to HF URL -> using the SingleLayerTranscoder checkpoint\"\n", " )\n", "else:\n", " base_it_cfg.circuit_tracer_cfg.transcoder_set = \"mntss/clt-gemma-2-2b-426k\"\n", " print(\n", " \"use_baseline_transcoder_arch=False: demo will use CrossLayerTranscoder \"\n", " \"instead of the default TranscoderSet of `SingleLayerTranscoder`s\"\n", " )\n", "\n", "print(pformat(base_it_cfg.circuit_tracer_cfg))" ] }, { "cell_type": "code", "execution_count": 10, "id": "ed79fb58", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:08.487570Z", "iopub.status.busy": "2026-07-30T19:07:08.487434Z", "iopub.status.idle": "2026-07-30T19:07:41.263550Z", "shell.execute_reply": "2026-07-30T19:07:41.262654Z" }, "papermill": { "duration": 32.789855, "end_time": "2026-07-30T19:07:41.264253+00:00", "exception": false, "start_time": "2026-07-30T19:07:08.474398+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e32e0164d469475892dc2e732c9a8f34", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Loading weights: 0%| | 0/288 [00:00= limit_analysis_batches >= 0:\n", " break\n", " # fetch the first test_token_limit from the first example in the batch\n", " first_ex_in_batch = batch[:1]\n", " first_ex_in_batch = first_ex_in_batch[\"input\"]\n", " first_ex_in_batch.squeeze_()\n", " if test_token_limit > 0:\n", " first_ex_in_batch = first_ex_in_batch[-test_token_limit:]\n", " first_ex_in_batch = first_ex_in_batch[first_ex_in_batch != 0]\n", " example_prompts.append(first_ex_in_batch)\n", "else:\n", " # Generate attribution graphs for a few example prompts\n", " example_prompts = [\n", " # \"The capital of France is\",\n", " \"The capital of the state containing Dallas is\",\n", " # \"When I look at the sky, I see\",\n", " ]" ] }, { "cell_type": "code", "execution_count": 12, "id": "37b8490c", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:41.368573Z", "iopub.status.busy": "2026-07-30T19:07:41.368142Z", "iopub.status.idle": "2026-07-30T19:07:41.380226Z", "shell.execute_reply": "2026-07-30T19:07:41.379280Z" }, "papermill": { "duration": 0.026194, "end_time": "2026-07-30T19:07:41.381261+00:00", "exception": false, "start_time": "2026-07-30T19:07:41.355067+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✓ Target tokens set: ['▁Dallas', '▁Austin'] → IDs: tensor([26865, 22605], device='cuda:0')\n" ] } ], "source": [ "# Set tokenizer context for analysis (if hooks are enabled)\n", "if enable_analysis_injection and analysis_injector:\n", " from it_examples.utils.analysis_injection.analysis_hook_patcher import HOOK_REGISTRY\n", " from it_examples.utils.example_helpers import TargetTokenAnalysis\n", "\n", " # Convert target_tokens to IDs using the model's tokenizer\n", " if analysis_injector.config.shared_context[\"target_tokens\"]:\n", " target_tokens = analysis_injector.config.shared_context[\"target_tokens\"]\n", " target_token_analysis = TargetTokenAnalysis(\n", " tokens=target_tokens, tokenizer=ct_module.model.tokenizer, default_device=ct_module.device\n", " )\n", "\n", " HOOK_REGISTRY.set_context(\n", " target_token_ids=target_token_analysis.token_ids, target_token_analysis=target_token_analysis\n", " )\n", " print(f\"✓ Target tokens set: {target_token_analysis.tokens} → IDs: {target_token_analysis.token_ids}\")" ] }, { "cell_type": "markdown", "id": "1b43a8ef", "metadata": { "papermill": { "duration": 0.011583, "end_time": "2026-07-30T19:07:41.404321+00:00", "exception": false, "start_time": "2026-07-30T19:07:41.392738+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Generate Basic Attribution Graph\n", " " ] }, { "cell_type": "code", "execution_count": 13, "id": "fefa89f3", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:41.429075Z", "iopub.status.busy": "2026-07-30T19:07:41.428829Z", "iopub.status.idle": "2026-07-30T19:07:52.250648Z", "shell.execute_reply": "2026-07-30T19:07:52.249756Z" }, "papermill": { "duration": 10.835599, "end_time": "2026-07-30T19:07:52.251578+00:00", "exception": false, "start_time": "2026-07-30T19:07:41.415979+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Generating attribution graphs for example prompts...\n", "\n", "Processing prompt 1: 'The capital of the state containing Dallas is'\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ ":0: Full backward hook is firing when gradients are computed with respect to module outputs since no inputs require gradients. See https://docs.pytorch.org/docs/main/generated/torch.nn.Module.html#torch.nn.Module.register_full_backward_hook for more details.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO] interpretune.utils.logging: Neuronpedia upload not requested. Set upload_to_np to `True` to automatically upload.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:interpretune.utils.logging:Neuronpedia upload not requested. Set upload_to_np to `True` to automatically upload.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Processed 1 prompts successfully\n", "📝 Analysis log available for inspection: /tmp/attribution_flow_analysis_20260730_120707.log\n", " The subsequent cells in the `Annotated Attribution Flow Analysis` will display key analysis points values with any associated annotations in context. You can also inspect the file above for the raw analysis point values and any additional debug information collected during graph generation.\n" ] } ], "source": [ "print(\"Generating attribution graphs for example prompts...\")\n", "slug_base = \"it_circuit_tracer_compute_specific_logits_demo\"\n", "results = []\n", "\n", "for i, prompt in enumerate(example_prompts):\n", " print(f\"\\nProcessing prompt {i + 1}: '{prompt}'\")\n", " slug = f\"{slug_base}_{i + 1}_{datetime.now().strftime('%Y%m%d_%H%M%S')}\"\n", " # Process the batch using the session, the adapter will handle tokenization and graph generation\n", " try:\n", " graph, local_graph_path, _ = ct_module.generate_graph(prompt=prompt, slug=slug)\n", " results.append(local_graph_path)\n", " except Exception as e:\n", " print(f\" - Error processing prompt: {e}\")\n", "\n", "print(f\"\\nProcessed {len(results)} prompts successfully\")\n", "# Check and display analysis log file location if available\n", "if enable_analysis_injection and analysis_injector.analysis_log:\n", " print(f\"📝 Analysis log available for inspection: {analysis_injector.analysis_log}\")\n", " print(\n", " \" The subsequent cells in the `Annotated Attribution Flow Analysis` will display key analysis points values\"\n", " \" with any associated annotations in context. You can also inspect the file above for the raw analysis point\"\n", " \" values and any additional debug information collected during graph generation.\"\n", " )" ] }, { "cell_type": "markdown", "id": "891b694d", "metadata": { "papermill": { "duration": 0.020688, "end_time": "2026-07-30T19:07:52.292262+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.271574+00:00", "status": "completed" }, "tags": [] }, "source": [ "## Annotated Attribution Flow Analysis" ] }, { "cell_type": "markdown", "id": "582c9cb9", "metadata": { "papermill": { "duration": 0.011546, "end_time": "2026-07-30T19:07:52.315428+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.303882+00:00", "status": "completed" }, "tags": [] }, "source": [ "When proceeding with the subsequent annotated attribution analysis, it may be helpful to refer to the following diagram outlining the `transformer_lens` hook architecture and nomenclature (might want to click [here](https://raw.githubusercontent.com/callummcdougall/computational-thread-art/master/example_images/misc/transformer-full-updated.png) to open it in a new tab):\n", "\n", "
\n", "Expand Diagram\n", "\n", "\"Transformer\n", "
\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "58594c4d", "metadata": { "papermill": { "duration": 0.011557, "end_time": "2026-07-30T19:07:52.338808+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.327251+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Model overview" ] }, { "cell_type": "markdown", "id": "ead5e59d", "metadata": { "papermill": { "duration": 0.011647, "end_time": "2026-07-30T19:07:52.362089+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.350442+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### ReplacementModel " ] }, { "cell_type": "markdown", "id": "72372df7", "metadata": { "papermill": { "duration": 0.01168, "end_time": "2026-07-30T19:07:52.386049+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.374369+00:00", "status": "completed" }, "tags": [] }, "source": [ "This module is common for `TranscoderSet` (set of `SingleLayerTranscoders`) and `CrossLayerTranscoder` transcoder architectures\n", "
\n", " ReplacementModel\n", "\n", " ```python\n", " ReplacementModel(\n", " (embed): Embed()\n", " (hook_embed): HookPoint()\n", " (blocks): ModuleList((0-25): 26 x TransformerBlock()) # see `TransformerBlock`\n", " (ln_final): RMSNorm(\n", " (hook_scale): HookPoint()\n", " (hook_normalized): HookPoint()\n", " )\n", " (unembed): ReplacementUnembed(\n", " (old_unembed): Unembed()\n", " (hook_pre): HookPoint()\n", " (hook_post): HookPoint()\n", " )\n", " (transcoders): ... # see relevant transcoder architecture below (e.g. `TranscoderSet` or `CrossLayerTranscoder`)\n", " )\n", " ```\n", "
\n", "
\n", "\n", "- `TransformerBlock` is a TransformerLens transformer block with additional hooks and a replacement MLP \n", "
\n", " TransformerBlock\n", "\n", " ```python\n", " (ln1): RMSNorm(\n", " (hook_scale): HookPoint()\n", " (hook_normalized): HookPoint()\n", " )\n", " (ln1_post): RMSNorm(\n", " (hook_scale): HookPoint()\n", " (hook_normalized): HookPoint()\n", " )\n", " (ln2): RMSNorm(\n", " (hook_scale): HookPoint()\n", " (hook_normalized): HookPoint()\n", " )\n", " (ln2_post): RMSNorm(\n", " (hook_scale): HookPoint()\n", " (hook_normalized): HookPoint()\n", " )\n", " (attn): GroupedQueryAttention(\n", " (hook_k): HookPoint()\n", " (hook_q): HookPoint()\n", " (hook_v): HookPoint()\n", " (hook_z): HookPoint()\n", " (hook_attn_scores): HookPoint()\n", " (hook_pattern): HookPoint()\n", " (hook_result): HookPoint()\n", " (hook_rot_k): HookPoint()\n", " (hook_rot_q): HookPoint()\n", " )\n", " (mlp): ReplacementMLP(\n", " (old_mlp): GatedMLP(\n", " (hook_pre): HookPoint()\n", " (hook_pre_linear): HookPoint()\n", " (hook_post): HookPoint()\n", " )\n", " (hook_in): HookPoint()\n", " (hook_out): HookPoint()\n", " )\n", " (hook_attn_in): HookPoint()\n", " (hook_q_input): HookPoint()\n", " (hook_k_input): HookPoint()\n", " (hook_v_input): HookPoint()\n", " (hook_mlp_in): HookPoint()\n", " (hook_attn_out): HookPoint()\n", " (hook_mlp_out): HookPoint(\n", " (hook_out_grad): HookPoint()\n", " )\n", " (hook_resid_pre): HookPoint()\n", " (hook_resid_mid): HookPoint()\n", " (hook_resid_post): HookPoint()\n", " ```\n", "
" ] }, { "cell_type": "markdown", "id": "41011e15", "metadata": { "papermill": { "duration": 0.011504, "end_time": "2026-07-30T19:07:52.409118+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.397614+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Transcoders" ] }, { "cell_type": "markdown", "id": "1f07ac88", "metadata": { "papermill": { "duration": 0.011828, "end_time": "2026-07-30T19:07:52.432633+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.420805+00:00", "status": "completed" }, "tags": [] }, "source": [ "- When using a `TranscoderSet` set of `SingleLayerTranscoders` as the transcoder architecture:\n", "\n", "
\n", " TranscoderSet\n", "\n", " ```python\n", " (transcoders): TranscoderSet(\n", " (transcoders): ModuleList(\n", " (0): SingleLayerTranscoder(\n", " (activation_function): JumpReLU(\n", " threshold=Parameter containing:\n", " tensor(0.5664, device='cuda:0', dtype=torch.bfloat16), bandwidth=0.1\n", " )\n", " )\n", " ...\n", " (25): SingleLayerTranscoder(\n", " (activation_function): JumpReLU(\n", " threshold=Parameter containing:\n", " tensor(6.1250, device='cuda:0', dtype=torch.bfloat16), bandwidth=0.1\n", " )\n", " )\n", " )\n", " )\n", " ```\n", "
\n", "
\n", "- When using a `CrossLayerTranscoder` as the transcoder architecture:\n", " ```python\n", " (transcoders): CrossLayerTranscoder()\n", " ```\n", "\n", "
\n", " Note on Lazy Decoders\n", "\n", " - `lazy_decoders` is by default `True` so to get W_dec (without excessive memory demands) we log W_dec manually from within `compute_attribution_components`\n", " ```python\n", " {n:p.shape for n,p in model.transcoders.named_parameters()}\n", " {'W_enc': torch.Size([26, 16384, 2304]), 'b_dec': torch.Size([26, 2304]), 'b_enc': torch.Size([26, 16384])}\n", " ```\n", " - notice since each CLT feature has a single encoder weight but decoder weights that output to each subqeuent layer MLP output, our W_dec are shaped accordingly, `n_output_layers = self.n_layers - layer_id`\n", " - the W_dec then is shaped `(d_transcoder, n_output_layers, d_model)`\n", " - in addition to lazily loading decoder weights on demand to save memory, only the needed feature ids are loaded as well\n", " ```python\n", " w_dec_shapes = {}\n", " for l in range(self.n_layers):\n", " path = os.path.join(self.clt_path, f\"W_dec_{l}.safetensors\")\n", " with safe_open(path, framework=\"pt\", device=self.device.type) as f:\n", " w_dec_shapes[l] = f.get_slice(f\"W_dec_{l}\")[:].to(device).to(dtype).shape\n", " print(w_dec_shapes)\n", " {\n", " 0: torch.Size([16384, 26, 2304]), \n", " 1: torch.Size([16384, 25, 2304]), \n", " ...\n", " 24: torch.Size([16384, 2, 2304]), \n", " 25: torch.Size([16384, 1, 2304])\n", " }\n", " ```\n", "
" ] }, { "cell_type": "markdown", "id": "d84892b0", "metadata": { "papermill": { "duration": 0.012398, "end_time": "2026-07-30T19:07:52.457136+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.444738+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Precompute Activations, Setup Hooks and Generate `AttributionContext`" ] }, { "cell_type": "markdown", "id": "5a6389c1", "metadata": { "papermill": { "duration": 0.011842, "end_time": "2026-07-30T19:07:52.481400+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.469558+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Attribution Hooks Setup and Activation Precomputation" ] }, { "cell_type": "markdown", "id": "9055d8a4", "metadata": { "papermill": { "duration": 0.011806, "end_time": "2026-07-30T19:07:52.505225+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.493419+00:00", "status": "completed" }, "tags": [] }, "source": [ "\n", "- This phase precomputes the ReplacementModel and transcoder activations as well as the error vectors, saving them and the token embeddings.\n", "```python\n", " ctx = model.setup_attribution(input_ids)\n", "```\n", "\n", "- `get_caching_hooks` is used to get mlp in and out caching hooks\n", "\n", " ```python\n", " mlp_in_cache, mlp_in_caching_hooks, _ = self.get_caching_hooks(\n", " lambda name: self.feature_input_hook in name\n", " ) \n", " mlp_out_cache, mlp_out_caching_hooks, _ = self.get_caching_hooks(\n", " lambda name: self.feature_output_hook in name\n", " )\n", " ```\n", "- `get_caching_hooks`: normal TL caching hook for getting targeted mlp activations\n", "\n", "- `run_with_hooks` is called with just the mlp in and out caching hooks\n", "\n", " - this collects the original model's activations and mlp in and out hook points (without using the trained transcoders)\n", " ```python\n", " logits = self.run_with_hooks(tokens, fwd_hooks=mlp_in_caching_hooks + mlp_out_caching_hooks)\n", " ```\n", "- Note `ReplacementModel` when configured adds a `hook_out_grad` `HookPoint` to the subblock that handles the skip connection after mlp_out and enables hooking into the gradients of the function (that wouldn't be possible with backward since the acts are detached)\n", "- the `hook_out_grad` hook is important as it is the output of the mlp put to the residual (after including the skip connection), used for lots of subsequent computation (error, scores)\n", " - e.g. when error is calculated below using `mlp_out_cache`, it uses the special `hook_out_grad` that was added" ] }, { "cell_type": "markdown", "id": "ebaa988e", "metadata": { "papermill": { "duration": 0.012695, "end_time": "2026-07-30T19:07:52.530717+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.518022+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Transcoder Architecture-Specific Attribution" ] }, { "cell_type": "markdown", "id": "0c47b62c", "metadata": { "papermill": { "duration": 0.01199, "end_time": "2026-07-30T19:07:52.554992+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.543002+00:00", "status": "completed" }, "tags": [] }, "source": [ "- `compute_attribution_components` is called to collect all attribution_data required for `AttributionContext`\n", "- this method is transcoder type specific, single layer transcoders form a `TranscoderSet` that have this method, `CrossLayerTranscoders` have a different version\n", "```python\n", " attribution_data = self.transcoders.compute_attribution_components(mlp_in_cache, self.zero_positions)\n", "```\n", "- we construct the `activation_matrix` using `compute_attribution_components` which allows multiple different transcoder architectures to be used\n", "\n", "- For `TranscoderSet` (set of `SingleLayerTranscoder`)\n", "\n", "
\n", " TranscoderSet Attribution Context
\n", "\n", " - note that this implementation is for the `SingleLayerTranscoders` not cross-layer transcoders. CLTs have same number of encoder parameters but `num_layers/2 `times more decoder parameters and have separate decoder vectors for each subsequent layer. We can see the current reconstruction uses just the corresponding input to that layer and collects that SLT's output. See the `CrossLayerTranscoder` version of `compute_attribution_components` for CLT mechanics\n", "\n", " - we construct per layer sparse activations (zeroing out bos activations also) using our trained transcoders (remember ReplacementMLP is the original model with specially instrumented MLP hooks for inspection/replacement of activations etc)\n", "\n", " - here is where the reconstruction is calculated by passing in the captured/cached mlp_input (mlp_in_cache) to the relevant trained transcoders \n", "\n", " - `compute_attribution_components` uses `encode_sparse` and `decode_sparse` methods on each transcoder layer to collect our required attribution context and package it in a `AttributionContext` dataclass:\n", " - `activation_matrix`: Sparse (n_layers, n_pos, d_transcoder) activations\n", " - `reconstruction`: (n_layers, n_pos, d_model) reconstructed outputs\n", " - `encoder_vecs`: Concatenated encoder vectors for active features\n", " - `decoder_vecs`: Concatenated decoder vectors (scaled by activations)\n", " - `encoder_to_decoder_map`: Mapping from encoder to decoder indices\n", "\n", " - `encode_sparse`\n", " - accepts incoming activations (for SLT, feature_input_hook is `ln2.hook_normalized`) and uses the current layer's self.W_enc and b_enc to calc pre_acts and the activation function to calc the acts, sets the bos acts to 0 and calculates the active (nnz, indices()) encoders\n", " - these are the local replacement model transcoder preactivations\n", " - gets non-zero indices, getting the trained transcoder encoder layers (transposed) for active features\n", " - **NOTE**: crucially, these target transcoder feature preactivations are linear in each upstream source transcoder feature activations since we freeze attention patterns and normalization denominators!\n", " - example sparse_acts, active_encoder shapes for gemma layer 0:\n", " ```python\n", " sparse_acts.shape \n", " torch.Size([9, 16384]) # n_pos, d_transcoder\n", " active_encoders.shape\n", " torch.Size([634, 2304]) # num_active_features for the layer, d_model\n", " ```\n", " - `decode_sparse`\n", " - accepts the decoded output activations and scales the relevant W_dec indices by the activations\n", " - Return decoder rows for **active** features only from the trained transcoders \n", " - uses `indices()` to get non-zero indices so requires sparse tensor\n", " - for each active feature index for each layer, creates a [n_active_features_for_layer, d_model] tensor\n", " - example shapes for gemma layer 0\n", " ```python\n", " W_dec.shape\n", " torch.Size([16384, 2304])\n", " transcoders[layer].W_dec[feat_idx].shape\n", " torch.Size([634, 2304])\n", " ```\n", "
\n", "
\n", "\n", "- For `CrossLayerTranscoder` Attribution Context\n", "\n", "
\n", " CrossLayerTranscoder Attribution Context
\n", " \n", " - a crucial difference between SLT and CLT attribution flows is that for SLT, `decode_sparse` returns a per-layer reconstruction, whereas for CLT, a separate `compute_reconstruction` step is required to calculate and store the reconstruction separately for each subsequent layer it outputs to (since each layer has a separate decoder for each subsequent layer)\n", " - encode_sparse\n", " ```python\n", " sparse_acts, active_encoders = transcoder.encode_sparse(mlp_inputs[layer], zero_first_pos=True)\n", " ```\n", " - select_decoder_vectors\n", " ```python\n", " pos_ids, layer_ids, feat_ids, decoder_vectors, encoder_to_decoder_map = (\n", " self.select_decoder_vectors(features)\n", " )\n", " ```\n", " - compute_reconstruction\n", " ```python\n", " reconstruction = self.compute_reconstruction(pos_ids, layer_ids, decoder_vectors) \n", " ```\n", "
" ] }, { "cell_type": "markdown", "id": "507676d5", "metadata": { "papermill": { "duration": 0.011858, "end_time": "2026-07-30T19:07:52.579102+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.567244+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of Transcoder-Specific attribution described above." ] }, { "cell_type": "code", "execution_count": 14, "id": "9c6deb9a", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:52.603975Z", "iopub.status.busy": "2026-07-30T19:07:52.603723Z", "iopub.status.idle": "2026-07-30T19:07:52.608809Z", "shell.execute_reply": "2026-07-30T19:07:52.607961Z" }, "papermill": { "duration": 0.019117, "end_time": "2026-07-30T19:07:52.609887+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.590770+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
attribution_data['reconstruction'].shape(26, 9, 2304)n_layers, n_pos, d_model
attribution_data['decoder_vecs'].shape(7140, 2304)num_active_features, d_model
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_compute_attribution_end\")" ] }, { "cell_type": "markdown", "id": "8145c40c", "metadata": { "papermill": { "duration": 0.011717, "end_time": "2026-07-30T19:07:52.633669+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.621952+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Error Vectors and Reconstruction" ] }, { "cell_type": "markdown", "id": "2d41cdb7", "metadata": { "papermill": { "duration": 0.011987, "end_time": "2026-07-30T19:07:52.657496+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.645509+00:00", "status": "completed" }, "tags": [] }, "source": [ "- Finally, we then compute the error vectors, the actual (replacement)MLP layers, the mlp_out_cache are the original acts and the reconstructed-from transcoder acts are the reconstruction\n", " ```python\n", " error_vectors = mlp_out_cache - attribution_data[\"reconstruction\"]\n", " ```\n", "- we also save the token vector positions\n", " ```python\n", " token_vectors = self.W_E[tokens].detach() # (n_pos, d_model)\n", " ```\n", "- all the per-layer active feature decoder vectors are scaled by how much each transcoder feature was activated \n", "- at the end of setup_attribution we, package the attribution components into an `AttributionContext`" ] }, { "cell_type": "markdown", "id": "a991c57f", "metadata": { "papermill": { "duration": 0.012389, "end_time": "2026-07-30T19:07:52.681721+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.669332+00:00", "status": "completed" }, "tags": [] }, "source": [ "\n", "\n", "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Precompute Activations, Setup Hooks and Generate AttributionContext`." ] }, { "cell_type": "code", "execution_count": 15, "id": "35a0290b", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:52.793838Z", "iopub.status.busy": "2026-07-30T19:07:52.793609Z", "iopub.status.idle": "2026-07-30T19:07:52.798312Z", "shell.execute_reply": "2026-07-30T19:07:52.797445Z" }, "papermill": { "duration": 0.105554, "end_time": "2026-07-30T19:07:52.799441+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.693887+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
ctx.activation_matrix.shape(26, 9, 16384)
ctx.decoder_vecs.shape(7140, 2304)
ctx.encoder_vecs.shape(7140, 2304)
ctx.logits.shape(1, 9, 256000)
ctx.token_vectors.shape(9, 2304)
ctx.error_vectors.shape(26, 9, 2304)
ctx.encoder_to_decoder_map.shape(7140,)
ctx.decoder_locations.shape(2, 7140)
n_layers26
_row_size7383total_active_feats + error_nodes ((n_layers + 1) * n_pos) # + logits later
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_setup_attribution_end\")" ] }, { "cell_type": "code", "execution_count": 16, "id": "e6cdf79e", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:52.825444Z", "iopub.status.busy": "2026-07-30T19:07:52.825240Z", "iopub.status.idle": "2026-07-30T19:07:52.829963Z", "shell.execute_reply": "2026-07-30T19:07:52.829101Z" }, "papermill": { "duration": 0.018681, "end_time": "2026-07-30T19:07:52.830884+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.812203+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
KeyValue
target_token_ids[26865, 22605]
target_logits[23.375, 25.75]
target_tokens['▁Dallas', '▁Austin']
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_precomputation_phase_end\")" ] }, { "cell_type": "markdown", "id": "68e5ac86", "metadata": { "papermill": { "duration": 0.011964, "end_time": "2026-07-30T19:07:52.855038+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.843074+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Forward Pass" ] }, { "cell_type": "markdown", "id": "de287cd2", "metadata": { "papermill": { "duration": 0.012608, "end_time": "2026-07-30T19:07:52.879878+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.867270+00:00", "status": "completed" }, "tags": [] }, "source": [ "- When we run the forward pass, we do so under the `install_hooks` context manager of the `AttributionContext` object we created above:\n", "\n", " ```python\n", " def install_hooks(self, model: \"ReplacementModel\"):\n", " \"\"\"Context manager instruments the hooks for the forward and backward passes.\"\"\"\n", " with model.hooks(\n", " fwd_hooks=self._caching_hooks(model.feature_input_hook), # type: ignore\n", " bwd_hooks=self._make_attribution_hooks(model.feature_output_hook), # type: ignore\n", " ):\n", " yield\n", " ```\n", "- `AttributionContext._caching_hooks` `fwd` hooks are installed\n", "\n", " - these cache the layerwise residual activations for the replacement model in `AttributionContext._resid_activations` (as well as the `unembed.hook_pre` acts after the last layer)\n", " - for this gemma example, the hook currently used for SLT is the feature_input_hook `ln2.hook_normalized` and for CLT `hook_resid_mid` (so the input for the MLP/transcoder)\n", "\n", "- `AttributionContext._make_attribution_hooks` `bwd` hooks are installed\n", " - these are the `bwd` hooks installed below via `_make_attribution_hooks`\n", " - a `model.forward` is run (stopping at the last layer), note the input is expanded to the `batch_size` num of nodes to process per position, so with context size 9 and a batch_size of 256 for this example, our input would be `torch.Size([256, 9])`\n", " - the final residual activations are set on the context manager `ctx._resid_activations[-1] = model.ln_final(residual)`\n", " - after the forward, the `ReplacementModel` MLP block modules can be offloaded (all original module MLP blocks `[block.mlp for block in model.blocks]`)\n" ] }, { "cell_type": "markdown", "id": "2a88101f", "metadata": { "papermill": { "duration": 0.012406, "end_time": "2026-07-30T19:07:52.904734+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.892328+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Key Bwd Hooks: `_make_attribution_hooks`" ] }, { "cell_type": "markdown", "id": "887f4bae", "metadata": { "papermill": { "duration": 0.011897, "end_time": "2026-07-30T19:07:52.929170+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.917273+00:00", "status": "completed" }, "tags": [] }, "source": [ "- `_make_attribution_hooks` is a function that associates the bwd attribution hook factory function `_compute_score_hook` with the feature/logit, error and token node types. Inside the TransformerLens `hooks` context manager, these hooks are enabled and installed during the forward pass in this phase (phase 1)\n", "- These hooks are subsequently used for both logit attribution (Phase 3) and feature attribution (Phase 4)\n", " - for phase 3, the gradient of the pre-softmax logit (minus the mean logit) is injected as the demeaned logits are passed to `compute_batch` and `backward` is run while injecting the relevant gradients to einsum with the source decoder vectors\n", " - for phase 4, the upstream target encoder vectors are similarly injected (see `key attribution gradient flow summary` below)\n", " - see `compute_batch` below for the mechanics of using backward hooks to orchestrate logit/feature/error/token node attribution\n", "\n", "- Attribution Hook Details\n", "
\n", " Node Attribution Hook Construction
\n", "\n", " - the feature/logit node attribution bwd hooks are constructed:\n", " 1. layerwise using active features (non-zero activations) from the precomputed `activation_matrix` sparse indices\n", " ```python\n", " nnz_layers, nnz_positions = self.decoder_locations\n", " ```\n", " - `nnz_layers, nnz_positions` are the non-zero indices for layers and positions dimensions of `activation_matrix.indices()`\n", " - for this gemma example, layer 25 has 265 active features distributed across all 8 positions (position 0 is bos so is zeroed out), e.g. 31 features active at position 8:\n", " ```python\n", " nnz_layers[-265:].unique()\n", " tensor([25], device='cuda:0')\n", " nnz_positions[-265:].unique()\n", " tensor([1, 2, 3, 4, 5, 6, 7, 8], device='cuda:0')\n", " nnz_positions[-31:].unique()\n", " tensor([8], device='cuda:0')\n", " ```\n", " 2. by passing in the appropriate feature decoder vectors (from the trained transcoders)\n", " ```python\n", " # Feature nodes\n", " feature_hooks = [\n", " self._compute_score_hook(\n", " f\"blocks.{layer}.{feature_output_hook}\",\n", " self.decoder_vecs[layer_mask],\n", " write_index=self.encoder_to_decoder_map[layer_mask], # type: ignore\n", " read_index=np.s_[:, nnz_positions[layer_mask]], # type: ignore\n", " )\n", " for layer in range(n_layers)\n", " if (layer_mask := nnz_layers == layer).any()\n", " ]\n", " ```\n", " - The `np.s_` indexing functionality is used to construct a slice that can be used to select the appropriate \"gradient\" vectors from the injected `logit_vecs` in our backward pass.\n", " - All rows in the first dimension (the batch dimension, in this case our `logit_vecs` batch where only 10 of the 256 rows will be active) are selected.\n", " - For the second dim, the residual activation position indices that are associated with non-zero feature activations in our `activation_matrix` for each layer. \n", " - This will result in the appropriate `logit_vec` \"grads\" vectors getting selected repeatedly for the appropriate positions that have active features for that layer.\n", " - Concretely, for layer 25 in our gemma SLT example, we see the `logit_vec` injected \"grads\" we einsum with our `output_vecs` `decoder_vecs` slice (265 active decoder vecs associated with features for that layer) for the given layer results in the desired shapes for our einsum. Each target `logit_vec` will be repeated 31 times for position 8 so the corresponding 31 active feature decoder vectors for position 8 will be einsum'd with them and written to the appropriate positions in our attribution score tensor:\n", " ```python\n", " # these are the pos indices associated with the active features for layer 25\n", " read_index[1]\n", " tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n", " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n", " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2,\n", " 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n", " 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,\n", " 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,\n", " 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5,\n", " 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,\n", " 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7,\n", " 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8,\n", " 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,\n", " 8], device='cuda:0')\n", " read_index[1].shape\n", " torch.Size([265])\n", "\n", " write_index.shape # corresponding active feature decoder vec indices\n", " torch.Size([265])\n", " write_index.min()\n", " tensor(6850, device='cuda:0')\n", " write_index.max()\n", " tensor(7114, device='cuda:0')\n", " \n", " grads.shape\n", " torch.Size([256, 9, 2304]) # batch_size, n_pos, d_model\n", " grads.to(output_vecs.dtype)[read_index].shape\n", " torch.Size([256, 265, 2304]) # batch_size, num active position/feature combos for layer, d_model\n", " \n", " output_vecs.shape\n", " torch.Size([265, 2304]) # num active position/feature combos for layer, d_model\n", " ```\n", "\n", " - the error and token node hooks are constructed similarly, except:\n", " 1. an `error_offset` function is used to calculate the appropriate node indices\n", " ```python\n", " def error_offset(layer: int) -> int: # starting row for this layer\n", " return self.activation_matrix._nnz() + layer * n_pos\n", " ```\n", " 2. instead of binding the scaled decoder vectors to the hooks, the error and token node hooks use the precomputed `error_vectors` and `token_vectors` respectively\"\n", " ```python\n", " error_hooks = [\n", " self._compute_score_hook(\n", " f\"blocks.{layer}.{feature_output_hook}\",\n", " self.error_vectors[layer],\n", " write_index=np.s_[error_offset(layer) : error_offset(layer + 1)],\n", " )\n", " for layer in range(n_layers)\n", " ]\n", "\n", " # Token-embedding nodes\n", " tok_start = error_offset(n_layers)\n", " token_hook = [\n", " self._compute_score_hook(\n", " \"hook_embed\",\n", " self.token_vectors,\n", " write_index=np.s_[tok_start : tok_start + n_pos],\n", " )\n", " ]\n", " ```\n", "\n", " - The actual bwd hook binds our `AttributionContext` ref to form the closure we use to output the attribution scores to the correct rows in our attribution score edge matrix. The scores are buffered on a per-batch basis, each batch calculating the source attribution scores for all nodes w.r.t. the current batch size of target nodes:\n", "\n", " ```python\n", " def _compute_score_hook(\n", " self,\n", " hook_name: str,\n", " output_vecs: torch.Tensor,\n", " write_index: slice,\n", " read_index: slice | np.ndarray = np.s_[:],\n", " ) -> tuple[str, Callable]:\n", " \"\"\"\n", " Factory that contracts *gradients* with an **output vector set**.\n", " The hook computes A_{s->t} and writes the result into an in-place buffer row.\n", " \"\"\"\n", "\n", " proxy = weakref.proxy(self)\n", "\n", " def _hook_fn(grads: torch.Tensor, hook: HookPoint) -> None:\n", " proxy._batch_buffer[write_index] += einsum(\n", " grads.to(output_vecs.dtype)[read_index],\n", " output_vecs,\n", " \"batch position d_model, position d_model -> position batch\",\n", " )\n", "\n", " return hook_name, _hook_fn\n", " ```\n", "
\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "ce8ce5ed", "metadata": { "papermill": { "duration": 0.011854, "end_time": "2026-07-30T19:07:52.952843+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.940989+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Forward Pass`." ] }, { "cell_type": "code", "execution_count": 17, "id": "a1e59447", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:52.978605Z", "iopub.status.busy": "2026-07-30T19:07:52.978331Z", "iopub.status.idle": "2026-07-30T19:07:52.983447Z", "shell.execute_reply": "2026-07-30T19:07:52.982477Z" }, "papermill": { "duration": 0.019224, "end_time": "2026-07-30T19:07:52.984001+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.964777+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
KeyValue
target_token_ids[26865, 22605]
target_logits[31.125, 39.0]
target_tokens['▁Dallas', '▁Austin']
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_forward_pass_end\")" ] }, { "cell_type": "markdown", "id": "fbc8b07c", "metadata": { "papermill": { "duration": 0.012454, "end_time": "2026-07-30T19:07:53.008990+00:00", "exception": false, "start_time": "2026-07-30T19:07:52.996536+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Build Input Vectors" ] }, { "cell_type": "markdown", "id": "d08a9e93", "metadata": { "papermill": { "duration": 0.012057, "end_time": "2026-07-30T19:07:53.033441+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.021384+00:00", "status": "completed" }, "tags": [] }, "source": [ "We filter active features of our cached transcoder activation_matrix and depending on our target logits mode, generate our feature matrices we want to use in subsequent analysis by using either the:\n", "\n", "1. configured cumulative probability and `max_n_logits` if in the baseline default `compute_salient_logits` target logits mode\n", "2. specific logits specified by token id or token if we use our `compute_specific_logits` target logits mode" ] }, { "cell_type": "markdown", "id": "d4910056", "metadata": { "papermill": { "duration": 0.012658, "end_time": "2026-07-30T19:07:53.058798+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.046140+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Build Input Vectors`." ] }, { "cell_type": "code", "execution_count": 18, "id": "79bf53b2", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.084946Z", "iopub.status.busy": "2026-07-30T19:07:53.084707Z", "iopub.status.idle": "2026-07-30T19:07:53.089974Z", "shell.execute_reply": "2026-07-30T19:07:53.089127Z" }, "papermill": { "duration": 0.019963, "end_time": "2026-07-30T19:07:53.090828+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.070865+00:00", "status": "completed" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
logit_idx[22605, 573, 235248, 9778, 9447, 235292, 26865, 476, 3151, 109]
logit_p[0.304688, 0.126953, 0.077148, 0.046875, 0.04126, 0.032227, 0.028442, 0.028442, 0.022095, 0.017212]logit probabilities
target_tokens['▁Austin', '▁Dallas']
target_logit_indices[0, 6]
target_logit_p[0.304688, 0.028442]
logit_cumulative_prob0.7265625
total_nodes7393
max_feature_nodes7140
total_active_feats7140
n_logits10
n_layers26
n_pos9
max_n_logits10
edge_matrix.shapetorch.Size([7150, 7393])
row_to_node_index.shapetorch.Size([7150])
logit_vecs.shapetorch.Size([10, 2304])
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_build_input_vectors_end\")" ] }, { "cell_type": "markdown", "id": "b8028288", "metadata": { "papermill": { "duration": 0.012126, "end_time": "2026-07-30T19:07:53.115479+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.103353+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Core `compute_batch` Attribution Logic" ] }, { "cell_type": "markdown", "id": "85d6bf33", "metadata": { "papermill": { "duration": 0.011979, "end_time": "2026-07-30T19:07:53.139520+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.127541+00:00", "status": "completed" }, "tags": [] }, "source": [ "In the `Compute Logit Attributions` and `Compute Feature Attributions` phases, it's important to note we aren't using conventional gradient propagation but rather using `backward()` as a convenient orchestration mechanism to calculate our desired node attributions via custom gradient injection.\n", "\n", "The core mechanics of this are:\n", "\n", "1. The `compute_batch` method of `AttributionContext` which is shared among both the Logit and Feature Node Attribution phases: \n", "
\n", " Snapshot of `compute_batch`
\n", "\n", " ```python\n", " def compute_batch(\n", " self,\n", " layers: torch.Tensor,\n", " positions: torch.Tensor,\n", " inject_values: torch.Tensor,\n", " retain_graph: bool = True,\n", " ) -> torch.Tensor:\n", " \"\"\"Return attribution rows for a batch of (layer, pos) nodes.\n", "\n", " The routine overrides gradients at **exact** residual-stream locations\n", " triggers one backward pass, and copies the rows from the internal buffer.\n", "\n", " Args:\n", " layers: 1-D tensor of layer indices *l* for the source nodes.\n", " positions: 1-D tensor of token positions *c* for the source nodes.\n", " inject_values: `(batch, d_model)` tensor with outer product\n", " a_s * W^(enc/dec) to inject as custom gradient.\n", "\n", " Returns:\n", " torch.Tensor: ``(batch, row_size)`` matrix - one row per node.\n", " \"\"\"\n", "\n", " assert self._resid_activations[0] is not None, \"Residual activations are not cached\"\n", " batch_size = self._resid_activations[0].shape[0]\n", " self._batch_buffer = torch.zeros(\n", " self._row_size,\n", " batch_size,\n", " dtype=inject_values.dtype,\n", " device=inject_values.device,\n", " )\n", "\n", " # Custom gradient injection (per-layer registration)\n", " batch_idx = torch.arange(len(layers), device=layers.device)\n", "\n", " def _inject(grads, *, batch_indices, pos_indices, values):\n", " grads_out = grads.clone().to(values.dtype)\n", " grads_out.index_put_((batch_indices, pos_indices), values)\n", " return grads_out.to(grads.dtype)\n", "\n", " handles = []\n", " layers_in_batch = layers.unique().tolist()\n", "\n", " for layer in layers_in_batch:\n", " mask = layers == layer\n", " if not mask.any():\n", " continue\n", " fn = partial(\n", " _inject,\n", " batch_indices=batch_idx[mask],\n", " pos_indices=positions[mask],\n", " values=inject_values[mask],\n", " )\n", " resid_activations = self._resid_activations[int(layer)]\n", " assert resid_activations is not None, \"Residual activations are not cached\"\n", " handles.append(resid_activations.register_hook(fn))\n", "\n", " try:\n", " last_layer = max(layers_in_batch)\n", " self._resid_activations[last_layer].backward(\n", " gradient=torch.zeros_like(self._resid_activations[last_layer]),\n", " retain_graph=retain_graph,\n", " )\n", " finally:\n", " for h in handles:\n", " h.remove()\n", "\n", " buf, self._batch_buffer = self._batch_buffer, None\n", " return buf.T[: len(layers)]\n", " ```\n", "

\n", "2. Logit and Feature Attribution-Specific invocations of `compute_batch`\n", "
\n", " Logit Attribution Computation
\n", "\n", " ```python\n", " for i in range(0, len(logit_idx), batch_size):\n", " batch = logit_vecs[i : i + batch_size]\n", " rows = ctx.compute_batch(\n", " layers=torch.full((batch.shape[0],), n_layers),\n", " positions=torch.full((batch.shape[0],), n_pos - 1),\n", " inject_values=batch,\n", " )\n", " edge_matrix[i : i + batch.shape[0], :logit_offset] = rows.cpu()\n", " row_to_node_index[i : i + batch.shape[0]] = (\n", " torch.arange(i, i + batch.shape[0]) + logit_offset\n", " )\n", " ```\n", "

\n", "\n", "
\n", " Feature Attribution Computation
\n", "\n", " ```python\n", " queue = [pending[i : i + batch_size] for i in range(0, len(pending), batch_size)]\n", "\n", " for idx_batch in queue:\n", " n_visited += len(idx_batch)\n", "\n", " rows = ctx.compute_batch(\n", " layers=feat_layers[idx_batch],\n", " positions=feat_pos[idx_batch],\n", " inject_values=ctx.encoder_vecs[idx_batch],\n", " retain_graph=n_visited < max_feature_nodes,\n", " )\n", "\n", " end = min(st + batch_size, st + rows.shape[0])\n", " edge_matrix[st:end, :logit_offset] = rows.cpu()\n", " row_to_node_index[st:end] = idx_batch\n", " visited[idx_batch] = True\n", " st = end\n", " pbar.update(len(idx_batch))\n", " ```" ] }, { "cell_type": "markdown", "id": "4e153c5d", "metadata": { "papermill": { "duration": 0.012314, "end_time": "2026-07-30T19:07:53.164121+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.151807+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Key Attribution Flow Summary" ] }, { "cell_type": "markdown", "id": "092139ad", "metadata": { "papermill": { "duration": 0.012611, "end_time": "2026-07-30T19:07:53.188900+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.176289+00:00", "status": "completed" }, "tags": [] }, "source": [ "The `_compute_score_hook` invoked by `compute_batch` as described below will calculate the tensor product of:\n", "1. the the unembed vectors (for target logit node attribution) or the relevant target layer encoder vecs (for target feature node attribution) \n", "2. and (for feature nodes) the relevant layer's activation-scaled transcoder decoder vectors (which represent the feature space transformation).\n", "\n", "**This allows us to score how much each feature (scaled by activation) contributes to the residual stream in the unembed (for logit attribution) or target transcoder encoder vec feature (for feature-feature attribution) vector's direction**" ] }, { "cell_type": "markdown", "id": "8c12ded7", "metadata": { "papermill": { "duration": 0.012607, "end_time": "2026-07-30T19:07:53.214112+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.201505+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Compute Logit Attributions" ] }, { "cell_type": "markdown", "id": "439470f1", "metadata": { "papermill": { "duration": 0.012573, "end_time": "2026-07-30T19:07:53.239181+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.226608+00:00", "status": "completed" }, "tags": [] }, "source": [ "Remembering our score computation hooks were registered layer-wise with individual per-layer hook registration, our custom gradient injection hooks are similarly registered layer-wise in `compute_batch`\n", "\n", "In the case of the gemma SLT example with `max_n_logits = 10`, we only have 1 set of `resid_activations` (from layer 26) to register our hooks for since we're computing source node attributions for all nodes w.r.t. only the 10 `logit_vecs` (`logit_vecs.shape = max_n_logits, d_model`), so `layers_in_batch` = [26]\n", "- concretely: \n", " ```python\n", " batch.shape\n", " torch.Size([10, 2304])\n", " logit_vecs.shape\n", " torch.Size([10, 2304])\n", " (layer, len(inject_values[mask]), batch_idx[mask], positions[mask])\n", " (26, 10, tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), tensor([8, 8, 8, 8, 8, 8, 8, 8, 8, 8])) \n", " ```\n", "- each batch will only have non-zero entries in the first 10 rows of dim 0:\n", " ```python\n", " has_nonzero = torch.any(grads.to(output_vecs.dtype)[read_index] != 0, dim=(1, 2))\n", " nz_indices = torch.where(has_nonzero)[0]\n", " nz_indices\n", " tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], device='cuda:0')\n", " ```\n", "We register a backward hook for the last layer (layer 26 in this case) `ctx._resid_activations[int(layer)]` that injects our custom `logit_vecs` (demeaned and transposed logit unembed col vectors) as gradient.\n", "\n", "For each batch, as `backward()` is executed the relevant layer-wise hooks are triggered filling in the corresponding rows of our attribution score buffer layer-by-layer via the einsum in our `_compute_score_hook` bwd hook. The number of elements filled per-hook correspond to the per-layer number of active nodes (feature, error or token): \n", "```python\n", "def _compute_score_hook(\n", "# ...\n", "\n", " proxy = weakref.proxy(self)\n", "\n", " def _hook_fn(grads: torch.Tensor, hook: HookPoint) -> None:\n", " proxy._batch_buffer[write_index] += einsum(\n", " grads.to(output_vecs.dtype)[read_index],\n", " output_vecs,\n", " \"batch position d_model, position d_model -> position batch\",\n", " )\n", " return hook_name, _hook_fn\n", "```\n", "\n", "Note we always `retain_graph` for these backward hooks for subsequent use. `compute_batch` then returns the batch buffer (which is simultaneously zeroed out on the object) which includes the feature nodes + error nodes + token nodes attributions for each of the logits (as `rows`)\n", "\n", "The edge matrix is then updated using the calculated node attributions in rows (while leaving the last `max_n_logit` (10) logit entries 0), so the first 10 rows in `edge_matrix` are populated except for the last 10 columns, the remaining elements are all zero at this point since we're just building the logit attribution entries.\n", "\n", "We finally update the `row_to_node_index` with the logit node mapping, so in this case, logit rows 0:10 map to the last 10 edge_matrix columns, while the 11th flattened node is the first feature node and doesn't have a mapping yet." ] }, { "cell_type": "markdown", "id": "d787088d", "metadata": { "papermill": { "duration": 0.012066, "end_time": "2026-07-30T19:07:53.263377+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.251311+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Reproducing A Specific Node Attribution Calculation" ] }, { "cell_type": "markdown", "id": "8cc09b33", "metadata": { "papermill": { "duration": 0.012009, "end_time": "2026-07-30T19:07:53.287392+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.275383+00:00", "status": "completed" }, "tags": [] }, "source": [ "- Let's inspect the attribution of a specific node (e.g. 6334), to make this concrete. We can set a break in our custom bwd hook and validate the calculation written to our buffer for the target index 6334. This is the einsum referenced above:\n", " ```python\n", " proxy._batch_buffer[write_index] += einsum(grads.to(output_vecs.dtype)[read_index], output_vecs,\n", " \"batch position d_model, position d_model -> position batch\",\n", " )\n", " ```\n", "- The relevant shapes of the tensors involved in this calculation are:\n", " ```python\n", " grads.to(output_vecs.dtype)[read_index].shape # logit_vecs (demeaned and transposed unembedding vectors)\n", " torch.Size([256, 81, 2304]) # batch_size, layer 20 active nodes, d_model\n", " output_vecs.shape # active decoder_vecs for layer\n", " torch.Size([81, 2304]) # layer 20 active nodes, d_model\n", " write_index.shape\n", " torch.Size([81]) # layer 20 active nodes\n", " read_index[1].shape\n", " torch.Size([81]) # layer 20 active nodes\n", " ```\n", "- In this case, we only run `compute_batch` one time since we have fewer than 256 target nodes we want to calculate source attribution scores for. Since we're using backward() for orchestration, we only need to trigger a single \n", "`backward()` that processes all layer-wise score hooks as appropriate for our source nodes. The write_index writes to our score buffer which for each batch (again, just 1 in this case) is of shape `(n_logits, logit_offset)`\n", "- All feature, error and token node scores are calculated, we show just the feature_hooks below. The returned rows for each logit computation batch (just 1) is shape (10, 7358) in this case because we have 10 target logits and 7358 total source nodes (feature, error and token nodes).\n", "- We isolate the relevant indices for our target node 6334, we only have 10 nodes in this batch since we're only calculating source attributions for the 10 target logit nodes.:\n", " ```python\n", " target_node_id = 6334\n", " target_buf_index = torch.where(write_index == target_node_id)\n", " orig_grad_slice = grads.to(output_vecs.dtype)[read_index].detach()\n", " grads_slice = orig_grad_slice[:10, target_buf_index, :].squeeze()\n", " output_vecs_slice = output_vecs[target_buf_index].squeeze()\n", " (grads_slice.shape, output_vecs_slice.shape)\n", " (torch.Size([10, 2304]), torch.Size([2304]))\n", " ```\n", "- We can then reproduce the einsum calculation directly to validate the attribution for our target node 6334\n", " ```python\n", " torch.matmul(grads_slice, output_vecs_slice)\n", " tensor([ 6.5625e+00, -1.4258e-01, -2.7344e-02, 8.3750e+00, 2.7656e+00,\n", " -2.0020e-01, -6.8359e-03, 6.0312e+00, -2.2339e-02, 2.8931e-02],\n", " device='cuda:0', dtype=torch.bfloat16)\n", " ```\n", "- So what are the attributions to our feature node 6334 for our `target_logit_indices`? \n", "\n", " - We projected our decoder vector (associated with node 6334) into the unembed vec direction for each of our top logits individually.\n", " - For our target token ('▁Austin', '▁Dallas') logit indexes, elements 0 and 7, we see 6.5625 and 6.0312.\n", " - As expected, our initial attribution for active feature index 6334 after compute logit attribution:\n", " - returned buffer:\n", " ```python\n", " buf.T[:10][[0,7], 6334] # returned buffer from `compute_batch`\n", " tensor([6.5625, 6.0312], device='cuda:0', dtype=torch.bfloat16)\n", " edge_matrix[[0, 7], 6334] # logit nodes are initially the first `max_n_logits` target nodes\n", " tensor([6.5625, 6.0312])\n", " ```\n" ] }, { "cell_type": "markdown", "id": "a4e8f6d9", "metadata": { "papermill": { "duration": 0.012065, "end_time": "2026-07-30T19:07:53.311477+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.299412+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Attribution Node Id to (Graph) Feature Id Mapping" ] }, { "cell_type": "markdown", "id": "fee35ff8", "metadata": { "papermill": { "duration": 0.012622, "end_time": "2026-07-30T19:07:53.336116+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.323494+00:00", "status": "completed" }, "tags": [] }, "source": [ "\n", "\n", "- If you want to associate these internal node attributions to feature ids on our attribution graphs, we need to map the node ids back to feature ids (we have helper functions for this but it's good to understand the mechanics)\n", "- remember the `ctx.activation_matrix` saves non-zero activations by feature_idx (`0:d_transcoder`) for each layer for each position:\n", "\n", "```python\n", " ctx.activation_matrix\n", " tensor(indices=tensor([[ 0, 0, 0, ..., 25, 25, 25],\n", " [ 1, 1, 1, ..., 8, 8, 8],\n", " [ 41, 96, 253, ..., 16014, 16302, 16326]]),\n", " values=tensor([ 2.5156, 3.5312, 0.7070, ..., 7.4375, 40.7500,\n", " 12.2500]),\n", " device='cuda:0', size=(26, 9, 16384), nnz=7115, dtype=torch.bfloat16,\n", " layout=torch.sparse_coo)\n", "```\n", "- the `encoder_to_decoder_map` map is just a vector `activation_matrix._nnz` long (for SLT) that can be used to associate active encoder and decoder vecs to the activation matrix.\n", "- for example for layer 20, find the positions in our `activation_matrix._nnz` length vector of active decoders\n", "```python\n", " per_layer_mask = (nnz_layers == 20)\n", " nnz_positions[per_layer_mask]\n", " tensor([1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3,\n", " 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,\n", " 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8,\n", " 8, 8, 8, 8, 8, 8, 8, 8, 8], device='cuda:0')\n", "```\n", "- so our active decoders for layer 20 are\n", "\n", "```python\n", " ctx.decoder_vecs[per_layer_mask].shape\n", " torch.Size([81, 2304])\n", "```\n", "\n", "- and the appropriate activation_matrix indices using encoder_to_decoder_map\n", "\n", "```python\n", " ctx.encoder_to_decoder_map[per_layer_mask]\n", " tensor([6254, 6255, 6256, 6257, 6258, 6259, 6260, 6261, 6262, 6263, 6264, 6265,\n", " 6266, 6267, 6268, 6269, 6270, 6271, 6272, 6273, 6274, 6275, 6276, 6277,\n", " 6278, 6279, 6280, 6281, 6282, 6283, 6284, 6285, 6286, 6287, 6288, 6289,\n", " 6290, 6291, 6292, 6293, 6294, 6295, 6296, 6297, 6298, 6299, 6300, 6301,\n", " 6302, 6303, 6304, 6305, 6306, 6307, 6308, 6309, 6310, 6311, 6312, 6313,\n", " 6314, 6315, 6316, 6317, 6318, 6319, 6320, 6321, 6322, 6323, 6324, 6325,\n", " 6326, 6327, 6328, 6329, 6330, 6331, 6332, 6333, 6334], device='cuda:0')\n", "```\n", "- for the our example feature we're probing in `ctx.encoder_to_decoder_map`, what is the active feature index that we got our active encoder and decoder vecs from in activation_matrix?\n", "\n", "```python\n", " ctx.activation_matrix.indices().T[6334]\n", " tensor([ 20, 8, 15589], device='cuda:0')\n", "```\n", "- so **layer 20**, **position 8**, **feature id 15589** is the feature that corresponds to node id 6334\n", "- using `activation_matrix` directly to probe all features active at layer 20, position 8 and their corresponding values, we see we can expect feature 15589 to have an activation of 52.0 in our graph UI\n", "\n", "```python\n", " active_indices = ctx.activation_matrix.indices()\n", " test_mask = (active_indices[0] == 20) & (active_indices[1] == 8)\n", " filtered_indices = active_indices[:, test_mask]\n", " filtered_values = ctx.activation_matrix.values()[test_mask]\n", "\n", " filtered_indices.T\n", " tensor([[ 20, 8, 114],\n", " [ 20, 8, 438],\n", " [ 20, 8, 3094],\n", " [ 20, 8, 5433],\n", " [ 20, 8, 5916],\n", " [ 20, 8, 6026],\n", " [ 20, 8, 10118],\n", " [ 20, 8, 10254],\n", " [ 20, 8, 15133],\n", " [ 20, 8, 15276],\n", " [ 20, 8, 15366],\n", " [ 20, 8, 15589]], device='cuda:0')\n", " filtered_values\n", " tensor([15.5625, 7.8125, 16.7500, 7.5312, 45.7500, 14.8750, 12.1875, 19.0000,\n", " 8.0000, 14.4375, 7.2812, 52.0000], device='cuda:0',\n", " dtype=torch.bfloat16)\n", "```" ] }, { "cell_type": "markdown", "id": "5a652633", "metadata": { "papermill": { "duration": 0.012637, "end_time": "2026-07-30T19:07:53.360975+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.348338+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Compute Logit Attributions`." ] }, { "cell_type": "code", "execution_count": 19, "id": "41b84411", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.388367Z", "iopub.status.busy": "2026-07-30T19:07:53.388127Z", "iopub.status.idle": "2026-07-30T19:07:53.392990Z", "shell.execute_reply": "2026-07-30T19:07:53.392050Z" }, "papermill": { "duration": 0.019774, "end_time": "2026-07-30T19:07:53.393538+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.373764+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
edge_matrix.shape(7150, 7393)
row_to_node_index.shape(7150,)
logit_section.shapetorch.Size([10, 7140])
logit_section_zeros_info(torch.Size([216]), 6895, 7110)logit section zeros shape, min node id, max node id
logit_section_zero_feature_range6894: (24, 8, 16258)
6895: (25, 1, 209)
7110: (25, 7, 15301)
7111: (25, 8, 553)		In the final layer, only the final position has non-zero feature attributions.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_compute_logit_attribution_end\")" ] }, { "cell_type": "markdown", "id": "50bd572b", "metadata": { "papermill": { "duration": 0.01271, "end_time": "2026-07-30T19:07:53.419073+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.406363+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Compute Feature Attributions" ] }, { "cell_type": "markdown", "id": "b40da514", "metadata": { "papermill": { "duration": 0.012903, "end_time": "2026-07-30T19:07:53.444617+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.431714+00:00", "status": "completed" }, "tags": [] }, "source": [ "- Note that the edge_matrix (adjacency matrix) analyzed below is indexed as (target, source) (targets are rows)\n", "- When the graph is pruned, source input edges are normalized for each source node so that they sum to 1 for each target. \n", "- if we are operating on all features (since our `max_feature_nodes` has been set to equal our `total_active_feats`) we can run `compute_batch` below on all the features\n", "- if we are only operating on a subset of features, we first have to run `compute_partial_influences` to sort all the features by logit influence and then score only those features\n", "- `compute_batch` is run once per batch (256 in this example) of nodes to analyze, similar to logit attribution above but instead of injecting `logit_vecs`, we inject the relevant target feature (transcoder encoder vecs) as the gradient. \n", "- Also, instead of just a single compute_batch (with only 10 of the 256 rows populated) and a single `backward` call, we have many more batches to process that will be fully populated (except for the last batch)\n", " - e.g., for this gemma SLT example, we need to compute the source influence vectors for all target active features (7115) which requires 28 batches of 256 nodes each (the last batch will only have 203 nodes)\n", " \n", " ```python\n", " len(queue)\n", " 28\n", " ```\n", "- Our node source vectors are the same as described in logits attribution above (ctx bound in the first forward pass) but we einsum the injected target `encoder_vecs` with them in our bwd hooks to calculate the feature-to-feature attributions\n", "- each batch involves registering the relevant cached resid activations for the layers in a given batch (a batch can cross layer boundaries)\n", "- the max layer in the batch then has backward() called which allows us to calculate the `matmul` of the scaled decoder vec for each active feature (or the token_vecs for the token_vec range, which is the case for layer 0 encoders) with the injected target `encoder_vecs` for active features. \n", "- Another non-feature case are attribution hooks where the `output_vecs` are the `error_vectors` `(n_layers, n_pos, d_model)`, playing the same role as the scaled decoder vecs do in the feature node attributions\n", "- we retain_graph for as long as we have another batch to compute\n", "- we queue up `update_interval` (default 4) * batch_size (so 1024 in this example) nodes unless we have fewer than that left to process\n" ] }, { "cell_type": "markdown", "id": "1f58b7ff", "metadata": { "papermill": { "duration": 0.012815, "end_time": "2026-07-30T19:07:53.470156+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.457341+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Post-Attribution Edge Matrix" ] }, { "cell_type": "markdown", "id": "7b6c79a9", "metadata": { "papermill": { "duration": 0.012296, "end_time": "2026-07-30T19:07:53.495084+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.482788+00:00", "status": "completed" }, "tags": [] }, "source": [ "- `edge_matrix` inspection after feature attributions but before reshaping to `full_edge_matrix` (so first 10 rows are still logit nodes)\n", "- at this point we have non-normalized influence scores for all logit/feature feature/feature edges so the target vectors are pretty dense:\n", "\n", "```python\n", " edge_matrix[0, :].count_nonzero()\n", " tensor(7091)\n", " edge_matrix[6344, :].count_nonzero()\n", " tensor(6423)\n", "```\n", "- An error node exists for every token for every layer. We also have the token nodes themselves.\n", "```python\n", "# n_error_nodes = (num_layers + 1) * num_tokens = num_layers * num_tokens + num_tokens\n", "```\n", "- for the gemma SLT example, `edge_matrix` at this point is shaped as (7125, 7368): \n", "```python\n", "# n_logits(10) + n_feature_nodes(7115) -> 7125\n", "# n_feature_nodes(7115) + n_error_nodes(234) + n_tokens(9) + n_logits(10) -> 7368\n", "```" ] }, { "cell_type": "markdown", "id": "0a694ff1", "metadata": { "papermill": { "duration": 0.01196, "end_time": "2026-07-30T19:07:53.519144+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.507184+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Compute Feature Attributions`." ] }, { "cell_type": "code", "execution_count": 20, "id": "e392f9ca", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.545000Z", "iopub.status.busy": "2026-07-30T19:07:53.544789Z", "iopub.status.idle": "2026-07-30T19:07:53.549807Z", "shell.execute_reply": "2026-07-30T19:07:53.548944Z" }, "papermill": { "duration": 0.019211, "end_time": "2026-07-30T19:07:53.550384+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.531173+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
target_tokens['▁Austin', '▁Dallas']
target_logit_p[0.304688, 0.028442]
top_init_edge_indices[6390, 7381, 7364, 6462, 6695]
[7381, 6390, 6462, 6067, 5952]
top_init_edge_features{6390: (20, 8, 15589), 6462: (21, 8, 5943), 6695: (23, 8, 12237), 7364: 'non-feature node', 7381: 'non-feature node'}
{5952: (14, 7, 2268), 6067: (16, 7, 25), 6390: (20, 8, 15589), 6462: (21, 8, 5943), 7381: 'non-feature node'}
top_init_edge_vals[6.5625, 6.1875, 3.96875, 3.546875, 2.84375]
[10.0625, 6.0625, 3.921875, 3.15625, 2.96875]
features_processed7140
max_features7140
progress100.0%
edge_matrix.shape(7150, 7393)( (n_logits + n_feature_nodes), n_feature_nodes + n_error_nodes + n_token_nodes + n_logits
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_compute_feature_attributions_end\")" ] }, { "cell_type": "markdown", "id": "fe2bc559", "metadata": { "papermill": { "duration": 0.012236, "end_time": "2026-07-30T19:07:53.575141+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.562905+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Graph Packaging" ] }, { "cell_type": "markdown", "id": "983c2b76", "metadata": { "papermill": { "duration": 0.012806, "end_time": "2026-07-30T19:07:53.600718+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.587912+00:00", "status": "completed" }, "tags": [] }, "source": [ "Before packaging the our attribution matrices into a `circuit-tracer` `Graph` object, note our top influencing features are not the same as the raw activations you'll see in the graph from `activation_matrix`. They are our computed feature influences (depending on context, normalized or not-yet-normalized)\n", "\n", "In our analysis point below you can see our top inspected token first order and second order feature attributions.\n", "\n", "- Note this is not yet converted to abs values and normalized so that the input edges sum to 1. The target logit attribution sums do not correspond directly to logit probs but are rather the non-normalized influence scores\n", "- We refer to the top influencing nodes of the top nodes influencing our inspected logits as per token `pre_prune_2nd_order` matrices.\n", "- Our helper functions mask out the error and token nodes from these 2nd order influence matrices since by definition those nodes won't have input feature nodes.\n", "\n", "Prior to graph packaging, we reshape our `edge_matrix` to have the logit nodes at the end and use it to populate the input `full_edge_matrix` which will be passed to the `Graph` constructor as our initial (pre-pruned) adjacency matrix.\n", "\n", "Our adjacency matrix will have the following shape (n_total_nodes, n_total_nodes). Where `n_total_nodes` is:\n", "```\n", "n_feature_nodes + n_error_nodes + n_token_nodes + n_logits\n", "```\n" ] }, { "cell_type": "markdown", "id": "e565257d", "metadata": { "papermill": { "duration": 0.012601, "end_time": "2026-07-30T19:07:53.625821+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.613220+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example prior to `Graph Packaging`." ] }, { "cell_type": "code", "execution_count": 21, "id": "0d8e7c38", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.653287Z", "iopub.status.busy": "2026-07-30T19:07:53.653038Z", "iopub.status.idle": "2026-07-30T19:07:53.658079Z", "shell.execute_reply": "2026-07-30T19:07:53.657167Z" }, "papermill": { "duration": 0.019925, "end_time": "2026-07-30T19:07:53.658695+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.638770+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
full_edge_matrix.shape(7393, 7393)
edge_matrix.shape(7150, 7393)
pre_normalized_logit_node_sum[33.383419, 28.747423]Sum of all source nodes for our target logits before normalization
n_total_nodes7393len(selected_features):(n_layers * n_pos):n_pos:n_logits
pre_prune_1st_order_idxs'▁Austin': [6390, 7381, 7364, 6462, 6695]
'▁Dallas': [7381, 6390, 6462, 6067, 5952]
pre_prune_1st_order_vals'▁Austin': [6.5625, 6.1875, 3.9688, 3.5469, 2.8438]
'▁Dallas': [10.0625, 6.0625, 3.9219, 3.1562, 2.9688]
pre_prune_2nd_order_idxs'▁Austin': [7381, 5952, 6067, 2218, 6249]
[3, 4, 1, 0, 2]
[3, 4, 1, 0, 2]
[6247, 7292, 6074, 7310, 6316]
[6462, 7381, 6390, 6552, 6314]
'▁Dallas': [3, 4, 1, 0, 2]
[7381, 5952, 6067, 2218, 6249]
[6247, 7292, 6074, 7310, 6316]
[7381, 5952, 4230, 2218, 7282]
[7381, 4230, 2218, 7228, 7377]
pre_prune_2nd_order_vals'▁Austin': [51.25, 17.125, 15.0625, 10.625, 10.25]
[-inf, -inf, -inf, -inf, -inf]
[-inf, -inf, -inf, -inf, -inf]
[13.0, 9.0625, 9.0, 6.875, 6.6875]
[10.9375, 7.875, 6.9062, 5.9062, 5.2812]
'▁Dallas': [-inf, -inf, -inf, -inf, -inf]
[51.25, 17.125, 15.0625, 10.625, 10.25]
[13.0, 9.0625, 9.0, 6.875, 6.6875]
[29.375, 22.25, 10.8125, 6.5, 5.0625]
[27.0, 7.5, 5.875, 1.2656, 1.0781]
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_graph_creation_start\")" ] }, { "cell_type": "markdown", "id": "a917c45f", "metadata": { "papermill": { "duration": 0.012814, "end_time": "2026-07-30T19:07:53.684555+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.671741+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Graph Pruning, Creation and Saving" ] }, { "cell_type": "markdown", "id": "cf47133e", "metadata": { "papermill": { "duration": 0.014456, "end_time": "2026-07-30T19:07:53.711910+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.697454+00:00", "status": "completed" }, "tags": [] }, "source": [ "#### Prune by Node/Edge Influence" ] }, { "cell_type": "markdown", "id": "4f0ee5ea", "metadata": { "papermill": { "duration": 0.014154, "end_time": "2026-07-30T19:07:53.739834+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.725680+00:00", "status": "completed" }, "tags": [] }, "source": [ "We next inspect graph pruning process `prune_graph` following the transformation of our target logits raw target (logit) node attribution through the `compute_node_influence` and `compute_edge_influence` functions.\n", "\n", "The first step to computing our node influences is to normalize our adjacency matrix above. We then use this normalized adjacency matrix to compute node influences influence via a numerical approach to Neumann series calculation.\n", "\n", "Iterative computing of our node influence vector starts with seeding our `logit_weights` via our direct logit probabilities. We then multiply those weights with our adjacency matrix `logit_weights @ A` to construct the initial `current_influence` vector and proceed to iteratively update this influence vector until it converges, multiplying increasing degrees of our adjacency matrix by the `current_influence`.\n", "```python\n", " current_influence = logit_weights @ A\n", " influence = current_influence\n", " iterations = 0\n", " while current_influence.any():\n", " if iterations >= max_iter:\n", " raise RuntimeError(\n", " f\"Influence computation failed to converge after {iterations} iterations\"\n", " )\n", " current_influence = current_influence @ A\n", " influence += current_influence\n", " iterations += 1\n", "```" ] }, { "cell_type": "markdown", "id": "4dc34fdd", "metadata": { "papermill": { "duration": 0.012869, "end_time": "2026-07-30T19:07:53.766145+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.753276+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Computing node influences" ] }, { "cell_type": "markdown", "id": "bbbe56ed", "metadata": { "papermill": { "duration": 0.013389, "end_time": "2026-07-30T19:07:53.792521+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.779132+00:00", "status": "completed" }, "tags": [] }, "source": [ "\n", "\n", "For this example, we first inspect the initial adjacency matrix post-normalization within the `compute_influence` function.\n", "\n", "Note we are not updating the entire adjacency matrix in compute_influence, just a single axis of the matrix, an influence/current_influence vector (size n_total_nodes), so we won't see 2nd order nodes updated in A.\n", "\n", "`logit_weights` starts off with as the targets axis of the adjacency matrix (dim 0) and then sets the last `n_logit` weights to the logit probabilities, all the other initial values are initially zero:\n", "```python\n", " logit_weights = torch.zeros(\n", " graph.adjacency_matrix.shape[0], device=graph.adjacency_matrix.device\n", " )\n", " logit_weights[-n_logits:] = graph.logit_probabilities\n", "```\n", "\n", "As we multiply by the adjacency matrix in `compute_influence`, the first iteration will result in the last `n_logits` elements of the first column (the source weights of the first feature) contributing to the dot product with the logit probs, yielding the weighted sum of contributions of the first feature node to the logits (and so on for the remaining nodes).\n", "See the validated/sampled calculation for the first iteration and first feature concretely:\n", "\n", "Observe the first iteration consists of the weighted sum of the direct influences for each source feature on the logit probs:" ] }, { "cell_type": "markdown", "id": "26310c3b", "metadata": { "papermill": { "duration": 0.01325, "end_time": "2026-07-30T19:07:53.818950+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.805700+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the beginning of the `Prune by Node Compute Influence`" ] }, { "cell_type": "code", "execution_count": 22, "id": "685b1cd3", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.845984Z", "iopub.status.busy": "2026-07-30T19:07:53.845708Z", "iopub.status.idle": "2026-07-30T19:07:53.851302Z", "shell.execute_reply": "2026-07-30T19:07:53.850351Z" }, "papermill": { "duration": 0.020398, "end_time": "2026-07-30T19:07:53.851962+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.831564+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
current_influence_shapetorch.Size([7393])
normalized_tgt_logit_nodes[1.0, 1.0]Verify normalization: `A[tta.graph_logit_indices, :].sum(dim=1)`
nodeinf_normed_1st_order_idxs'▁Austin': [6390, 7381, 6884, 7364, 6462]
'▁Dallas': [7381, 6390, 6884, 6462, 7134]
nodeinf_normed_1st_order_vals'▁Austin': [0.0387, 0.0365, 0.031, 0.0234, 0.0209]
'▁Dallas': [0.0564, 0.034, 0.0301, 0.022, 0.0183]
logit_weights[0.304688, 0.126953, 0.077148, 0.046875, 0.04126, 0.032227, 0.028442, 0.028442, 0.022095, 0.017212]logit probabilites are injected as logit_weights to compute the initial influence vector
initial_influence_feat_03.3e-05Validate the initial element of our initial influence vector
`logit_weights[-max_n_logits:].dot(A[-max_n_logits:, 0])`
current_influence_feat_03.3e-05The first element of our current influence vector `current_influence[0]` consists of the weighted sum of the direct influences for each source feature on the logit probs
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_node_compute_influence_init\", skip=[\"trace_dict\", \"context\", \"iteration\"])" ] }, { "cell_type": "markdown", "id": "9f932c00", "metadata": { "papermill": { "duration": 0.012587, "end_time": "2026-07-30T19:07:53.877344+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.864757+00:00", "status": "completed" }, "tags": [] }, "source": [ "See below the evolution of `current_influences` in the node influence computation for this example:" ] }, { "cell_type": "markdown", "id": "2f2d1588", "metadata": { "papermill": { "duration": 0.013308, "end_time": "2026-07-30T19:07:53.903483+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.890175+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Neumann Series Convergence" ] }, { "cell_type": "code", "execution_count": 23, "id": "86ecfd78", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:53.931212Z", "iopub.status.busy": "2026-07-30T19:07:53.930955Z", "iopub.status.idle": "2026-07-30T19:07:54.137729Z", "shell.execute_reply": "2026-07-30T19:07:54.137005Z" }, "papermill": { "duration": 0.223109, "end_time": "2026-07-30T19:07:54.139461+00:00", "exception": false, "start_time": "2026-07-30T19:07:53.916352+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "fill": "toself", "fillcolor": "#440154", "hovertemplate": "Iteration 0
Mean: 9.81e-05
Max: 2.85e-02
Nonzero: 7134/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#440154", "width": 1 }, "name": "Iteration 0", "opacity": 0.6, "type": "scatter", "x": { "bdata": "aHHOwI5Fy8CzGcjA2O3EwP7BwcAilr7ASGq7wG0+uMCSErXAuOaxwNy6rsACj6vAJ2OowEw3pcByC6LAlt+ewLyzm8Dhh5jABlyVwCwwksBRBI/AdtiLwJusiMDAgIXA5lSCwBZSfsBg+nfAqqJxwPRKa8A/82TAiptewNRDWMAe7FHAaJRLwLM8RcD+5D7ASI04wJI1MsDc3SvAJ4YlwHEuH8C81hjAB38SwFEnDMCbzwXAyu/+v2BA8r/2kOW/iuHYvx4yzL8eMsy/iuHYv/aQ5b9gQPK/yu/+v5vPBcBRJwzAB38SwLzWGMBxLh/AJ4YlwNzdK8CSNTLASI04wP7kPsCzPEXAaJRLwB7sUcDUQ1jAiptewD/zZMD0SmvAqqJxwGD6d8AWUn7A5lSCwMCAhcCbrIjAdtiLwFEEj8AsMJLABlyVwOGHmMC8s5vAlt+ewHILosBMN6XAJ2OowAKPq8Dcuq7AuOaxwJIStcBtPrjASGq7wCKWvsD+wcHA2O3EwLMZyMCORcvAaHHOwA==", "dtype": "f4" }, "y": { "bdata": "GA2q/7l8dT8AAAAAAAAAAGQtwPfw23o/dMu0vyLNgj9fkJR/6NuKP6Qquh9XdZE/dMu0vyLNkj8btwmhXHWhPxgNqv+5fKU/0om/f4sdsD8q6JWX9YW6P1t3tUsposI/B2Cn9P2Fyj+9frmTUKDRPwLqlml/Rdo/G7G9frmT4D/zJrh7Q/bhPwEn6RQDFec/mpmZmZmZ6T8rdJxs9eLoP3pcTCHPbOg/PO2bOXEq5z/YPdBlAnLlPzvavE9xyeA/wpeYxw/a2T+tQfQAE5TWP71+uZNQoNE/RTyVC++wyj8nNIR6ep7AP2oih49lN74/OOJ5WLDStT9ZR5v3KS65P9KJv3+LHbA/xO5LD1UsqD/eG7KPCHmjPxLzeMs/IaI/pCq6H1d1kT90y7S/Is2SP8TuSw9VLIg/u06fP1EsiD/Sib9/ix1wP6KBppSQHXA/GA2q/7l8ZT+7Tp8/USyIPwAAAAAAAAAAGA2q/7l8ZT+EVzPGwHxVPxgNqv+5fFU/GA2q/7l8VT8YDar/uXxVPwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -6.451343536376953, -1.5952794551849365 ], "y": [ 0.0, 0.0 ] }, { "fill": "toself", "fillcolor": "#440154", "hovertemplate": "Iteration 1
Mean: 7.45e-05
Max: 2.17e-02
Nonzero: 7104/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#440154", "width": 1 }, "name": "Iteration 1", "opacity": 0.6, "type": "scatter", "x": { "bdata": "WhL2wOos8sB6R+7ACWLqwJh85sAol+LAuLHewEbM2sDW5tbAZgHTwPUbz8CENsvAFFHHwKRrw8Ayhr/AwqC7wFK7t8Dg1bPAcPCvwAALrMCQJajAHkCkwK5aoMA+dZzAzI+YwFyqlMDsxJDAfN+MwAr6iMCaFIXAKi+BwHKTesCQyHLAr/1qwM4yY8DtZ1vADJ1TwCvSS8BJB0TAaDw8wIdxNMCmpizAxdskwOQQHcADRhXAIXsNwECwBcC+yvu//DTsvzqf3L86n9y//DTsv77K+79AsAXAIXsNwANGFcDkEB3AxdskwKamLMCHcTTAaDw8wEkHRMAr0kvADJ1TwO1nW8DOMmPAr/1qwJDIcsByk3rAKi+BwJoUhcAK+ojAfN+MwOzEkMBcqpTAzI+YwD51nMCuWqDAHkCkwJAlqMAAC6zAcPCvwODVs8BSu7fAwqC7wDKGv8Cka8PAFFHHwIQ2y8D1G8/AZgHTwNbm1sBGzNrAuLHewCiX4sCYfObACWLqwHpH7sDqLPLAWhL2wA==", "dtype": "f4" }, "y": { "bdata": "qVQCfOAH8D92Yid2YifwP08oCLKRG/A/N3IjN3Ij8D+bzxMe9ELwP6mVWqmVWvA/ls0akpVa8D/q9xvQhV7wP63USq3USvA/R6US4AM/8D8rtVIrtVLwP5NMHkxmZvA/5EZu5EZu8D8/Ih0OdmLwP5NMHkxmZvA/GHqhF3qh8D9+RDoc7MTwPyGXQwxu5PA/vdALvdAL8T+GXuiFXujxP9t8nvCgF/I/PDb/CoVe8z9LrdRKrdT0P6aIMIBkZvc/XPf/nG8j+j/NzMzMzMz8P8EP/MAP/Ps/cAlVS/FC+z+sFZfQ+cD4P63USq3USvc/2JCA7a8T9T94fzMsUiv0P+DGRaz3gfM/glCkJlIr8j/RC73QC73xP7rlWuQyM/E/T+zETuzE8D88oSDIRm7wP+h2H4pWavA/N3IjN3Ij8D9SqQT4wA/wP73QC73QC/A/VCoBPvAD8D8AAAAAAADwP1QqAT7wA/A/VCoBPvAD8D8AAAAAAADwP6lUAnzgB/A/fuAHfuAH8D/9fgO60AvwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8AAAAAAADwPwAAAAAAAPA/AAAAAAAA8D8=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -7.689740180969238, -1.723609209060669 ], "y": [ 1.0, 1.0 ] }, { "fill": "toself", "fillcolor": "#440154", "hovertemplate": "Iteration 2
Mean: 5.20e-05
Max: 3.47e-02
Nonzero: 6903/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#440154", "width": 1 }, "name": "Iteration 2", "opacity": 0.6, "type": "scatter", "x": { "bdata": "rnUEwWBBAsETDQDBjLH7wPBI98BV4PLAunfuwB4P6sCEpuXA6D3hwE7V3MCybNjAGATUwHybz8DiMsvARsrGwKthwsAQ+b3AdJC5wNontcA+v7DApFaswAjup8BuhaPA0hyfwDe0msCcS5bAAeORwGZ6jcDKEYnAMKmEwJRAgMDzr3fAvd5uwIcNZsBRPF3AG2tUwOSZS8CtyELAd/c5wEEmMcALVSjA1YMfwJ+yFsBp4Q3AMxAFwPp9+L+O2+a/IDnVv7OWw7+zlsO/IDnVv47b5r/6ffi/MxAFwGnhDcCfshbA1YMfwAtVKMBBJjHAd/c5wK3IQsDkmUvAG2tUwFE8XcCHDWbAvd5uwPOvd8CUQIDAMKmEwMoRicBmeo3AAeORwJxLlsA3tJrA0hyfwG6Fo8AI7qfApFaswD6/sMDaJ7XAdJC5wBD5vcCrYcLARsrGwOIyy8B8m8/AGATUwLJs2MBO1dzA6D3hwISm5cAeD+rAunfuwFXg8sDwSPfAjLH7wBMNAMFgQQLBrnUEwQ==", "dtype": "f4" }, "y": { "bdata": "jkf0DDUWAEDJ0ua03gkAQEha/B0DJQBAYHhQfVkxAEBTRrLBeicAQFt4tB6cHQBAuGrNbfIpAEAeM1F38ikAQDZRpdZINgBA+yCj6H1MAEAkSE6PjkcAQOMCT4knQABAEz/3R9RYAEB+q6H0S1sAQBM/90fUWABAXfcx8ed4AECxdwgiZsUAQN+qNbfJpwBAnkozbXJZAUBBPiPxTPwBQN5Stl2l6wJAUkCeI+3fA0CAP8rMZYMFQBc/YCEiVQZAZmZmZmZmBkDYRDnzeWEGQP+9tEURnAVAfM2Gzy5IBUB8aYsiODsEQDi2CcNi/wJA4vtydA01AkAc9tRhvrQBQNuw1VtXrQFA64OMovcxAUDqTeyh3ccAQITAngzoeABAeJak3K89AEC+tEURnB0AQDA8qL6sGABA1thUWe8EAEBrbKqsdwIAQAAAAAAAAABAAAAAAAAAAEBrbKqsdwIAQGtsqqx3AgBAa2yqrHcCAEBrbKqsdwIAQNbYVFnvBABAykg2WO8EAECHEPKsdwIAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -8.278730392456055, -1.5280364751815796 ], "y": [ 2.0, 2.0 ] }, { "fill": "toself", "fillcolor": "#482878", "hovertemplate": "Iteration 3
Mean: 2.94e-05
Max: 3.00e-02
Nonzero: 6752/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#482878", "width": 1 }, "name": "Iteration 3", "opacity": 0.6, "type": "scatter", "x": { "bdata": "jOoTwZ9rEcGy7A7BxG0MwdbuCcHpbwfB/PAEwQ5yAsFB5v/AZuj6wIzq9cCw7PDA1u7rwPvw5sAg8+HARfXcwGr318CP+dLAtPvNwNr9yMD+/8PAJAK/wEgEusBuBrXAkwiwwLgKq8DdDKbAAg+hwCgRnMBME5fAchWSwJYXjcC8GYjA4BuDwAs8fMBVQHLAn0RowOpIXsA1TVTAf1FKwMlVQMATWjbAXV4swKdiIsDxZhjAO2sOwIVvBMCe5/S/NPDgv8f4zL/H+My/NPDgv57n9L+FbwTAO2sOwPFmGMCnYiLAXV4swBNaNsDJVUDAf1FKwDVNVMDqSF7An0RowFVAcsALPHzA4BuDwLwZiMCWF43AchWSwEwTl8AoEZzAAg+hwN0MpsC4CqvAkwiwwG4GtcBIBLrAJAK/wP7/w8Da/cjAtPvNwI/50sBq99fARfXcwCDz4cD78ObA1u7rwLDs8MCM6vXAZuj6wEHm/8AOcgLB/PAEwelvB8HW7gnBxG0MwbLsDsGfaxHBjOoTwQ==", "dtype": "f4" }, "y": { "bdata": "Oca2biALCEAwPWKfsBAIQDkE0mcpJAhAkbcm3hEyCECrUiRMYSEIQIH4BamBLAhAAHw28hEyCEAJWJbDSS8IQMjzmnz6PwhAXsroisJCCECvHh+ak14IQPX8oi1zUwhA2Ei6IXxsCEDIMbZ1A1kIQAtJohVNiAhAHhUaB8+0CEB7goeJPq8IQJKRYa1XkwlAlzaMMy9hCkBUPrPyGGELQCyHbRSVzQtAOuPK4k/pDEDaNgmWQgIOQGZmZmZmZg5AYzU+Frm8DUANQqo9qOMNQNNVmzig2AxAxp/SCQ/TDEAWgbNve3QMQD3Fo6SajQtARlWrzcK/CkDi3JPbIu8JQLN3H5bBzQlAcmEfF/ZxCUCDn1Wn+zEJQKUbwscGsghAqeNF3BpLCEAAfDbyETIIQHVD+NhAFghAlzK6orAQCEBvCB8byAIIQAAAAAAAAAhAbwgfG8gCCEAAAAAAAAAIQAAAAAAAAAhATBldUVgICEAAAAAAAAAIQN0QPjaQBQhAHGNbN5AFCEAqC0kayAIIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEAAAAAAAAAIQAAAAAAAAAhAAAAAAAAACEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -9.244762420654297, -1.6013420820236206 ], "y": [ 3.0, 3.0 ] }, { "fill": "toself", "fillcolor": "#482878", "hovertemplate": "Iteration 4
Mean: 1.30e-05
Max: 1.68e-02
Nonzero: 6658/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#482878", "width": 1 }, "name": "Iteration 4", "opacity": 0.6, "type": "scatter", "x": { "bdata": "JMEpwQ7mJsH3CiTB4S8hwcpUHsG0eRvBnp4YwYfDFcFx6BLBWg0QwUQyDcEuVwrBGHwHwQGhBMHrxQHBqdX9wHwf+MBQafLAIrPswPb85sDKRuHAnJDbwHDa1cBDJNDAFm7KwOq3xMC9Ab/AkEu5wGSVs8A2363ACimowN5yosCwvJzAhAaXwFhQkcAqmovA/uOFwNEtgMBJ73TA74JpwJYWXsA9qlLA4z1HwInRO8AwZTDA1vgkwHyMGcAkIA7AyrMCwN6O7r/eju6/yrMCwCQgDsB8jBnA1vgkwDBlMMCJ0TvA4z1HwD2qUsCWFl7A74JpwEnvdMDRLYDA/uOFwCqai8BYUJHAhAaXwLC8nMDecqLACimowDbfrcBklbPAkEu5wL0Bv8Dqt8TAFm7KwEMk0MBw2tXAnJDbwMpG4cD2/ObAIrPswFBp8sB8H/jAqdX9wOvFAcEBoQTBGHwHwS5XCsFEMg3BWg0QwXHoEsGHwxXBnp4YwbR5G8HKVB7B4S8hwfcKJMEO5ibBJMEpwQ==", "dtype": "f4" }, "y": { "bdata": "BNExyuIEEECMOf3aQQMQQAwOpkxJEBBADam2OuoREEAP9m0PLBUQQBbmKr1QHRBA1ZLs/MwWEEAeI58/tygQQBF66ATNFhBAccJX5zovEEAdiI5RFicQQCdgE8IdNBBAuhtHCcZCEEAnYBPCHTQQQDja+8bqShBAyhG1GPJXEEBJVOTLt2EQQLVED64m7BBA1y/uNCiXEUB4Tiw2gfoRQDDuNCiXIRJAkTx/qpdaEkAzMzMzMzMTQFsjXgqwLBNAQaJijUEUE0B3dcr5rvMSQASLshVttxJA6OMbmtKJEkDQtpSpykMSQOyrnWEbYRJAWiuN9snREUB4+E22W7kRQMmpFNflWhFArZVG++ToEEARtC1lqvIQQDU5OqyMvhBAZi12TX1rEEAqMUWMADkQQGUWzElJEBBAjnoEXKgOEEAGB1OmJAgQQAAAAAAAABBA15vH7aABEEAAAAAAAAAQQAAAAAAAABBAjDn92kEDEEABmxDuoAEQQAI2IdxBAxBAxpx+7aABEECuN4/bQQMQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEAAAAAAAAAQQAAAAAAAABBAAAAAAAAAEEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -10.60965347290039, -1.8637349605560303 ], "y": [ 4.0, 4.0 ] }, { "fill": "toself", "fillcolor": "#482878", "hovertemplate": "Iteration 5
Mean: 4.52e-06
Max: 6.73e-03
Nonzero: 6568/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#482878", "width": 1 }, "name": "Iteration 5", "opacity": 0.6, "type": "scatter", "x": { "bdata": "CBI+wcTuOsGAyzfBOqg0wfaEMcGyYS7BbD4rwSgbKMHj9yTBntQhwVqxHsEUjhvB0GoYwYxHFcFGJBLBAgEPwb7dC8F4ugjBNJcFwfBzAsFVof7AzFr4wEIU8sC5zevAMIflwKZA38Ac+tjAlLPSwAptzMCAJsbA99+/wG6ZucDkUrPAWwytwNLFpsBIf6DAvjiawDbyk8Csq43AImWHwJkegcAesHXADCNpwPqVXMDmCFDA1HtDwMLuNsCuYSrAnNQdwIpHEcCKRxHAnNQdwK5hKsDC7jbA1HtDwOYIUMD6lVzADCNpwB6wdcCZHoHAImWHwKyrjcA28pPAvjiawEh/oMDSxabAWwytwORSs8BumbnA99+/wIAmxsAKbczAlLPSwBz62MCmQN/AMIflwLnN68BCFPLAzFr4wFWh/sDwcwLBNJcFwXi6CMG+3QvBAgEPwUYkEsGMRxXB0GoYwRSOG8FasR7BntQhweP3JMEoGyjBbD4rwbJhLsH2hDHBOqg0wYDLN8HE7jrBCBI+wQ==", "dtype": "f4" }, "y": { "bdata": "ViCMtGMFFECwnvNTkwwUQKxAGGnHChRAH8F9BPcRFEBGS3vAWhcUQMwBlm2+HBRAIiIiIiIiFECvobyG8hoUQL96zk9NMhRAXOLU77klFEDRw96oFD0UQJgDLNt8ORRAjZb2gLUuFEBeQ3kN5TUUQIBlmy8HWBRAR4b82vlyFEDYUF5DeQ0VQEN5DeU1lBVAROc4T23zFUC0YwXCSDsWQHujUvSwNxZAGQE793+aFkBQqoq4gvsWQDMzMzMzMxdAQQFo0bb5FkDKayivobwWQKlmDcq/5xZAd7j44+ksFkBKBo1/mmQWQJLQ8Fj3ERZAIiIiIiIiFkAGpWvfQeMVQIXou/PNlxVAcrUuPCFXFUDCE+e4FQgVQEUPe6NS9BRAxw4ilEKuFEBW+5SPmmQUQAgj7ViBMBRAifE3ICsQFEDmAMs2Xw4UQFYgjLRjBRRAAAAAAAAAFEAAAAAAAAAUQB1g2ebLARRAGfJr58sBFEAdYNnmywEUQDnAss2XAxRAAAAAAAAAFEBWIIy0YwUUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEAAAAAAAAAUQAAAAAAAABRAAAAAAAAAFEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -11.879402160644531, -2.269991397857666 ], "y": [ 5.0, 5.0 ] }, { "fill": "toself", "fillcolor": "#3e4989", "hovertemplate": "Iteration 6
Mean: 1.28e-06
Max: 2.03e-03
Nonzero: 6490/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#3e4989", "width": 1 }, "name": "Iteration 6", "opacity": 0.6, "type": "scatter", "x": { "bdata": "bKhTwZ1AUMHO2EzBAHFJwTIJRsFjoULBlDk/wcbRO8H3aTjBKAI1wVqaMcGMMi7Bvcoqwe5iJ8Eg+yPBUZMgwYIrHcG0wxnB5VsWwRf0EsFIjA/BeiQMwau8CMHcVAXBDu0BwX4K/cDiOvbARGvvwKeb6MAKzOHAbPzawNAs1MAyXc3Alo3GwPi9v8Bb7rjAvh6ywCBPq8CDf6TA5q+dwEnglsCsEJDAD0GJwHJxgsCqQ3fAbqRpwDQFXMD6ZU7AwMZAwIYnM8CGJzPAwMZAwPplTsA0BVzAbqRpwKpDd8BycYLAD0GJwKwQkMBJ4JbA5q+dwIN/pMAgT6vAvh6ywFvuuMD4vb/Alo3GwDJdzcDQLNTAbPzawArM4cCnm+jARGvvwOI69sB+Cv3ADu0BwdxUBcGrvAjBeiQMwUiMD8EX9BLB5VsWwbTDGcGCKx3BUZMgwSD7I8HuYifBvcoqwYwyLsFamjHBKAI1wfdpOMHG0TvBlDk/wWOhQsEyCUbBAHFJwc7YTMGdQFDBbKhTwQ==", "dtype": "f4" }, "y": { "bdata": "VrLtHfUDGEAYLNHs2Q0YQKx7pJXJExhArsRcfNQPGECCuXZgrh0YQFyJufioHxhALAeJQrkZGEAEc+3AXDsYQAlOFnV9LxhAg9A/uo0pGEAJThZ1fS8YQC9jrak2SRhACU4WdX0vGEAEc+3AXDsYQF6vkfmNhhhAM636AzIFGUBw7jS+Xa8ZQMtBolBuBhpA/PKAl14MGkDfNguopU8aQM2G/V0MKhpALXvqZDjUGkDFOAcrixMbQDMzMzMzMxtArzRhxebxGkCHa8Mhm7AaQAHst3NZyBpAVn1zHYpZGkCjxM8o8TMaQKsNxdlYDhpAq7Zpo7VJGkDWf/zufgAaQKsPmbQcxxlA1i2zW8WJGUCA9pCwXVIZQBERERERERlA1jLF/q7XGEArFzQp+7sYQLDukVYmTxhAsuvnNpMnGEAsB4lCuRkYQFdiLj7qBxhAK9n2jvoBGEAAAAAAAAAYQCvZ9o76ARhAK9n2jvoBGECWmIuP+gEYQFay7R31AxhAAAAAAAAAGED4GlSt7wUYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEAAAAAAAAAYQAAAAAAAABhAAAAAAAAAGEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -13.228618621826172, -2.7992873191833496 ], "y": [ 6.0, 6.0 ] }, { "fill": "toself", "fillcolor": "#3e4989", "hovertemplate": "Iteration 7
Mean: 3.04e-07
Max: 4.97e-04
Nonzero: 6414/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#3e4989", "width": 1 }, "name": "Iteration 7", "opacity": 0.6, "type": "scatter", "x": { "bdata": "sxNpwa1vZcGmy2HBoCdewZqDWsGU31bBjjtTwYeXT8GB80vBe09IwXWrRMFvB0HBaGM9wWK/OcFcGzbBVncywU/TLsFJLyvBQ4snwT3nI8E3QyDBMJ8cwSr7GMEkVxXBHrMRwRcPDsERawrBC8cGwQUjA8H+/f7A8LX3wORt8MDYJenAzN3hwL+V2sCyTdPApgXMwJq9xMCNdb3AgC22wHTlrsBonafAXFWgwE8NmcBCxZHANn2KwCo1g8A82nfAIkppwAi6WsAIulrAIkppwDzad8AqNYPANn2KwELFkcBPDZnAXFWgwGidp8B05a7AgC22wI11vcCavcTApgXMwLJN08C/ldrAzN3hwNgl6cDkbfDA8LX3wP79/sAFIwPBC8cGwRFrCsEXDw7BHrMRwSRXFcEq+xjBMJ8cwTdDIME95yPBQ4snwUkvK8FP0y7BVncywVwbNsFivznBaGM9wW8HQcF1q0TBe09IwYHzS8GHl0/BjjtTwZTfVsGag1rBoCdewabLYcGtb2XBsxNpwQ==", "dtype": "f4" }, "y": { "bdata": "NrDqoXkMHECvJYe4yhQcQHr8+jzzGBxARPMYoFgjHEAOQy7+3hYcQByGXPy9LRxAOdEkxxsdHEABLmcrgSccQNjAqofmMRxAAS5nK4EnHEBfSw5xlSkcQJX7+BIPNhxA8UudoohCHEDMB7ILi5EcQDogJP2CSB1As9KLQfnXHUA82MCqh+YdQBRrBAft8B1AFGsEB+3wHUC2e8TsWRYeQEzTi1Zrmx5A4PMfl0myHkAzMzMzMzMfQPwmejq31x5AT+kQTiP9HkAkOGiHT34eQO7jSzzYwB5AB1Y9NI8BHkDsK6+O0yIeQL6q+WJm/R1AWDzurmhMHkDDkIu/twUeQAb6bt2Msh1ArshOZrqCHUDB2O4Rs2cdQMAn4gRnKx1A9v6jMan9HEAQ+8qr47QcQABXeRROixxAoz4nEe5MHEDl1XFaRCEcQPPqOC2iEBxAG1j10DwGHEAAAAAAAAAcQF4dp0UUAhxAXh2nRRQCHEAbWPXQPAYcQAAAAAAAABxAAAAAAAAAHEDFsYfQPAYcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEAAAAAAAAAcQAAAAAAAABxAAAAAAAAAHEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -14.567309379577637, -3.417604446411133 ], "y": [ 7.0, 7.0 ] }, { "fill": "toself", "fillcolor": "#3e4989", "hovertemplate": "Iteration 8
Mean: 6.35e-08
Max: 1.23e-04
Nonzero: 6318/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#3e4989", "width": 1 }, "name": "Iteration 8", "opacity": 0.6, "type": "scatter", "x": { "bdata": "4xqAwThMfMGsYnjBHnl0wZKPcMEEpmzBeLxowerSZMFd6WDB0P9cwUMWWcG2LFXBKUNRwZxZTcEOcEnBgoZFwfScQcFosz3B2sk5wU7gNcHA9jHBNA0uwaYjKsEaOibBjFAiwQBnHsFyfRrB5ZMWwViqEsHLwA7BPtcKwbHtBsEkBAPBLTX+wBNi9sD5ju7A37vmwMXo3sCrFdfAkULPwHdvx8BdnL/AQ8m3wCn2r8API6jA9U+gwNt8mMDBqZDAp9aIwI0DgcCNA4HAp9aIwMGpkMDbfJjA9U+gwA8jqMAp9q/AQ8m3wF2cv8B3b8fAkULPwKsV18DF6N7A37vmwPmO7sATYvbALTX+wCQEA8Gx7QbBPtcKwcvADsFYqhLB5ZMWwXJ9GsEAZx7BjFAiwRo6JsGmIyrBNA0uwcD2McFO4DXB2sk5wWizPcH0nEHBgoZFwQ5wScGcWU3BKUNRwbYsVcFDFlnB0P9cwV3pYMHq0mTBeLxowQSmbMGSj3DBHnl0waxieME4THzB4xqAwQ==", "dtype": "f4" }, "y": { "bdata": "rdp4VTgCIEANINUAUg0gQArYn4D9CSBAXf0mK8UHIEBgRVyrGQsgQBSwPwH7EyBAZ9XGq8IRIEBkjZErbg4gQOa5+dI1DCBAhEpqh08XIEDgPmfQhxkgQEuFz+OCLSBALM/Nl65hIEDLKLKMIssgQC+2Q7iQ2SBAVxbDuoYBIUBQhli63fogQILbymJY1yBAoGOq5FD1IECkjow/Hk4hQDlxkBMHOSFAmpmZmZmZIUBh0Q8W/WAhQDyccm7UkSFAxV6hQWtvIUDt23Np6EEhQO3bc2noQSFAufl1IaAMIUCuM5HRagAhQGBSFHWBFSFAjPDN1bwNIUDoCvB4hgEhQDZGrrg54CBAxJhHjHnEIEC6wKcLfLogQE1bdl8QoiBA1R2lMqd/IEBhKAmGkmAgQOaiAtnUOiBAkn17Lg09IEDLYljXMCAgQArYn4D9CSBAWrXxqnAEIECt2nhVOAIgQAAAAAAAACBAVm28KhwBIEADSDWAVAMgQAAAAAAAACBAAAAAAAAAIEBIzv5/VAMgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEAAAAAAAAAgQAAAAAAAACBAAAAAAAAAIEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -16.01312828063965, -4.031683444976807 ], "y": [ 8.0, 8.0 ] }, { "fill": "toself", "fillcolor": "#31688e", "hovertemplate": "Iteration 9
Mean: 1.21e-08
Max: 2.77e-05
Nonzero: 6228/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#31688e", "width": 1 }, "name": "Iteration 9", "opacity": 0.6, "type": "scatter", "x": { "bdata": "p4OIwTt+hsHPeITBZHOCwfhtgMEY0XzBQMZ4wWi7dMGQsHDBuKVsweCaaMEJkGTBMYVgwVp6XMGCb1jBqmRUwdJZUMH6TkzBIkRIwUs5RMFzLkDBnCM8wcQYOMHsDTTBFAMwwTz4K8Fk7SfBjeIjwbXXH8HezBvBBsIXwS63E8FWrA/BfqELwaaWB8HPiwPB7gH/wD/s9sCQ1u7A4MDmwDCr3sCAldbA0X/OwCJqxsByVL7Awz62wBQprsBkE6bAtP2dwATolcAE6JXAtP2dwGQTpsAUKa7Awz62wHJUvsAiasbA0X/OwICV1sAwq97A4MDmwJDW7sA/7PbA7gH/wM+LA8GmlgfBfqELwVasD8EutxPBBsIXwd7MG8G11x/BjeIjwWTtJ8E8+CvBFAMwwewNNMHEGDjBnCM8wXMuQMFLOUTBIkRIwfpOTMHSWVDBqmRUwYJvWMFaelzBMYVgwQmQZMHgmmjBuKVswZCwcMFou3TBQMZ4wRjRfMH4bYDBZHOCwc94hME7fobBp4OIwQ==", "dtype": "f4" }, "y": { "bdata": "dqsQXrcKIkCDvqAv6AsiQFyF8LpVCCJAaHDCGvQFIkBSBwZD3BEiQMYiSpOfFiJAdqsQXrcKIkD5abGNnxYiQLB0kgNLJyJAj7gj7og7IkBjDAg+XrciQNNJDiyd5CJANmVNWVPWIkCe+QomDPoiQO1vLs/+5iJA7og74o64IkDGNn5abOMiQNAX9AV9QSNAqvBWalg6I0BFqvdQpHojQKJI9R6KVCNAXUWja/qCI0CamZmZmZkjQB+KVO+hSCNAgqWTHFg6I0B8KFK9hyIjQI7InVj0BSNAGiYgo2ECI0DsViG8bhUjQNU5MM6GCSNAfUFf0Bf0IkCkeg9FqvciQPuby7O/uSJA+5vLs7+5IkB7h+MPj7giQEWq91CkeiJAWVPWlDVlIkC6nroFu1UiQFps46fFNiJAnMuzv7k8IkBvFcLrViEiQJDRMAEZDSJAp10AvVUIIkAaJiCjYQIiQA0TkNEwASJASHGSdZIDIkAAAAAAAAAiQAAAAAAAACJADROQ0TABIkAcjmujYQIiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkAAAAAAAAAiQAAAAAAAACJAAAAAAAAAIkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -17.06428337097168, -4.684572219848633 ], "y": [ 9.0, 9.0 ] }, { "fill": "toself", "fillcolor": "#31688e", "hovertemplate": "Iteration 10
Mean: 2.19e-09
Max: 5.72e-06
Nonzero: 6158/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#31688e", "width": 1 }, "name": "Iteration 10", "opacity": 0.6, "type": "scatter", "x": { "bdata": "bCeVwfT8ksF+0pDBBqiOwY59jMEYU4rBoCiIwSr+hcGy04PBPKmBwYj9fsGaqHrBrFN2wb7+ccHPqW3B4VRpwfP/ZMEFq2DBFlZcwSgBWME6rFPBTFdPwV0CS8FvrUbBgVhCwZMDPsGkrjnBtlk1wcgEMcHaryzB61oowf0FJMEPsR/BIVwbwTIHF8FEshLBVl0OwWgICsF5swXBi14BwToT+sBeafHAgb/owKQV4MDIa9fA68HOwA4YxsAybr3AVsS0wHoarMB6GqzAVsS0wDJuvcAOGMbA68HOwMhr18CkFeDAgb/owF5p8cA6E/rAi14BwXmzBcFoCArBVl0OwUSyEsEyBxfBIVwbwQ+xH8H9BSTB61oowdqvLMHIBDHBtlk1waSuOcGTAz7BgVhCwW+tRsFdAkvBTFdPwTqsU8EoAVjBFlZcwQWrYMHz/2TB4VRpwc+pbcG+/nHBrFN2wZqoesGI/X7BPKmBwbLTg8Eq/oXBoCiIwRhTisGOfYzBBqiOwX7SkMH0/JLBbCeVwQ==", "dtype": "f4" }, "y": { "bdata": "KZdi0A0IJEBC/mq5wAUkQFtlc6JzAyRAh9a2QecGJEA6x7y3aBIkQJUsMFrcFSRAplyKQTcgJEDG+kAsQhEkQHVQd1N8SCRA6yTqbk7YJEBt/5RBpsskQH/NHgHVpiRAgFEaySruJEDtNHRuQNAkQFU2vDDHniRAmPYkzjLIJEAETvA7rPkkQB459Oy0PiVANAmw5ptFJUBxZ7mN0E4lQIkMxJLYKCVAmpmZmZmZJUD+mj0Cqk0lQLva1GQ+JCVARw5Z2RcjJUBfs2PeH/0kQI49uZaC4SRADUecIa8QJUAdtfgkX/ckQE9D5wYE7SRA9B2WVFHvJEB/jxzlf8okQOIrPolLwSRAwcuJupWsJEBqU5va1KYkQG0ZyVEkVSRAfkkjOX9fJEBEvYO3VUckQOAjR/mfMiRAU/DChMYzJED5ik/iUjAkQCFgtM61FCRAoZ2Gzg0IJEDPMe8tmgQkQM8x7y2aBCRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAdMx7iyYBJEDnmPcWTQIkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEAAAAAAAAAkQAAAAAAAACRAAAAAAAAAJEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -18.644248962402344, -5.378232002258301 ], "y": [ 10.0, 10.0 ] }, { "fill": "toself", "fillcolor": "#31688e", "hovertemplate": "Iteration 11
Mean: 3.77e-10
Max: 1.09e-06
Nonzero: 6094/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#31688e", "width": 1 }, "name": "Iteration 11", "opacity": 0.6, "type": "scatter", "x": { "bdata": "cvCgwd+mnsFMXZzBuBOawSTKl8GRgJXB/jaTwWrtkMHWo47BRFqMwbAQisEcx4fBiH2FwfYzg8Fi6oDBnUF9wXaueMFPG3TBKIhvwQL1asHaYWbBtM5hwYw7XcFmqFjBPxVUwRiCT8Hx7krByltGwaTIQcF8NT3BVqI4wS4PNMEIfC/B4OgqwbpVJsGSwiHBbC8dwUWcGMEeCRTB+HUPwdDiCsGqTwbBgrwBwbdS+sBpLPHAGwbowM3f3sB/udXAMpPMwORsw8DkbMPAMpPMwH+51cDN397AGwbowGks8cC3UvrAgrwBwapPBsHQ4grB+HUPwR4JFMFFnBjBbC8dwZLCIcG6VSbB4OgqwQh8L8EuDzTBVqI4wXw1PcGkyEHByltGwfHuSsEYgk/BPxVUwWaoWMGMO13BtM5hwdphZsEC9WrBKIhvwU8bdMF2rnjBnUF9wWLqgMH2M4PBiH2FwRzHh8GwEIrBRFqMwdajjsFq7ZDB/jaTwZGAlcEkypfBuBOawUxdnMHfpp7BcvCgwQ==", "dtype": "f4" }, "y": { "bdata": "Ak4DgZoDJkBOBoD+mwkmQFxZYAMDDCZAMA6g/J4VJkAKhhCFBBImQPkOcHzSFiZAc7OEjqYzJkBKjKDeztUmQM4bFrs00iZAo7XQJYiGJkDFhvBfZs0mQMegwNkJ9SZA+3IqJ1aLJkCx1+eswZ8mQN2NQN412CZA8PtKy9wLJ0DqzEDPTTgnQLkeJtq2QCdANnFeE+c1J0ApCYuSsCgnQJqZmZmZmSdARBFFFFFEJ0AsscQSSywnQCkJi5KwKCdAiRzaQtXtJkCCpaDYPvwmQOfDPcdy/SZA/2DqD6b+JkD9KNePcv0mQOlwkA4H6SZA29CpDZ3aJkDF4E8M/sQmQLUIVotgtSZAuHh8i8e3JkCEEEIIIYQmQFugtAVKWyZAUKgHhXpQJkAb9iqN2C4mQDDAAAMMMCZAKXCNAtcoJkAyMCcDczImQCNYLYLVIiZADWjTgDYNJkAKwJkAnAkmQAE4E4AzASZAAAAAAAAAJkAAAAAAAAAmQAE4E4AzASZAAAAAAAAAJkDEOOP/ZgImQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkAAAAAAAAAmQAAAAAAAACZAAAAAAAAAJkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -20.11740493774414, -6.10704231262207 ], "y": [ 11.0, 11.0 ] }, { "fill": "toself", "fillcolor": "#26828e", "hovertemplate": "Iteration 12
Mean: 6.05e-11
Max: 1.87e-07
Nonzero: 6017/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#26828e", "width": 1 }, "name": "Iteration 12", "opacity": 0.6, "type": "scatter", "x": { "bdata": "5IutwbQgq8GFtajBVkqmwSbfo8H2c6HBxwifwZidnMFoMprBOMeXwQhclcHZ8JLBqoWQwXoajsFKr4vBGkSJwezYhsG8bYTBjAKCwbkuf8FaWHrB+oF1wZyrcME81WvB3v5mwX4oYsEgUl3BwHtYwWClU8ECz07BovhJwUQiRcHkS0DBhnU7wSafNsHIyDHBaPIswQgcKMGqRSPBSm8eweyYGcGMwhTBLuwPwc4VC8FuPwbBEGkBwWEl+cCjeO/A5cvlwCgf3MAoH9zA5cvlwKN478BhJfnAEGkBwW4/BsHOFQvBLuwPwYzCFMHsmBnBSm8ewapFI8EIHCjBaPIswcjIMcEmnzbBhnU7weRLQMFEIkXBovhJwQLPTsFgpVPBwHtYwSBSXcF+KGLB3v5mwTzVa8Gcq3DB+oF1wVpYesG5Ln/BjAKCwbxthMHs2IbBGkSJwUqvi8F6Go7BqoWQwdnwksEIXJXBOMeXwWgymsGYnZzBxwifwfZzocEm36PBVkqmwYW1qMG0IKvB5IutwQ==", "dtype": "f4" }, "y": { "bdata": "Bpsx9ecNKEC6NT8sUgYoQHiofG3ZCChAcZZK4rIRKEAMNmPqzxsoQM3MzMzMzChAR2bhOEriKECWByTrXHkoQGs9kr1hkShAg8vmTGwEKUDj9c8bhLMoQBiOVI8KcyhAkKxz5QHJKECejsKhHtMoQHQc4CcfFilA7mhyX4U5KUD3EtpLaC8pQA1F+Jn9NilAdBzgJx8WKUCamZmZmZkpQEQPzLg3SylATJMAQNInKUCXYnJ8QTgpQIC6GqS58ihA/Dt9C3sJKUCxbAvPC/koQMuJjOAoAylA+3NjukUNKUAfVjlOFeYoQLFsC88L+ShAMGOXb2LUKECeTMXua8EoQD3o45jHtChAgy9E3U63KEBN4ln7wVkoQE3iWfvBWShAM1Ew5LxBKECxu2SK+yooQA6YNOi3KShADpg06LcpKEA/SiUTCjAoQFXflCw/LChAkhbSgPYSKEAb5WkPHQooQEdHYESHAihAAAAAAAAAKECjIzCiQwEoQAAAAAAAAChAAAAAAAAAKEBbLqNEhwIoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEAAAAAAAAAoQAAAAAAAAChAAAAAAAAAKEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -21.69330596923828, -6.878803253173828 ], "y": [ 12.0, 12.0 ] }, { "fill": "toself", "fillcolor": "#26828e", "hovertemplate": "Iteration 13
Mean: 8.90e-12
Max: 2.88e-08
Nonzero: 5951/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#26828e", "width": 1 }, "name": "Iteration 13", "opacity": 0.6, "type": "scatter", "x": { "bdata": "WVq4wefYtcF1V7PBA9awwZFUrsEf06vBrVGpwTrQpsHITqTBVs2hweRLn8FyypzBAEmawY7Hl8EcRpXBqsSSwThDkMHGwY3BU0CLweG+iMFvPYbB/buDwYs6gcEycn3BTm94wWpsc8GGaW7BomZpwb5jZMHZYF/B9V1awRFbVcEtWFDBSVVLwWVSRsGAT0HBnEw8wbhJN8HURjLB8EMtwQxBKMEoPiPBQzsewV84GcF7NRTBlzIPwbMvCsHOLAXB6ikAwQxO9sAMTvbA6ikAwc4sBcGzLwrBlzIPwXs1FMFfOBnBQzsewSg+I8EMQSjB8EMtwdRGMsG4STfBnEw8wYBPQcFlUkbBSVVLwS1YUMERW1XB9V1awdlgX8G+Y2TBomZpwYZpbsFqbHPBTm94wTJyfcGLOoHB/buDwW89hsHhvojBU0CLwcbBjcE4Q5DBqsSSwRxGlcGOx5fBAEmawXLKnMHkS5/BVs2hwchOpME60KbBrVGpwR/Tq8GRVK7BA9awwXVXs8Hn2LXBWVq4wQ==", "dtype": "f4" }, "y": { "bdata": "wO/E4ZQMKkBHQ/fB+g0qQEOAMlf1pCpAT6W+JWgMK0D8PDuV+pYqQIxFAgtJTipAyZJ4vq7YKkCTYYpZmRArQJmrelJogypAyxRRsAd0KkCCT4H8s8oqQK1Erz0X0ypAHTzox8gbK0Ajb6RrWDYrQLI2f2j6HytAfoy8ka5hK0Ar40yIlB4rQJ7ZHOfkVytAmpmZmZmZK0DA3eMoxiIrQHhZAO4eRytAqsLWS743K0CPtflD0/8qQMhRjEUCCytAXUwj5jMPK0Dkn1XGmRArQF1MI+YzDytA+rpio6H7KkAZfG5QNggrQCwkOQFB7CpAWFpTu1DCKkCenUp9S9AqQHSoHDzoxypADCWMznJnKkAMJYzOcmcqQCOLzJtLRypA8RevxocvKkA/z06lviUqQI6G7oP1GypA8RevxocvKkBqxHzmIS4qQBk+dZwcPCpAY5HAQpITKkCySGAhyQkqQA6nZMDLAipAh1My4GUBKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAOp2TAywIqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkAAAAAAAAAqQAAAAAAAACpAAAAAAAAAKkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -23.04411506652832, -7.697027206420898 ], "y": [ 13.0, 13.0 ] }, { "fill": "toself", "fillcolor": "#26828e", "hovertemplate": "Iteration 14
Mean: 1.17e-12
Max: 3.90e-09
Nonzero: 5918/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#26828e", "width": 1 }, "name": "Iteration 14", "opacity": 0.6, "type": "scatter", "x": { "bdata": "Il/LwT6fyMFa38XBdx/DwZRfwMGwn73BzN+6weofuMEGYLXBIqCywUDgr8FcIK3BeGCqwZWgp8Gy4KTBziCiwepgn8EIoZzBJOGZwUAhl8FeYZTBeqGRwZbhjsGzIYzB0GGJweyhhsEI4oPBJSKBwYTEfMG9RHfB9sRxwTBFbMFoxWbBokVhwdvFW8EURlbBTsZQwYZGS8HAxkXB+UZAwTLHOsFsRzXBpMcvwd5HKsEWyCTBUEgfwYrIGcHCSBTB+8gOwTVJCcE1SQnB+8gOwcJIFMGKyBnBUEgfwRbIJMHeRyrBpMcvwWxHNcEyxzrB+UZAwcDGRcGGRkvBTsZQwRRGVsHbxVvBokVhwWjFZsEwRWzB9sRxwb1Ed8GExHzBJSKBwQjig8HsoYbB0GGJwbMhjMGW4Y7BeqGRwV5hlMFAIZfBJOGZwQihnMHqYJ/BziCiwbLgpMGVoKfBeGCqwVwgrcFA4K/BIqCywQZgtcHqH7jBzN+6wbCfvcGUX8DBdx/DwVrfxcE+n8jBIl/LwQ==", "dtype": "f4" }, "y": { "bdata": "GMyJINoDLEBtew1EiB0sQPKECnJxzSxAPNcZLUXjLEAqmuSh4VksQMaDsSqCLyxAQS0riUjaLEDffWy39vMsQOcthnTeYixAIlZh7JhYLECDzNyEd7ssQAKcXE16sixA/415jRn5LEDt52H6OActQLDlWHZC7CxAk54sHoxVLUBuRqd6EwstQOfB72w/9SxAmpmZmZmZLUAsp/V+5CktQOfB72w/9SxAN3gJ/7FRLUA+rpM5XwMtQCuCLwiXySxAH56GYzz+LEBQSoOK0+4sQJWaNwZ/ES1AVl3T4LDpLECHOEs1PP4sQHHFPsr84SxA2S0NGMC8LEC/nQENvcUsQCawxWJ3uyxARbPaBUdyLEDyvU2r5FAsQGBaG/xyXCxAMcRCaWIhLEAXNDdeXyosQOVYvwnUFSxA1izYJKsiLEDLyLP+0B4sQJWRZ/2hPSxASj0ZiTwlLECyZhLKHw4sQGb7jmqRAixAEIgGa5ECLEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAQiAZrkQIsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEAAAAAAAAAsQAAAAAAAACxAAAAAAAAALEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -25.421451568603516, -8.58037281036377 ], "y": [ 14.0, 14.0 ] }, { "fill": "toself", "fillcolor": "#1f9e89", "hovertemplate": "Iteration 15
Mean: 1.35e-13
Max: 4.60e-10
Nonzero: 5865/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#1f9e89", "width": 1 }, "name": "Iteration 15", "opacity": 0.6, "type": "scatter", "x": { "bdata": "bjXdwbQ/2sH7SdfBQVTUwYde0cHOaM7BFHPLwVp9yMGgh8XB55HCwS2cv8FzprzBurC5wQC7tsFGxbPBjM+wwdPZrcEZ5KrBX+6nwaX4pMHsAqLBMg2fwXgXnMG/IZnBBSyWwUs2k8GRQJDB2EqNwR5VisFkX4fBq2mEwfFzgcFu/HzB+xB3wYglccEUOmvBoE5lwS1jX8G6d1nBRoxTwdKgTcFgtUfB7MlBwXjeO8EF8zXBkgcwwR4cKsGrMCTBOEUewcRZGMHEWRjBOEUewaswJMEeHCrBkgcwwQXzNcF43jvB7MlBwWC1R8HSoE3BRoxTwbp3WcEtY1/BoE5lwRQ6a8GIJXHB+xB3wW78fMHxc4HBq2mEwWRfh8EeVYrB2EqNwZFAkMFLNpPBBSyWwb8hmcF4F5zBMg2fwewCosGl+KTBX+6nwRnkqsHT2a3BjM+wwUbFs8EAu7bBurC5wXOmvMEtnL/B55HCwaCHxcFafcjBFHPLwc5ozsGHXtHBQVTUwftJ18G0P9rBbjXdwQ==", "dtype": "f4" }, "y": { "bdata": "urbkMp5ULkD/peMyufcuQDbbJ6oHjS5AbTSHrW4dLkB0LGUS7CouQDLIExaAtS5AXcaTihT/LkDOi2h5fE4uQLao8W7dOi5AmxfR8vicLkDmhhiQjqUuQNKxlEmwqy5AytSH8o3mLkBiq1sH+N0uQB808s7f4i5AECNOWQVQL0BuiAHpWMouQF3Gk4oU/y5AmpmZmZmZL0CEcJsX0fIuQLOfqUQy3y5AeqYLHgo1L0AW28HgFP8uQKMPr1d2qi5AJizQM13wLkAmLNAzXfAuQMjnBFDp7S5AugZJKabhLkC7Cl9uiAEvQAnpPNu00S5ATMPB2H+1LkDtfvb0C7MuQH5q5Txery5AVo1R0kZzLkAQXiXMWkguQHc/e/qFWS5A/wPQiBwhLkCIAelYyiQuQAmtqiGfEy5AgQkL9EwXLkCgWQVtkCMuQPVa9e+ZLi5AMbX72dMvLkAH+N1kfQ0uQKJPzYQJCy5AL6Ll8TkBLkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkBeRMvjcwIuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkAAAAAAAAAuQAAAAAAAAC5AAAAAAAAALkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -27.65108871459961, -9.521915435791016 ], "y": [ 15.0, 15.0 ] }, { "fill": "toself", "fillcolor": "#1f9e89", "hovertemplate": "Iteration 16
Mean: 1.35e-14
Max: 4.68e-11
Nonzero: 5272/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#1f9e89", "width": 1 }, "name": "Iteration 16", "opacity": 0.6, "type": "scatter", "x": { "bdata": "7jHmwdA248GzO+DBlUDdwXhF2sFaStfBPE/UwR5U0cEBWc7B413LwcZiyMGoZ8XBimzCwW1xv8FPdrzBMXu5wRSAtsH2hLPB2ImwwbuOrcGdk6rBf5inwWKdpMFEoqHBJqeewQmsm8HrsJjBzbWVwbC6ksGSv4/BdcSMwVfJicE5zobBG9ODwf7XgMHAuXvBhcN1wUrNb8EO12nB1OBjwZjqXcFc9FfBIv5RweYHTMGqEUbBbxtAwTQlOsH4LjTBvTguwYJCKMGCQijBvTguwfguNME0JTrBbxtAwaoRRsHmB0zBIv5RwVz0V8GY6l3B1OBjwQ7XacFKzW/BhcN1wcC5e8H+14DBG9ODwTnOhsFXyYnBdcSMwZK/j8GwupLBzbWVweuwmMEJrJvBJqeewUSiocFinaTBf5inwZ2TqsG7jq3B2ImwwfaEs8EUgLbBMXu5wU92vMFtcb/BimzCwahnxcHGYsjB413LwQFZzsEeVNHBPE/UwVpK18F4RdrBlUDdwbM74MHQNuPB7jHmwQ==", "dtype": "f4" }, "y": { "bdata": "s/3Ys8EUMEBaN5gin3UwQC13ajGRTzBAlzA6MoYXMEC8OA8dwwswQDPGKZ2AOzBAQvVJR91gMEA+6wZT5DMwQCgStfgcQzBAvljl9yd7MECe/ogF3GkwQLnzL7+zbjBAQGIBd/qjMEBAC0AUYoEwQOsouIm2XDBACIf43W+nMEDNzMzMzMwwQEda/39RbTBAWjeYIp91MEAuCAwED50wQDVXy2/+iDBALXdqMZFPMEDVlqukUW0wQM5xD9X/fzBApz9iAXd6MEBnjFM6AXcwQJfX0dGIhTBArj+7GlB2MEDgTn/EAm4wQMCd/ogFXDBA8I1twippMECEQ/zut1MwQHp5kX3POjBAca8mDOchMEBfE4Ky9zUwQNM/MliuEjBAQGSodF8TMEBeHIh8TQgwQLsePpz+CDBAGDvGGEwRMEBcMUf2mhAwQLxS4lU3GDBAdldOlZkZMECumCN7TQgwQOUyYX7EAjBAC7jTH7EAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEBymTA/YgEwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEAAAAAAAAAwQAAAAAAAADBAAAAAAAAAMEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -28.77437973022461, -10.516237258911133 ], "y": [ 16.0, 16.0 ] }, { "fill": "toself", "fillcolor": "#1f9e89", "hovertemplate": "Iteration 17
Mean: 1.15e-15
Max: 4.06e-12
Nonzero: 4777/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#1f9e89", "width": 1 }, "name": "Iteration 17", "opacity": 0.6, "type": "scatter", "x": { "bdata": "jD7vwY9A7MGSQunBlkTmwZpG48GcSODBoErdwaRM2sGnTtfBqlDUwa5S0cGyVM7BtFbLwbhYyMG8WsXBvlzCwcJev8HGYLzByWK5wcxktsHQZrPB1GiwwdZqrcHabKrB3m6nweBwpMHkcqHB6HSewex2m8HueJjB8nqVwfZ8ksH4fo/B/ICMwQCDicEChYbBBoeDwQqJgMEaFnvBIRp1wSgeb8EuImnBNSZjwTwqXcFDLlfBSjJRwVE2S8FYOkXBXz4/wWVCOcFlQjnBXz4/wVg6RcFRNkvBSjJRwUMuV8E8Kl3BNSZjwS4iacEoHm/BIRp1wRoWe8EKiYDBBoeDwQKFhsEAg4nB/ICMwfh+j8H2fJLB8nqVwe54mMHsdpvB6HSeweRyocHgcKTB3m6nwdpsqsHWaq3B1GiwwdBms8HMZLbByWK5wcZgvMHCXr/BvlzCwbxaxcG4WMjBtFbLwbJUzsGuUtHBqlDUwadO18GkTNrBoErdwZxI4MGaRuPBlkTmwZJC6cGPQOzBjD7vwQ==", "dtype": "f4" }, "y": { "bdata": "cQN/q1UUMUCB3R5rGToxQKzLJRj+RDFAJB7tM7EYMUAphT6BgB4xQHaQTdlQYTFAd/mR7i98MUDvSviXokAxQH3pvTVsVjFAZdtFcfOhMUASTmle/IkxQOJNrpgQUjFAUlEtRGZmMUDNzMzMzMwxQIkaZgbenDFAtRCyrsRiMUDvL3wqS3ExQIUb+FutojFAYnIFMbmCMUCwypPCLj8xQGhiMXj1XDFAYJUnhV1+MUDrkjov128xQCjRwC0CejFAaZMD2r5yMUA0WCubEY8xQKP/QI07XDFAK8lWUSBnMUAY0DYZI18xQPCRqNkHajFAHDL22uVHMUA7yKZsqDAxQETGgsEJJTFA5Ix86b01MUCQQzGWDxUxQBJDaOsnEjFAdT5UGP0HMUCIIqHscAkxQM8WFa2eCzFAF0LWlVgMMUA3op8DbhExQJln48VSHDFAz8OVa4MWMUB1PlQY/QcxQD+AZNVzATFAxWLw6rkAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUA/gGTVcwExQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUAAAAAAAAAxQAAAAAAAADFAAAAAAAAAMUA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -29.905540466308594, -11.578709602355957 ], "y": [ 17.0, 17.0 ] }, { "fill": "toself", "fillcolor": "#35b779", "hovertemplate": "Iteration 18
Mean: 8.35e-17
Max: 2.97e-13
Nonzero: 4425/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#35b779", "width": 1 }, "name": "Iteration 18", "opacity": 0.6, "type": "scatter", "x": { "bdata": "auL4wYjh9cGm4PLBxN/vweLe7MH/3enBHd3mwTvc48FZ2+DBd9rdwZXZ2sGz2NfB0dfUwe/W0cEN1s7BK9XLwUnUyMFm08XBhNLCwaLRv8HA0LzB3s+5wfzOtsEazrPBOM2wwVXMrcFzy6rBkcqnwa/JpMHNyKHB68eewQnHm8EnxpjBRcWVwWPEksGBw4/Bn8KMwb3BicHbwIbB+L+DwRa/gMFofHvBpHp1weB4b8Ecd2nBWHVjwZRzXcHQcVfBC3BRwUZuS8FGbkvBC3BRwdBxV8GUc13BWHVjwRx3acHgeG/BpHp1wWh8e8EWv4DB+L+DwdvAhsG9wYnBn8KMwYHDj8FjxJLBRcWVwSfGmMEJx5vB68eewc3IocGvyaTBkcqnwXPLqsFVzK3BOM2wwRrOs8H8zrbB3s+5wcDQvMGi0b/BhNLCwWbTxcFJ1MjBK9XLwQ3WzsHv1tHB0dfUwbPY18GV2drBd9rdwVnb4ME73OPBHd3mwf/d6cHi3uzBxN/vwabg8sGI4fXBauL4wQ==", "dtype": "f4" }, "y": { "bdata": "IiIiIiIiMkBuheJWKG4yQGg5f5bzZzJAHanXkXodMkCRAv7CGDAyQN7d3d3dXTJAR+p1pF7HMkBlk0022WQyQLScP8v5MzJASpCnBHlKMkDNzMzMzMwyQDisfcPatzJAY8A0BkxjMkBDcSsUt0IyQBS3QnErlDJAIiIiIiKiMkBqDJjGgGkyQO5MyFrfMDJA5vxZzp9lMkCBaQyYxoAyQF100UUXXTJA7+7u7u5uMkB2pF5H6nUyQAOmMWAagzJA3v3ywM9yMkBmZmZmZmYyQGLtG9a+YTJA2zesfcNaMkBxKxS3QnEyQDisfcPaNzJAKkGeEuQpMkCoBHlKkCcyQLkViluhODJAmpmZmZkZMkAQPvjggw8yQImIiIiICDJAh7VvWPsGMkCMLrroogsyQAyYxoBpDDJAMPBuFDANMkAREREREREyQJ0/y/mzHDJAFLdCcSsUMkCHtW9Y+wYyQII8JchTAjJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAC0xgwjQEyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkAAAAAAAAAyQAAAAAAAADJAAAAAAAAAMkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -31.110553741455078, -12.714422225952148 ], "y": [ 18.0, 18.0 ] }, { "fill": "toself", "fillcolor": "#35b779", "hovertemplate": "Iteration 19
Mean: 5.06e-18
Max: 1.82e-14
Nonzero: 3889/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#35b779", "width": 1 }, "name": "Iteration 19", "opacity": 0.6, "type": "scatter", "x": { "bdata": "anIBwmLi/8Hw3/zBft35wQ3b9sGb2PPBKtbwwbjT7cFG0erB1c7nwWPM5MHyyeHBgMfewQ7F28GdwtjBK8DVwbq90sFIu8/B1rjMwWW2ycHzs8bBgrHDwRCvwMGfrL3BLaq6wbunt8FKpbTB2KKxwWegrsH1navBg5uowRKZpcGglqLBL5Sfwb2RnMFLj5nB2oyWwWiKk8H3h5DBhYWNwRODisGigIfBMH6Ewb57gcGa8nzBt+12wdTocMHx42rBDt9kwSraXsEq2l7BDt9kwfHjasHU6HDBt+12wZryfMG+e4HBMH6EwaKAh8ETg4rBhYWNwfeHkMFoipPB2oyWwUuPmcG9kZzBL5SfwaCWosESmaXBg5uowfWdq8FnoK7B2KKxwUqltMG7p7fBLaq6wZ+svcEQr8DBgrHDwfOzxsFltsnB1rjMwUi7z8G6vdLBK8DVwZ3C2MEOxdvBgMfewfLJ4cFjzOTB1c7nwUbR6sG40+3BKtbwwZvY88EN2/bBft35wfDf/MFi4v/BanIBwg==", "dtype": "f4" }, "y": { "bdata": "54QSDdwiM0CpTmDPEFIzQJoXfyLhlDNAcIZktBNKM0BKNHCLgB4zQCQe7TOxGDNAIniOuspSM0DNzMzMzMwzQEFheUG8ejNAbFaSraJAM0DRAbs91jIzQFq7wJTUdzNAOWTstcuPM0CbHPNkkXAzQOWIR/tMLDNAMrVNqus3M0AojPsul2AzQI54tM/EYjNA0FVLTLVNM0BV12/MDVozQN2ghZDyZDNASPqv9ntrM0DO0IELwmozQMwNWggpTzNAe+3yI91fM0C4hs1KslUzQO/D/xOaWDNAIniOuspSM0CZXEFMriAzQITVtI43JzNAhNW0jjcnM0Deh7i2NC8zQLkFl3qDFjNAOKV58ScSM0AmFoNXzwUzQLDbYy1DBzNAwJ7NL0MHM0DAns0vQwczQGC3x1qGDjNAw2Yl2SoKM0DX8eaEEg0zQPLNkpWAHjNAYLfHWoYOM0DZ+BSDWwQzQIrF4NVzATNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0D0/qLVcwEzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0AAAAAAAAAzQAAAAAAAADNAAAAAAAAAM0A=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -32.361732482910156, -13.928262710571289 ], "y": [ 19.0, 19.0 ] }, { "fill": "toself", "fillcolor": "#35b779", "hovertemplate": "Iteration 20
Mean: 2.53e-19
Max: 9.15e-16
Nonzero: 3097/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#35b779", "width": 1 }, "name": "Iteration 20", "opacity": 0.6, "type": "scatter", "x": { "bdata": "plYFwijcA8KsYQLCLucAwmTZ/sFq5PvBcO/4wXb69cF8BfPBghDwwYkb7cGPJurBlTHnwZs85MGhR+HBp1Lewa5d28G0aNjBunPVwcB+0sHGic/BzJTMwdOfycHZqsbB37XDweXAwMHry73B8da6wffht8H97LTBBPixwQoDr8EQDqzBFhmpwR0kpsEjL6PBKTqgwS9FncE1UJrBO1uXwUFmlMFHcZHBTXyOwVSHi8FakojBYJ2FwWaogsHZZn/B5nx5wfKSc8HyknPB5nx5wdlmf8FmqILBYJ2FwVqSiMFUh4vBTXyOwUdxkcFBZpTBO1uXwTVQmsEvRZ3BKTqgwSMvo8EdJKbBFhmpwRAOrMEKA6/BBPixwf3stMH34bfB8da6wevLvcHlwMDB37XDwdmqxsHTn8nBzJTMwcaJz8HAftLBunPVwbRo2MGuXdvBp1LewaFH4cGbPOTBlTHnwY8m6sGJG+3BghDwwXwF88F2+vXBcO/4wWrk+8Fk2f7BLucAwqxhAsIo3APCplYFwg==", "dtype": "f4" }, "y": { "bdata": "ERERERERNEDv7u7u7j40QM3MzMzMzDRAmpmZmZmpNEAzZxStmTk0QImIiIiIKDRAmpmZmZkpNEB3d3d3d3c0QKuqqqqqyjRAcumwqneXNEAREREREUE0QAAAAAAAMDRAZmZmZmZmNEDNzMzMzHw0QN7d3d3dbTRAmpmZmZlZNEA5d96fd3c0QFVVVVVVdTRA7+7u7u6ONEBVVVVVVZU0QLy7u7u7ezRAChgv7MxcNECrqqqqqoo0QAAAAAAAcDRAAAAAAABwNEBVVVVVVWU0QN7d3d3dPTRAq6qqqqoqNEB3d3d3dzc0QJqZmZmZOTRAaNGBvqo6NECrqqqqqho0QCIiIiIiEjRAqVFri4gINECamZmZmQk0QImIiIiICDRAVVVVVVUFNEAzMzMzMxM0QLy7u7u7CzRA7+7u7u4ONEARERERESE0QBERERERITRAq6qqqqoKNEA/fZBoZgY0QCIiIiIiAjRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEA3e34iIgI0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEAAAAAAAAA0QAAAAAAAADRAAAAAAAAANEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -33.334617614746094, -15.22337532043457 ], "y": [ 20.0, 20.0 ] }, { "fill": "toself", "fillcolor": "#35b779", "hovertemplate": "Iteration 21
Mean: 1.02e-20
Max: 3.71e-17
Nonzero: 2351/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#35b779", "width": 1 }, "name": "Iteration 21", "opacity": 0.6, "type": "scatter", "x": { "bdata": "Ml8JwqTsB8IWegbCiAcFwvqUA8JsIgLC3q8AwqB6/sGFlfvBabD4wU3L9cEx5vLBFQHwwfkb7cHdNurBwVHnwaVs5MGKh+HBbqLewVK928E22NjBGvPVwf4N08HiKNDBxkPNwapeysGOecfBcpTEwVavwcE6yr7BHuW7wQIAucHmGrbByjWzwa5QsMGSa63Bd4aqwVuhp8E/vKTBI9ehwQfynsHsDJzB0CeZwbRClsGYXZPBfHiQwWCTjcFErorBKMmHwQzkhMEM5ITBKMmHwUSuisFgk43BfHiQwZhdk8G0QpbB0CeZwewMnMEH8p7BI9ehwT+8pMFboafBd4aqwZJrrcGuULDByjWzweYatsECALnBHuW7wTrKvsFWr8HBcpTEwY55x8GqXsrBxkPNweIo0MH+DdPBGvPVwTbY2MFSvdvBbqLewYqH4cGlbOTBwVHnwd026sH5G+3BFQHwwTHm8sFNy/XBabD4wYWV+8Ggev7B3q8AwmwiAsL6lAPCiAcFwhZ6BsKk7AfCMl8Jwg==", "dtype": "f4" }, "y": { "bdata": "YtyvSyofNUADXCRnfS81QAfPESjaajVAzczMzMzMNUBnfS/ARXI1QCRnfS/ARTVABHPtwFw7NUDHi+wtIiI1QCRQtNXgOTVASLcxBaF/NUAJFG01eI41QEZy1vcCXDVAJ8Ohlj11NUBJ5cO4X5c1QGiU+BklfjVAy4dxvy6pNUBsB+bagbk1QD7Y06gerzVAqE6GQy17NUCJiIiIiIg1QAorNo9XmjVAqpPhUMueNUDJWd8LcJE1QCfDoZY9dTVAJX5GiZ9RNUDiZ5T4GSU1QGMKQv/oNjVARC176mQ4NUBGctb3Alw1QKTbmILQPzVAQv/oNqYgNUBgrh2Yawc1QICL5KzvBTVAARfJWd8LNUABF8lZ3ws1QNiZJnJjCjVAYcXm8UoTNUABF8lZ3ws1QMFcOzDXDjVA4lDLnjoZNUDjfl1S+TA1QOXqqBmeIzVAARfJWd8LNUCgaKvBcwQ1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUDARXLW9wI1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUAAAAAAAAA1QAAAAAAAADVAAAAAAAAANUA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -34.34296417236328, -16.611351013183594 ], "y": [ 21.0, 21.0 ] }, { "fill": "toself", "fillcolor": "#6ece58", "hovertemplate": "Iteration 22
Mean: 3.22e-22
Max: 1.18e-18
Nonzero: 1772/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#6ece58", "width": 1 }, "name": "Iteration 22", "opacity": 0.6, "type": "scatter", "x": { "bdata": "fpwMwiY4C8LQ0wnCeG8IwiILB8LMpgXCdEIEwh7eAsLGeQHCcBUAwjJi/cGEmfrB1tD3wSgI9cF6P/LBzXbvwR+u7MFx5enBwxznwRZU5MFoi+HBusLewQz628FfMdnBsWjWwQOg08FV19DBpw7OwfpFy8FMfcjBnrTFwfHrwsFDI8DBlVq9weeRusE5ybfBiwC1wd43ssEwb6/BgqaswdXdqcEnFafBeUykwcuDocEdu57BcPKbwcIpmcEUYZbBZpiTwbjPkMG4z5DBZpiTwRRhlsHCKZnBcPKbwR27nsHLg6HBeUykwScVp8HV3anBgqaswTBvr8HeN7LBiwC1wTnJt8HnkbrBlVq9wUMjwMHx68LBnrTFwUx9yMH6RcvBpw7OwVXX0MEDoNPBsWjWwV8x2cEM+tvBusLewWiL4cEWVOTBwxznwXHl6cEfruzBzXbvwXo/8sEoCPXB1tD3wYSZ+sEyYv3BcBUAwsZ5AcIe3gLCdEIEwsymBcIiCwfCeG8IwtDTCcImOAvCfpwMwg==", "dtype": "f4" }, "y": { "bdata": "xALu9EcsNkD2QdOiXR82QNgB9E2FHTZAiQXc6Y9YNkCZzRMsGV42QGKEI5kcRjZAJwRl72tCNkDsg6ZFuz42QKdFuz5oWjZAMMjqif+CNkDyZkBlwpE2QPTK2H5HrzZAQ0qdgTWkNkDNzMzMzMw2QJz8gQ7YhDZA1wdNi3Z9NkCcCjuAvqk2QKZLFHxZujZA14r5KW+tNkDnu4cRhaA2QBzJ5DDCkTZAHUY4kslhNkBO/kAHbEI2QPZB06JdHzZA7IOmRbs+NkAnBGXva0I2QDqGF+ehYzZAsQPomwo7NkDqqY5YhR02QB5A31TYATZA9VRHrMIONkDPwBpS6gw2QHYAfVNhBzZAdgB9U2EHNkAKQdn7mhA2QApB2fuaEDZAz8AaUuoMNkC4fzUBEgs2QLvBFPmsGzZA+JOCsIE1NkA7A2tIqTM2QM/AGlLqDDZAlEBcqDkJNkA7gL6psAM2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkA91RGrsAM2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkAAAAAAAAA2QAAAAAAAADZAAAAAAAAANkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -35.15282440185547, -18.101425170898438 ], "y": [ 22.0, 22.0 ] }, { "fill": "toself", "fillcolor": "#6ece58", "hovertemplate": "Iteration 23
Mean: 7.69e-24
Max: 2.84e-20
Nonzero: 1440/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#6ece58", "width": 1 }, "name": "Iteration 23", "opacity": 0.6, "type": "scatter", "x": { "bdata": "mhgQwr7DDsLibg3CBBoMwijFCsJLcAnCbhsIwpLGBsK0cQXC2BwEwvzHAsIecwHCQh4AwsqS/cER6frBWD/4wZ6V9cHl6/LBLELwwXKY7cG47urB/0TowUab5cGM8eLB00fgwRme3cFg9NrBpkrYwe2g1cE099LBek3QwcGjzcEH+srBTlDIwZSmxcHb/MLBIlPAwWipvcGv/7rB9VW4wTystcGCArPByViwwRCvrcFWBavBnVuoweOxpcEqCKPBcF6gwbe0ncG3tJ3BcF6gwSoIo8HjsaXBnVuowVYFq8EQr63ByViwwYICs8E8rLXB9VW4wa//usFoqb3BIlPAwdv8wsGUpsXBTlDIwQf6ysHBo83Bek3QwTT30sHtoNXBpkrYwWD02sEZnt3B00fgwYzx4sFGm+XB/0Towbju6sFymO3BLELwweXr8sGelfXBWD/4wRHp+sHKkv3BQh4Awh5zAcL8xwLC2BwEwrRxBcKSxgbCbhsIwktwCcIoxQrCBBoMwuJuDcK+ww7CmhgQwg==", "dtype": "f4" }, "y": { "bdata": "lWB/CioqN0AqKioqKio3QAkzfw02NjdABox+GGJiN0CFOP4deHg3QERERERERDdAKsH+FFRUN0CFOP4deHg3QKltfhpqajdAzczMzMzMN0AGjH4YYmI3QBFm/hpsbDdAiR+IW5KSN0B89BVtrq43QDBNfS22tjdAyQASaKamN0Cj+pNqqqo3QIU4/h14eDdAVWlwPWJiN0CgOv8MNDQ3QM3ODxQgIDdA8n5/CCIiN0BOkSMtSEg3QDuOZC5KSjdAEWb+GmxsN0Bbd/8IJCQ3QBrbCw8YGDdA7fxAAQICN0DR8P8ABAQ3QAAAAAAAADdAAAAAAAAAN0Cuu38EEhI3QHTS/wIMDDdAje2FBwwMN0Cg8EQGCgo3QKLh/wEICDdAU+RICxISN0BA4YkMFBQ3QM9J/wswMDdAQxz/Djw8N0Bn5wcKEBA3QI3thQcMDDdA0fD/AAQEN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0DR8P8ABAQ3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0AAAAAAAAA3QAAAAAAAADdAAAAAAAAAN0A=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -36.024024963378906, -19.713239669799805 ], "y": [ 23.0, 23.0 ] }, { "fill": "toself", "fillcolor": "#b5de2b", "hovertemplate": "Iteration 24
Mean: 1.30e-25
Max: 4.83e-22
Nonzero: 1151/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#b5de2b", "width": 1 }, "name": "Iteration 24", "opacity": 0.6, "type": "scatter", "x": { "bdata": "WVIWwuQBFcJusRPC+GASwoMQEcINwA/CmG8OwiIfDcKszgvCN34KwsEtCcJM3QfC1owGwmA8BcLr6wPCdZsCwgBLAcIU9f/BKVT9wT6z+sFSEvjBaHH1wXzQ8sGSL/DBpo7twbrt6sHQTOjB5KvlwfoK48EOauDBIsndwTgo28FMh9jBYubVwXZF08GKpNDBoAPOwbRiy8HKwcjB3iDGwfN/w8EI38DBHD6+wTKdu8FG/LjBW1u2wXC6s8GEGbHBmniuwa7Xq8Gu16vBmniuwYQZscFwurPBW1u2wUb8uMEynbvBHD6+wQjfwMHzf8PB3iDGwcrByMG0YsvBoAPOwYqk0MF2RdPBYubVwUyH2ME4KNvBIsndwQ5q4MH6CuPB5KvlwdBM6MG67erBpo7twZIv8MF80PLBaHH1wVIS+ME+s/rBKVT9wRT1/8EASwHCdZsCwuvrA8JgPAXC1owGwkzdB8LBLQnCN34KwqzOC8IiHw3CmG8Owg3AD8KDEBHC+GASwm6xE8LkARXCWVIWwg==", "dtype": "f4" }, "y": { "bdata": "9GsPuawYOEAvcMAxAyU4QOGNps6vPThA+9Y48fhzOECTrvuY+HM4QNlgvRMgtDhAgKrKnCC0OEAEZweMw104QCgdhbWORzhAY0GqlJFsOEBRY0GqlJE4QJB6tQJ3wDhApKnzaw+5OECQerUCd8A4QM3MzMzMzDhAyGXFQAlvOEDyWYkHF0U4QFNGssF6JzhAjCbNJ5wdOEC+tEURnB04QIavpwJMWzhA2Ppi9H1MOEAeWmqAPoU4QEZfB4zAODhA7SGXFQMlOEAAAAAAAAA4QFdaIgdnBzhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAcnMLrXcCOEAAAAAAAAA4QMnNLbTeCThAPEE5YVYMOECVkWyw3gk4QCAoULtFEThAPEE5YVYMOEB8kdnX4S44QNj6YvR9TDhA7SGXFQMlOEBGk+YTzg44QMpINljvBDhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEDl5hZa7wQ4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEAAAAAAAAA4QAAAAAAAADhAAAAAAAAAOEA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -37.58041763305664, -21.48031234741211 ], "y": [ 24.0, 24.0 ] }, { "fill": "toself", "fillcolor": "#b5de2b", "hovertemplate": "Iteration 25
Mean: 1.39e-27
Max: 5.18e-24
Nonzero: 695/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#b5de2b", "width": 1 }, "name": "Iteration 25", "opacity": 0.6, "type": "scatter", "x": { "bdata": "Q6YZwldtGMJsNBfCgPsVwpTCFMKoiRPCvFASwtAXEcLk3g/C+aUOwg1tDcIhNAzCNfsKwkrCCcJeiQjCclAHwoYXBsKa3gTCrqUDwsJsAsLXMwHC1vX/wf6D/cEmEvvBUKD4wXgu9sGgvPPByErxwfDY7sEZZ+zBQvXpwWqD58GSEeXBup/iweMt4MEMvN3BNErbwVzY2MGEZtbBrPTTwdaC0cH+EM/BJp/MwU4tysF2u8fBn0nFwcjXwsHwZcDBGPS9wUCCu8FAgrvBGPS9wfBlwMHI18LBn0nFwXa7x8FOLcrBJp/Mwf4Qz8HWgtHBrPTTwYRm1sFc2NjBNErbwQy83cHjLeDBup/iwZIR5cFqg+fBQvXpwRln7MHw2O7ByErxwaC888F4LvbBUKD4wSYS+8H+g/3B1vX/wdczAcLCbALCrqUDwpreBMKGFwbCclAHwl6JCMJKwgnCNfsKwiE0DMINbQ3C+aUOwuTeD8LQFxHCvFASwqiJE8KUwhTCgPsVwmw0F8JXbRjCQ6YZwg==", "dtype": "f4" }, "y": { "bdata": "u8EU+awbOUBiAXf6IxY5QFcE3hI2ITlAZmZmZmZmOUC/JgRl72s5QLlKGtWWqzlAzczMzMzMOUCXq6RRbbk5QJUY6moyvDlAOoYX56FjOUDdYIp81g05QN1ginzWDTlAYgF3+iMWOUCIggGLXxM5QApB2fuaEDlASaPacpU0OUCspFFtuUo5QDqGF+ehYzlApshn3WCKOUDeht+dlTQ5QJgin3WDKTlAAAAAAAAAOUAs4E5/xAI5QJxf4QCJBTlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlALOBOf8QCOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAnF/hAIkFOUDdYIp81g05QN78CgdILDlAMEU+6wZTOUCYIp91gyk5QApB2fuaEDlAN7/CARILOUDOr3CAxAI5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUCcX+EAiQU5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUAAAAAAAAA5QAAAAAAAADlAAAAAAAAAOUA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -38.4123649597168, -23.4385986328125 ], "y": [ 25.0, 25.0 ] }, { "fill": "toself", "fillcolor": "#b5de2b", "hovertemplate": "Iteration 26
Mean: 7.00e-30
Max: 2.63e-26
Nonzero: 9/7393
Log10(Value): %{x:.2f}
", "line": { "color": "#b5de2b", "width": 1 }, "name": "Iteration 26", "opacity": 0.6, "type": "scatter", "x": { "bdata": "m80QwunxD8I4Fg/ChjoOwtReDcIigwzCcKcLwr/LCsIN8AnCWxQJwqo4CML4XAfCRoEGwpSlBcLiyQTCMO4Dwn8SA8LNNgLCG1sBwmp/AMJwR//BDJD9wajY+8FEIfrB4mn4wX6y9sEa+/TBtkPzwVOM8cHw1O/BjB3uwShm7MHFrurBYvfowf4/58GaiOXBNtHjwdQZ4sFwYuDBDKvewajz3MFGPNvB4oTZwX7N18EaFtbBtl7UwVSn0sHw79DBjDjPwSiBzcEogc3BjDjPwfDv0MFUp9LBtl7UwRoW1sF+zdfB4oTZwUY828Go89zBDKvewXBi4MHUGeLBNtHjwZqI5cH+P+fBYvfowcWu6sEoZuzBjB3uwfDU78FTjPHBtkPzwRr79MF+svbB4mn4wUQh+sGo2PvBDJD9wXBH/8FqfwDCG1sBws02AsJ/EgPCMO4DwuLJBMKUpQXCRoEGwvhcB8KqOAjCWxQJwg3wCcK/ywrCcKcLwiKDDMLUXg3ChjoOwjgWD8Lp8Q/Cm80Qwg==", "dtype": "f4" }, "y": { "bdata": "pTu9KmZmOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkClO70qZmY6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QKU7vSpmZjpApTu9KmZmOkCt1g+iZmY6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkClO70qZmY6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkClO70qZmY6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkDNzMzMzMw6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkAAAAAAAAA6QAAAAAAAADpAAAAAAAAAOkA=", "dtype": "f8" } }, { "hoverinfo": "skip", "line": { "color": "black", "width": 1 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ -36.20078659057617, -25.688064575195312 ], "y": [ 26.0, 26.0 ] } ], "layout": { "annotations": [ { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=9.8e-05", "x": -4.008277987205572, "y": 0.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=2.8e-02", "x": -1.5457277616278364, "y": 0.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=7.4e-05", "x": -4.127988302637768, "y": 1.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=2.2e-02", "x": -1.6627302138494087, "y": 1.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=5.2e-05", "x": -4.283984538319214, "y": 2.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=3.5e-02", "x": -1.4591520115773475, "y": 2.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=2.9e-05", "x": -4.531424306113376, "y": 3.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=3.0e-02", "x": -1.5233476519199698, "y": 3.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=1.3e-05", "x": -4.886991623874976, "y": 4.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=1.7e-02", "x": -1.7744908568741027, "y": 4.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=4.5e-06", "x": -5.345076813651129, "y": 5.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=6.7e-03", "x": -2.1719360462802517, "y": 5.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=1.3e-06", "x": -5.893591665626662, "y": 6.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=2.0e-03", "x": -2.6928656241122475, "y": 6.6 }, { "bgcolor": "white", "bordercolor": "blue", "borderwidth": 1, "font": { "color": "blue", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "μ=3.0e-07", "x": -6.516947032299294, "y": 7.3 }, { "bgcolor": "white", "bordercolor": "red", "borderwidth": 1, "font": { "color": "red", "size": 8 }, "opacity": 0.9, "showarrow": false, "text": "max=5.0e-04", "x": -3.3038318978913104, "y": 7.6 } ], "height": 800, "legend": { "orientation": "v", "x": 1.02, "xanchor": "left", "y": 1, "yanchor": "top" }, "plot_bgcolor": "white", "showlegend": true, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmap" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermap": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermap" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "sequentialminus": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "font": { "size": 16 }, "text": "Neumann Series Convergence Trace Ridgeline Plot", "x": 0.5 }, "width": 1200, "xaxis": { "gridcolor": "lightgray", "showgrid": true, "title": { "text": "Log10(Value + ε)" } }, "yaxis": { "showgrid": false, "tickmode": "array", "ticktext": [ "Iter 0", "Iter 1", "Iter 2", "Iter 3", "Iter 4", "Iter 5", "Iter 6", "Iter 7", "Iter 8", "Iter 9", "Iter 10", "Iter 11", "Iter 12", "Iter 13", "Iter 14", "Iter 15", "Iter 16", "Iter 17", "Iter 18", "Iter 19", "Iter 20", "Iter 21", "Iter 22", "Iter 23", "Iter 24", "Iter 25", "Iter 26", "Iter 27" ], "tickvals": { "bdata": "AAAAAAAAAAAAAAAAAADwPwAAAAAAAABAAAAAAAAACEAAAAAAAAAQQAAAAAAAABRAAAAAAAAAGEAAAAAAAAAcQAAAAAAAACBAAAAAAAAAIkAAAAAAAAAkQAAAAAAAACZAAAAAAAAAKEAAAAAAAAAqQAAAAAAAACxAAAAAAAAALkAAAAAAAAAwQAAAAAAAADFAAAAAAAAAMkAAAAAAAAAzQAAAAAAAADRAAAAAAAAANUAAAAAAAAA2QAAAAAAAADdAAAAAAAAAOEAAAAAAAAA5QAAAAAAAADpAAAAAAAAAO0A=", "dtype": "f8" }, "title": { "text": "Iteration" } } } } }, "metadata": {}, "output_type": "display_data" } ], "source": [ "trace_dict = analysis_injector[\"ap_node_compute_influence\"][\"trace_dict\"][\"node\"]\n", "stacked_trace = torch.stack(trace_dict)\n", "fig = plot_ridgeline_convergence(\n", " data=stacked_trace, stats=None, title=\"Neumann Series Convergence Trace Ridgeline Plot\"\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "c32dcca2", "metadata": { "papermill": { "duration": 0.022053, "end_time": "2026-07-30T19:07:54.184710+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.162657+00:00", "status": "completed" }, "tags": [] }, "source": [ "Note you should see very little marginal distributional change after the first few iterations.\n", "For most examples, you should see that after around the first few iterations, the distribution stabilizes, which suggests contributions from longer paths\n", "are not significantly contributing to logit values.\n", "\n", "- Iteration 0 (initial logit prob dot product) captures the weighted sum of the direct influences for each feature on each of the logit probs\n", "- Iteration 1 captures for each feature, that feature's weighted influence on the iteration 0 direct logit prob influences (in other words, the weighted direct logit prob influences of features mediated one-hop through each feature)\n", "- Iteration 2 the weighted one-hop prob influences of features mediated two-hop through each feature (i.e. the second iteration reflects the marginal influence of two-hop paths, multiplying the adjacency matrix feature influences by all the 1-hop current_influence vector elements)\n", "- `current_influences` continues to get smaller and smaller as the order/path length increases!" ] }, { "cell_type": "code", "execution_count": 24, "id": "155648e6", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:54.229199Z", "iopub.status.busy": "2026-07-30T19:07:54.229013Z", "iopub.status.idle": "2026-07-30T19:07:54.233563Z", "shell.execute_reply": "2026-07-30T19:07:54.233008Z" }, "papermill": { "duration": 0.027177, "end_time": "2026-07-30T19:07:54.234240+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.207063+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Convergence occurred at iteration: 27\n" ] }, { "data": { "text/html": [ "
IterationTop k ValuesTop k Indices
0[0.028462, 0.018605, 0.015856, 0.012868, 0.010059][7382, 7381, 6390, 6884, 7364]
2[0.034741, 0.024363, 0.021626, 0.018278, 0.014635][7376, 7374, 7375, 7380, 7379]
4[0.016808, 0.011746, 0.010822, 0.006048, 0.005398][7376, 7375, 7374, 7380, 7379]
8[0.000123, 0.000105, 8.9e-05, 1.6e-05, 1.5e-05][7375, 7374, 7376, 7379, 7380]
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Nicely format top-k influence values and indices by iteration using orchestrator helper\n", "convergence_iteration = analysis_injector[\"ap_node_compute_influence\"][\"iteration\"]\n", "print(f\"Convergence occurred at iteration: {convergence_iteration}\")\n", "\n", "sample_tensor_output(stacked_trace, (0, 2, 4, 8), [\"Iteration\", \"Top k Values\", \"Top k Indices\"], 5, tablefmt=\"html\")" ] }, { "cell_type": "markdown", "id": "68c58fd0", "metadata": { "papermill": { "duration": 0.019283, "end_time": "2026-07-30T19:07:54.272689+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.253406+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Computing `node_mask`" ] }, { "cell_type": "markdown", "id": "dd11bb90", "metadata": { "papermill": { "duration": 0.017916, "end_time": "2026-07-30T19:07:54.309010+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.291094+00:00", "status": "completed" }, "tags": [] }, "source": [ "We then calculate a node_mask by finding a threshold (`find_threshold`) that meets our targeted explained influence using our `node_influence` and configured node_threshold.\n", "\n", "```python\n", "def find_threshold(scores: torch.Tensor, threshold: float):\n", " # Find score threshold that keeps the desired fraction of total influence\n", " sorted_scores = torch.sort(scores, descending=True).values\n", " cumulative_score = torch.cumsum(sorted_scores, dim=0) / torch.sum(sorted_scores)\n", " threshold_index = torch.searchsorted(cumulative_score, threshold)\n", " # make sure we don't go out of bounds (only really happens at threshold=1.0)\n", " threshold_index = min(threshold_index, len(cumulative_score) - 1)\n", " return sorted_scores[threshold_index]\n", "```" ] }, { "cell_type": "markdown", "id": "a44b7129", "metadata": { "papermill": { "duration": 0.01874, "end_time": "2026-07-30T19:07:54.346428+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.327688+00:00", "status": "completed" }, "tags": [] }, "source": [ "Before we compute our `edge_mask`, our `node_mask` has filtered the number of non-zero src and target features to include only features that explain the specified `node_threshold` of influence\n", "\n", "```python\n", "pruned_matrix = graph.adjacency_matrix.clone()\n", "pruned_matrix[~node_mask] = 0\n", "pruned_matrix[:, ~node_mask] = 0\n", "```" ] }, { "cell_type": "markdown", "id": "548d6f82", "metadata": { "papermill": { "duration": 0.01859, "end_time": "2026-07-30T19:07:54.384017+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.365427+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Prune by Node Influence`." ] }, { "cell_type": "code", "execution_count": 25, "id": "1e1ab094", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:54.421733Z", "iopub.status.busy": "2026-07-30T19:07:54.421471Z", "iopub.status.idle": "2026-07-30T19:07:54.426052Z", "shell.execute_reply": "2026-07-30T19:07:54.425267Z" }, "papermill": { "duration": 0.024759, "end_time": "2026-07-30T19:07:54.426967+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.402208+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
n_logits10
n_tokens9
node_mask_nonzero1601Number of non-zero elements in the node mask, includes token and logit nodes plus minimum required nodes to meet specified node threshold
node_threshold0.8
pruned_matrix_nonzero768341
node_mask_nonlogit_nontoken_nonzero1582We always keep token and logit nodes, so we subtract them from the count
percent_of_nodes_kept21.4%The percentage of nodes kept after pruning
percent_of_elements_kept1.41%The percentage of non-zero elements kept in the pruned adjacency matrix after applying our node_mask to both dimensions
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_graph_prune_node_influence_end\")" ] }, { "cell_type": "markdown", "id": "435c8852", "metadata": { "papermill": { "duration": 0.018543, "end_time": "2026-07-30T19:07:54.464551+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.446008+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Computing `edge_mask`" ] }, { "cell_type": "markdown", "id": "ac183794", "metadata": { "papermill": { "duration": 0.0182, "end_time": "2026-07-30T19:07:54.501149+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.482949+00:00", "status": "completed" }, "tags": [] }, "source": [ "We can further reduce the graph size by filtering on the cumulative influence of all edges (0.98 threshold in this case).\n", "\n", "- `compute_edge_influence` starts with a node-threshold-pruned adjacency matrix:\n", " ```python\n", " edge_scores = compute_edge_influence(pruned_matrix, logit_weights)\n", " ```\n", "- The pruned_matrix is not initially normalized, but `compute_edge_influence` will normalize it prior to `compute_influence`\n", "- After `computing_influence` (same fn/process as above) we see we have the feature influences vector for nodes that influence logits (weighted average of all collectively) above the given threshold\n", "- Since we want to have our completed edge matrix to include the logit probs, we add in logit_weights to the `pruned_influence` vector before returning `edge_scores` as normalized_pruned matrix elementwise multiplied by the each element of the `pruned_influence` vector.\n", "- We'll see below at the end we no longer have normalized target influence vectors but rather the sum of the src features will be the total influence that target feature had on the overall logits\n", "\n", "- So our new `edge_scores` matrix will have rows where each target feature is the `pruned_influence` value for that feature multiplied by the `normalize_pruned` vector for each feature (from the `normalized_pruned` matrix which is the threshold-pruned adjacency matrix) \n", "- As we see below, this effectively scales our normalized feature scores for each target feature by our `pruned_influence` scores (sum of all paths above a threshold)\n", "- By multiplying the normalized vector of influences for each target feature by every other source feature by the total influence of that target feature on the logits, **we scale for each target feature the source feature influences so they are weighted for that target feature's influence on logits by all paths**\n", "\n", "- Because we scaled our normalized features by the pruned_influence values, *the sum of our target feature rows will now be the pruned_influence values*.\n", "\n", "- You should observe below that the feature influences are quite distributed. In the case of the default example prompt, for the top logit, only 0.19 of the 0.29 logit probability (64%) is accounted for by the top 100 features, and the top feature only accounted for 0.0144 (4.8%) of the logit prob for our top logit." ] }, { "cell_type": "markdown", "id": "a0a09c59", "metadata": { "papermill": { "duration": 0.01834, "end_time": "2026-07-30T19:07:54.539499+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.521159+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Compute Edge Influence`." ] }, { "cell_type": "code", "execution_count": 26, "id": "e90cb899", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:54.578238Z", "iopub.status.busy": "2026-07-30T19:07:54.577974Z", "iopub.status.idle": "2026-07-30T19:07:54.584306Z", "shell.execute_reply": "2026-07-30T19:07:54.583477Z" }, "papermill": { "duration": 0.027435, "end_time": "2026-07-30T19:07:54.585355+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.557920+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
edge_scores.shape(7393, 7393)total_nodes, total_nodes
normalized_pruned.shape(7393, 7393)total_nodes, total_nodes
pruned_influence.shape(7393,)total_nodes
top_target_edge_vals'▁Austin': [0.0146, 0.0138, 0.0117, 0.0088, 0.0079]
'▁Dallas': [0.002, 0.0012, 0.0011, 0.0008, 0.0007]
top_target_edge_idxs'▁Austin': [6390, 7381, 6884, 7364, 6462]
'▁Dallas': [7381, 6390, 6884, 6462, 7134]
target_token_logit_prob'▁Austin': 0.3047
'▁Dallas': 0.0284
top_1_edge_percent_logit_prob'▁Austin': 0.0479
'▁Dallas': 0.0708
top_10_edges_percent_logit_prob'▁Austin': 0.2839
'▁Dallas': 0.2977
top_100_edges_percent_logit_prob'▁Austin': 0.6368
'▁Dallas': 0.6564
top_1000_edges_percent_logit_prob'▁Austin': 0.9693
'▁Dallas': 0.9738
prenormed_top_features[101.296, 0.0, 0.0, 82.297, 77.543]
[0.0, 101.296, 82.297, 54.113, 44.113]		Sum of pre-normalization pruned matrix values for our top initial edge indices
postnormed_top_features[1.0, 0.0, 0.0, 1.0, 1.0]
[0.0, 1.0, 1.0, 1.0, 1.0]		Verify normalization of the post-normalization pruned matrix values for our top initial edge indices
max_logit_max_feature_idx6390Index of the feature with the maximum edge score for our top logit
total_pruned_influence_max_logit_max_feature0.024Total pruned influence for the feature with max edge score to our top logit
total_logit_prob_influence_max_logit_max_feature0.02Sum of edge scores to all logits from the feature with max edge score for our top logit
total_nonlogit_influence_max_logit_max_feature0.004Sum of edge scores to non-logit nodes from the feature with max edge score for our top logit.
pruned_influence_top_init_feature_vals[0.024, 0.07, 0.013, 0.014, 0.008]
[0.07, 0.024, 0.014, 0.012, 0.012]		Total influence from pruned influence for each of the top initial edges (top feature nodes influencing logits)
pruned_influence_top_nontoken_nonlogit_feature_vals[0.024, 0.017, 0.014, 0.013, 0.012]Top 5 non-token, non-logit feature total influences from pruned influence
pruned_influence_top_nontoken_nonlogit_feature_idxs[6390, 6884, 6462, 7364, 6067]Indices of top 5 non-token, non-logit feature total influences from pruned influence
pruned_influence_top_features_pruned[0.024, 0.07, 0.017, 0.013, 0.014]
[0.07, 0.024, 0.017, 0.014, 0.009]		Overall influence for each of the top features associated with top target logit edges
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_graph_prune_edge_influence_post_norm\")" ] }, { "cell_type": "markdown", "id": "762de5bc", "metadata": { "papermill": { "duration": 0.01934, "end_time": "2026-07-30T19:07:54.624115+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.604775+00:00", "status": "completed" }, "tags": [] }, "source": [ "We then compute our `edge_mask` using the same `find_threshold` function as above to meet our `edge_threshold` target.\n", "As we'll see, we're able to dramatically reduce the number of nonzero elements even while retaining a very large fraction of total edge influence.\n", "Note below the sum of our edge_score attributions to target logits will equal the aggregate target logit probabilities.\n" ] }, { "cell_type": "code", "execution_count": 27, "id": "2caea7e2", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:54.662762Z", "iopub.status.busy": "2026-07-30T19:07:54.662521Z", "iopub.status.idle": "2026-07-30T19:07:54.667460Z", "shell.execute_reply": "2026-07-30T19:07:54.666537Z" }, "papermill": { "duration": 0.025134, "end_time": "2026-07-30T19:07:54.667984+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.642850+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
edge_mask_numel54656449Number of elements in the edge mask
edge_mask_count_nonzero400245Number of non-zero elements in the edge mask
edge_mask_percent_nonzero0.73%Percentage of non-zero elements in the edge mask
node_mask_numel7393Number of elements in the node mask
node_mask_count_nonzero1601Number of non-zero elements in the node mask
node_mask_percent_nonzero21.66%Percentage of non-zero elements in the node mask
sum_edge_score_attribution_to_target_logits0.725Sum of edge score attribution to target logits, `edge_scores[-n_logits:, :].sum()`
sum_logit_probabilities0.725Sum of logit probabilities for target logits, `logit_weights[-n_logits:].sum()`, should match aggregate edge score attribution
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_graph_prune_edge_influence_pre_mask\")" ] }, { "cell_type": "markdown", "id": "7da1edd3", "metadata": { "papermill": { "duration": 0.018012, "end_time": "2026-07-30T19:07:54.704682+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.686670+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Applying Edge Mask and Finalizing Score Matrix" ] }, { "cell_type": "markdown", "id": "9548ae82", "metadata": { "papermill": { "duration": 0.018158, "end_time": "2026-07-30T19:07:54.740635+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.722477+00:00", "status": "completed" }, "tags": [] }, "source": [ "- We next ensure proper graph connection by ensuring all feature and error nodes have outgoing edges and all feature nodes have incoming edges\n", "\n", "```python\n", " old_node_mask = node_mask.clone()\n", " # Ensure feature and error nodes have outgoing edges\n", " node_mask[: -n_logits - n_tokens] &= edge_mask[:, : -n_logits - n_tokens].any(0)\n", " # Ensure feature nodes have incoming edges\n", " node_mask[:n_features] &= edge_mask[:n_features].any(1)\n", "```\n", "\n", "- `node_mask[: -n_logits - n_tokens] &= edge_mask[:, : -n_logits - n_tokens].any(0)`\n", " - **Slices** `node_mask` to exclude the last `n_logits + n_tokens` elements.\n", " - Uses `&=` (in-place logical AND) to update the mask.\n", " - `edge_mask[:, : -n_logits - n_tokens]` selects columns corresponding to the same nodes.\n", " - `.any(0)` checks if **any edge exists** for each node (across all rows dim=0), returning a boolean tensor.\n", " - The mask is updated so that only nodes with at least one outgoing edge remain `True`. (source nodes (columns) that are not currently masked and do not have any outgoing edges to target nodes will now be masked)\n", "- For the default example prompt, concretely, we examine the 7349 source influence columns in this case and return whether any target node (outgoing edge) exists\n", "\n", " ```python\n", " node_mask.count_nonzero()\n", " tensor(1584, device='cuda:0')\n", " edge_mask[:, : -n_logits - n_tokens].shape\n", " torch.Size([7368, 7349])\n", " edge_mask[:, : -n_logits - n_tokens].any(0).shape\n", " torch.Size([7349])\n", " ```" ] }, { "cell_type": "markdown", "id": "f18251bb", "metadata": { "papermill": { "duration": 0.018541, "end_time": "2026-07-30T19:07:54.777461+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.758920+00:00", "status": "completed" }, "tags": [] }, "source": [ "- `node_mask[:n_features] &= edge_mask[:n_features].any(1)`\n", " - Slices `node_mask` to the first `n_features` elements.\n", " - `edge_mask[:n_features]` selects rows for feature nodes.\n", " - `.any(1)` checks if **any incoming edge exists** for each feature node (across all columns).\n", " - Updates the mask so that only feature nodes with at least one incoming edge remain `True`.\n", "\n", "- We prune iteratively all nodes that are missing incoming or outgoing edges (in this case, no further pruning was necessary because our pruned node_mask already equaled our original mask, meaning our prune operations didn't make any changes)\n", " ```python\n", " torch.all(node_mask == old_node_mask)\n", " ```\n", "- We finally calculate the cumulative influence scores, sorting by `node_influence` descending and calculating the cumulative scores as a fraction of the total sorted scores and return our calculated node_mask, `edge_mask` and final_scores as a `PruneResult`\n", "```python\n", " # Calculate cumulative influence scores\n", " sorted_scores, sorted_indices = torch.sort(node_influence, descending=True)\n", " cumulative_scores = torch.cumsum(sorted_scores, dim=0) / torch.sum(sorted_scores)\n", " final_scores = torch.zeros_like(node_influence)\n", " final_scores[sorted_indices] = cumulative_scores\n", "```\n", "\n", "- Be aware that pytorch will round representations to 4 digits by default, but many of our non-zero contributing influences will appear to be 0 with this granularity.\n", "- In the case of the default example:\n", " - 7089 of our 7368 nodes contribute to the cumulative total score (279 zero scores in cumulative_scores)\n", " - The top 30 feature influences account for 34% of our aggregate logit probability." ] }, { "cell_type": "markdown", "id": "e644b4d2", "metadata": { "papermill": { "duration": 0.018983, "end_time": "2026-07-30T19:07:54.815532+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.796549+00:00", "status": "completed" }, "tags": [] }, "source": [ "**Analysis Point Data**: See below sampled data for the current attribution example at the end of `Applying Edge Mask and Finalizing Score Matrix`." ] }, { "cell_type": "code", "execution_count": 28, "id": "9e8c8de4", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:54.858503Z", "iopub.status.busy": "2026-07-30T19:07:54.858193Z", "iopub.status.idle": "2026-07-30T19:07:54.863692Z", "shell.execute_reply": "2026-07-30T19:07:54.862825Z" }, "papermill": { "duration": 0.029887, "end_time": "2026-07-30T19:07:54.864566+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.834679+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
KeyInspected OutputAnnotation
cumulative_scores_first_10[0.057, 0.099, 0.136, 0.164, 0.191, 0.214, 0.237, 0.251, 0.261, 0.27]First 10 values of cumulative_scores
cumulative_scores_last_10[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]Last 10 values of cumulative_scores
cumulative_scores_30th_feature0.339Cumulative score of the 30th feature
contributors_mask_sum7140Number of non-zero contributors in cumulative_scores (< 1)
sorted_indices_first_10[7376, 7374, 7375, 7381, 7380, 7382, 7379, 7377, 7378, 6390]First 10 values of sorted_indices
sorted_scores_first_10[0.115, 0.085, 0.075, 0.056, 0.056, 0.046, 0.045, 0.029, 0.021, 0.018]First 10 values of sorted_scores (from node_influence)
final_scores_first_10[0.746, 0.653, 0.896, 0.617, 0.641, 0.679, 0.358, 0.723, 0.657, 0.527]The returned final_scores reflects the CUMULATIVE node influences of each feature up to that feature number (create_nodes requires the cumulative_scores form of features for graph construction)
node_mask_count_nonzero1601Number of non-zero elements in node_mask
node_mask_sparsity21.66%Final sparsity of node_mask (percentage of non-zero elements)
edge_mask_count_nonzero400245Number of non-zero elements in edge_mask
edge_mask_sparsity0.73%Final sparsity of edge_mask (percentage of non-zero elements)
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "analysis_injector.get_output(\"ap_graph_prune_edge_influence_end\")" ] }, { "cell_type": "markdown", "id": "a61a4c43", "metadata": { "papermill": { "duration": 0.026324, "end_time": "2026-07-30T19:07:54.915601+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.889277+00:00", "status": "completed" }, "tags": [] }, "source": [ "##### Graph Creation and Saving" ] }, { "cell_type": "markdown", "id": "a0f74c3c", "metadata": { "papermill": { "duration": 0.020684, "end_time": "2026-07-30T19:07:54.954465+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.933781+00:00", "status": "completed" }, "tags": [] }, "source": [ "- We then use our node_mask, edge_mask and cumulative_scores to construct our graph:\n", " ```python\n", " tokenizer = AutoTokenizer.from_pretrained(graph.cfg.tokenizer_name)\n", " nodes = create_nodes(graph, node_mask, tokenizer, cumulative_scores, scan)\n", " used_nodes, used_edges = create_used_nodes_and_edges(graph, nodes, edge_mask)\n", " model = build_model(graph, used_nodes, used_edges, slug, scan, node_threshold, tokenizer)\n", "\n", " # Write the output locally\n", " with open(os.path.join(output_path, f\"{slug}.json\"), \"w\") as f:\n", " f.write(model.model_dump_json(indent=2))\n", " add_graph_metadata(model.metadata.model_dump(), output_path)\n", " logger.info(f\"Graph data written to {output_path}\")\n", "\n", " total_time_ms = (time.time() - total_start_time) * 1000\n", " logger.info(f\"Total execution time: {total_time_ms=:.2f} ms\")\n", " ```\n", "- When the graph is initially constructed, the `graph.active_features` tensor is created from this mapping in the Graph instantiation:\n", " ```python\n", " ...\n", " active_features=activation_matrix.indices().T,\n", " activation_values=activation_matrix.values(),\n", " ...\n", " adjacency_matrix=full_edge_matrix,\n", " ```\n", "- defined by:\n", " ```python\n", " active_features (torch.Tensor): A tensor of shape (n_active_features, 3)\n", " containing the indices (layer, pos, feature_idx) of the non-zero features\n", " of the model on the given input string.\n", " adjacency_matrix (torch.Tensor): The adjacency matrix. Organized as\n", " [active_features, error_nodes, embed_nodes, logit_nodes], where there are\n", " model.cfg.n_layers * len(input_tokens) error nodes, len(input_tokens) embed\n", " nodes, len(logit_tokens) logit nodes. The rows represent target nodes, while\n", " columns represent source nodes.\n", " ```" ] }, { "cell_type": "markdown", "id": "eba5b455", "metadata": { "papermill": { "duration": 0.018417, "end_time": "2026-07-30T19:07:54.994533+00:00", "exception": false, "start_time": "2026-07-30T19:07:54.976116+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Cleanly teardown analysis injection" ] }, { "cell_type": "code", "execution_count": 29, "id": "40000483", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:55.031470Z", "iopub.status.busy": "2026-07-30T19:07:55.031243Z", "iopub.status.idle": "2026-07-30T19:07:55.035915Z", "shell.execute_reply": "2026-07-30T19:07:55.035048Z" }, "papermill": { "duration": 0.023841, "end_time": "2026-07-30T19:07:55.036459+00:00", "exception": false, "start_time": "2026-07-30T19:07:55.012618+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Disabling analysis hooks...\n", "✓ Analysis injector cleaning torn down.\n", "Analysis log available for inspection: /tmp/attribution_flow_analysis_20260730_120707.log\n" ] } ], "source": [ "# Teardown hooks after graph generation\n", "if enable_analysis_injection:\n", " print(\"\\nDisabling analysis hooks...\")\n", " try:\n", " analysis_injector.teardown()\n", " print(\"✓ Analysis injector cleaning torn down.\")\n", " if log_path := getattr(analysis_injector, \"analysis_log\", None):\n", " print(f\"Analysis log available for inspection: {log_path}\")\n", " except Exception as e:\n", " print(f\"Error while tearing down analysis injector: {e}\")" ] }, { "cell_type": "markdown", "id": "abf2ed0c", "metadata": { "papermill": { "duration": 0.017521, "end_time": "2026-07-30T19:07:55.074731+00:00", "exception": false, "start_time": "2026-07-30T19:07:55.057210+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Saving and Visualizing Attribution Graphs\n", "\n", "In this section, we'll demonstrate how to save the generated attribution graphs and prepare them for visualization. The CircuitTracerAdapter integrates with Interpretune's AnalysisStore to persistently store graph data." ] }, { "cell_type": "code", "execution_count": 30, "id": "7d89ab42", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:55.117302Z", "iopub.status.busy": "2026-07-30T19:07:55.117086Z", "iopub.status.idle": "2026-07-30T19:07:55.127097Z", "shell.execute_reply": "2026-07-30T19:07:55.126181Z" }, "papermill": { "duration": 0.031935, "end_time": "2026-07-30T19:07:55.127642+00:00", "exception": false, "start_time": "2026-07-30T19:07:55.095707+00:00", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Not using port forwarding. Use the IFrame below, or open your graph here directly at http://speediedl:8046/index.html\n" ] } ], "source": [ "import os\n", "import socket\n", "\n", "from circuit_tracer.frontend.local_server import serve\n", "\n", "\n", "enable_iframe = False # whether to enable the IFrame display or not\n", "\n", "port = 8046\n", "server = serve(data_dir=ct_module.circuit_tracer_cfg.graph_output_dir, port=port)\n", "port_forwarding = False # whether to use port forwarding or not\n", "# Host used to build the graph URL. Defaults to this machine's hostname, which is what you want when\n", "# viewing from another machine on the same network. Override with IT_GRAPH_SERVER_HOST, or set\n", "# port_forwarding=True above to use localhost (e.g. over an SSH tunnel).\n", "hostname = os.environ.get(\"IT_GRAPH_SERVER_HOST\") or socket.gethostname()\n", "\n", "if port_forwarding:\n", " hostname = \"localhost\" # use localhost for port forwarding\n", " print(\n", " f\"Using port forwarding (ensure it is configured) and localhost. \"\n", " f\"Open your graph here at http://{hostname}:{port}/index.html\"\n", " )\n", "else:\n", " print(\n", " f\"Not using port forwarding. Use the IFrame below, or open your graph here \"\n", " f\"directly at http://{hostname}:{port}/index.html\"\n", " )\n", "\n", "if enable_iframe:\n", " from IPython.display import IFrame\n", "\n", " # Display the IFrame with the graph visualization\n", " print(f\"Displaying graph visualization in IFrame at http://{hostname}:{port}/index.html\")\n", " display(IFrame(src=f\"http://{hostname}:{port}/index.html\", width=\"100%\", height=\"800px\"))" ] }, { "cell_type": "code", "execution_count": 31, "id": "a4bdd56d", "metadata": { "execution": { "iopub.execute_input": "2026-07-30T19:07:55.164576Z", "iopub.status.busy": "2026-07-30T19:07:55.164373Z", "iopub.status.idle": "2026-07-30T19:07:55.627072Z", "shell.execute_reply": "2026-07-30T19:07:55.626094Z" }, "papermill": { "duration": 0.482651, "end_time": "2026-07-30T19:07:55.628491+00:00", "exception": false, "start_time": "2026-07-30T19:07:55.145840+00:00", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "server.stop()" ] }, { "cell_type": "markdown", "id": "7818f7c0", "metadata": { "papermill": { "duration": 0.018102, "end_time": "2026-07-30T19:07:55.665079+00:00", "exception": false, "start_time": "2026-07-30T19:07:55.646977+00:00", "status": "completed" }, "tags": [] }, "source": [ "### Next Steps and Future Extensions\n", "\n", "This notebook demonstrates the basic scaffolding for the CircuitTracerAdapter. The current implementation provides:\n", "\n", "1. **Basic Integration**: CircuitTracerAdapter integrates with Interpretune's session management\n", "2. **Configuration**: CircuitTracerConfig allows customization of attribution parameters\n", "3. **Protocol Support**: CircuitAnalysisBatchProtocol defines the interface for batch processing\n", "4. **Adapter Composition**: Seamless integration with other Interpretune adapters\n", "\n", "#### Future Extensions:\n", "\n", "1. **Full Implementation**: Complete the adapter methods to actually generate attribution graphs\n", "2. **Batch Processing**: Support for efficient batch attribution analysis\n", "3. **Advanced Analysis**: Integration with AnalysisOp for complex circuit analysis workflows\n", "4. **Visualization**: Built-in support for graph visualization and exploration\n", "5. **Caching**: Intelligent caching of attribution results for faster iteration\n", "6. **Model Support**: Extended support for different model architectures beyond GPT-2\n", "\n", "#### Resources:\n", "\n", "- [Circuit Tracer Documentation](https://github.com/jacobdunefsky/circuit-tracer)\n", "- [Interpretune Documentation](https://github.com/speediedan/interpretune)\n", "- [Attribution Methods Paper](https://arxiv.org/abs/2310.10348)\n", "\n", "This scaffold provides a solid foundation for building sophisticated circuit analysis workflows with Interpretune and Circuit Tracer." ] } ], "metadata": { "kernelspec": { "display_name": "it_latest (3.13.11)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.11" }, "papermill": { "default_parameters": {}, "duration": 64.437406, "end_time": "2026-07-30T19:07:58.605888+00:00", "environment_variables": {}, "exception": null, "input_path": "/home/speediedan/repos/interpretune/src/it_examples/notebooks/publish/attribution_analysis/attribution_analysis.ipynb", "output_path": "/home/speediedan/repos/interpretune/docs/notebook_artifacts/attribution_analysis/attribution_analysis.ipynb", "parameters": {}, "start_time": "2026-07-30T19:06:54.168482+00:00", "version": "2.7.0" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "017e0f2712c1466d84729ab1e1df423f": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "08d7fd581ad0453595156c0d6a8111b1": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0dc1b355c7fd4e0fa0ad2993da94cd2d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_7a5ef9b60acd4f609681da7d2a2f72f4", "placeholder": "​", "style": "IPY_MODEL_63b283a86f6d4fa1a7f7b6f8d66f438e", "tabbable": null, "tooltip": null, "value": " 0.00/0.00 [00:00<?, ?B/s]" } }, "14895550cfb74f7385fba1be60793c4b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "1787cca271184c8eb27abbaab3be300e": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "20px" } }, "179bbfce69834d009fda571a4865b548": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6733e0407ff74db697862d53fd1fad02", "IPY_MODEL_682012501eea4d8ba975e6cdce481276", "IPY_MODEL_94591cb337ad4470bf6c27bb4c0b581b" ], "layout": "IPY_MODEL_9d9d3ac199264392b1563e615693326c", "tabbable": null, "tooltip": null } }, "1a77c7f73d384f6daa897558da9cd3ff": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "1ae8bddcf1d94e20a5a6944b07287177": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_4dea4272331442878365bd6e12021cd2", "placeholder": "​", "style": "IPY_MODEL_e656151670744d32ae7bf813bad27140", "tabbable": null, "tooltip": null, "value": "Saving the dataset (1/1 shards): 100%" } }, "1bae58de72cf40d2856e6fcbc681cef1": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "234db4f92ffa49ac96e70a22212f3a26": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "241ad3a7a55c4bcf8f3fcd91981b9e75": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_78b4e2f7276645019440b3b0bb844547", "placeholder": "​", "style": "IPY_MODEL_ac9594db97774104b8f55acb742aee59", "tabbable": null, "tooltip": null, "value": " 288/288 [00:00<00:00, 3833.61it/s]" } }, "27fca6558abc43e0a44c2fa877e8fb62": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_c8d5382ef973447486479e1d23353ee0", "IPY_MODEL_2f63eca3544d4bde9cfc98d60a5fc8e9", "IPY_MODEL_f174ce229eb543e49d7b7fdd61f86f1b" ], "layout": "IPY_MODEL_290390e4e4c44b389764d08a13779dd4", "tabbable": null, "tooltip": null } }, "290390e4e4c44b389764d08a13779dd4": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "29687635da2548d9a8071a464ec7a2ba": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "298674ab9fb74b9baf48a584d2873649": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "2befc058d1424c43a9a1ed07de42f7fb": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2c33267623f046f9817bff2b51b26023": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "2f63eca3544d4bde9cfc98d60a5fc8e9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_e8b212a96bb3427fbe76d143ee8701e3", "max": 2490.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_8bb2b8e4e20f418cb5d8d925cc77dd1f", "tabbable": null, "tooltip": null, "value": 2490.0 } }, "32f947aa4f3846dfaec97ab718f78da9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_8efd78d2cf2d4282ac58ac2e918c27fc", "max": 26.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_9b17011fa4b0435db12ab20abf1d9ed6", "tabbable": null, "tooltip": null, "value": 26.0 } }, "330246bf76ad42eb864cc70e0b018cf4": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "33a87192f9974b90be54fa26780e1979": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_80f5071359244b4ab6bcad0ed90f083c", "placeholder": "​", "style": "IPY_MODEL_f2fe5376461e4ddf896ac48102ce590b", "tabbable": null, "tooltip": null, "value": "Map: 100%" } }, "33b70f63d23347f9a0a09109bafb4feb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_33a87192f9974b90be54fa26780e1979", "IPY_MODEL_4fa8730073b846db83650dff4923607f", "IPY_MODEL_89d41f073f4643d6ad8b2edc4c65fcc9" ], "layout": "IPY_MODEL_29687635da2548d9a8071a464ec7a2ba", "tabbable": null, "tooltip": null } }, "3c5a9f4ac1984337a32897668dac3473": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3f749af92e554cba8607e00d25d01ae2": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3fef6f0c0aac4e2691253c90293b44e8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "40c93f661208431cb93053b259782dc6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_f8ffa1c07a4f4b6d9413c519f3cc3dd7", "max": 3000.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_4dd57987493f44d1ba1160f4ce74f311", "tabbable": null, "tooltip": null, "value": 3000.0 } }, "40fb358ecfc0467290a7fcb6e7d58121": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "4357f236e12246c694d3894da9e27abc": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "45efd641e8d04c6fbff8567396528f9e": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4d09d35673b341a0bc0e678d3d8f1d20": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "4d9c529675c74448a298c3b1b8b87cee": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4dd57987493f44d1ba1160f4ce74f311": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "4dea4272331442878365bd6e12021cd2": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4fa8730073b846db83650dff4923607f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_4d9c529675c74448a298c3b1b8b87cee", "max": 3000.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_14895550cfb74f7385fba1be60793c4b", "tabbable": null, "tooltip": null, "value": 3000.0 } }, "5a82dea1901147269252d5bf406077ef": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_08d7fd581ad0453595156c0d6a8111b1", "placeholder": "​", "style": "IPY_MODEL_db59a1505cc9489d96b8d06210933f16", "tabbable": null, "tooltip": null, "value": "Saving the dataset (1/1 shards): 100%" } }, "5e600663a3654dfe823b2a990c2e9a43": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_80184934b038437199f7a33c33d72cf8", "IPY_MODEL_40c93f661208431cb93053b259782dc6", "IPY_MODEL_e94f9bd5c6034b359585c34981ad31cc" ], "layout": "IPY_MODEL_c27228b2cd7a400db7dc2bad5470a041", "tabbable": null, "tooltip": null } }, "63b283a86f6d4fa1a7f7b6f8d66f438e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "6733e0407ff74db697862d53fd1fad02": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_1bae58de72cf40d2856e6fcbc681cef1", "placeholder": "​", "style": "IPY_MODEL_a8cfbd3dda5a4b52b8272ce020887878", "tabbable": null, "tooltip": null, "value": "Map: 100%" } }, "682012501eea4d8ba975e6cdce481276": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_7e43d7da88dc4efc89673184f9de8b28", "max": 277.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_1a77c7f73d384f6daa897558da9cd3ff", "tabbable": null, "tooltip": null, "value": 277.0 } }, "6dd9a51230c94066aba8c88d629d0e84": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6e47a6be005e4eec9129f3a6e8786321": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "785dd58e0bc141439da572f51329a62c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "78b4e2f7276645019440b3b0bb844547": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7a5ef9b60acd4f609681da7d2a2f72f4": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7e16ac051aff4197941d7e5605ddd20d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_ccc91ffbb30e4a11886b960be5689dd5", "IPY_MODEL_32f947aa4f3846dfaec97ab718f78da9", "IPY_MODEL_a5f0ec59221b4f88b44729eec964cac8" ], "layout": "IPY_MODEL_017e0f2712c1466d84729ab1e1df423f", "tabbable": null, "tooltip": null } }, "7e43d7da88dc4efc89673184f9de8b28": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "80184934b038437199f7a33c33d72cf8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_3f749af92e554cba8607e00d25d01ae2", "placeholder": "​", "style": "IPY_MODEL_fa3ef44355854597afe5bbb3ff96b909", "tabbable": null, "tooltip": null, "value": "Saving the dataset (1/1 shards): 100%" } }, "80f5071359244b4ab6bcad0ed90f083c": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "811a665ef0d547879390fde0dcfb09a6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "82e5ec3274a841d0b6bad211dd4b8a1f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_5a82dea1901147269252d5bf406077ef", "IPY_MODEL_c222e63820334a7d979482091c5ce6e7", "IPY_MODEL_d59e0828c4bd42d291bd68f2605e811d" ], "layout": "IPY_MODEL_d4ec80130e1f44268ac6c061f6460697", "tabbable": null, "tooltip": null } }, "83e0b886cde94ce0ace3a14187cc017c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_2befc058d1424c43a9a1ed07de42f7fb", "max": 2490.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_dc03651ed3504e5e93ef01bbd86d0d30", "tabbable": null, "tooltip": null, "value": 2490.0 } }, "86a67621c1564f7a8638bf4e45255f48": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_4357f236e12246c694d3894da9e27abc", "placeholder": "​", "style": "IPY_MODEL_9220586248bd47e4b18bd94d79d6430b", "tabbable": null, "tooltip": null, "value": "Loading weights: 100%" } }, "8830ea90bd7d4b71a48f4694073d4c08": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_cf3ae4114b35495abf710122faa6f8b4", "IPY_MODEL_b74a01503f1646de8b315d4cd981e415", "IPY_MODEL_0dc1b355c7fd4e0fa0ad2993da94cd2d" ], "layout": "IPY_MODEL_330246bf76ad42eb864cc70e0b018cf4", "tabbable": null, "tooltip": null } }, "8933b0fe525a4367b1360207fd6facce": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "89d41f073f4643d6ad8b2edc4c65fcc9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_e977e167a457478484623e2b9b793486", "placeholder": "​", "style": "IPY_MODEL_e9336ddf350b4839b0976597a4228e93", "tabbable": null, "tooltip": null, "value": " 3000/3000 [00:00<00:00, 16348.49 examples/s]" } }, "8bb2b8e4e20f418cb5d8d925cc77dd1f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "8efd78d2cf2d4282ac58ac2e918c27fc": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9220586248bd47e4b18bd94d79d6430b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "94591cb337ad4470bf6c27bb4c0b581b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_f63b1b8d4be246d48e7fb36942dd7867", "placeholder": "​", "style": "IPY_MODEL_8933b0fe525a4367b1360207fd6facce", "tabbable": null, "tooltip": null, "value": " 277/277 [00:00<00:00, 7356.70 examples/s]" } }, "96880a106242442eac8204c684cde05a": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9714fc698c5e4efb938a20171a231085": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9ab5ad338dca4fdab71c49ddaa53f6eb": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9b17011fa4b0435db12ab20abf1d9ed6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9c1107885ffb462dabf1b54a4a3d33ab": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1ae8bddcf1d94e20a5a6944b07287177", "IPY_MODEL_83e0b886cde94ce0ace3a14187cc017c", "IPY_MODEL_f607014bc43a4b088f5381a8168ea3f8" ], "layout": "IPY_MODEL_45efd641e8d04c6fbff8567396528f9e", "tabbable": null, "tooltip": null } }, "9d9d3ac199264392b1563e615693326c": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a5f0ec59221b4f88b44729eec964cac8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_6e47a6be005e4eec9129f3a6e8786321", "placeholder": "​", "style": "IPY_MODEL_785dd58e0bc141439da572f51329a62c", "tabbable": null, "tooltip": null, "value": " 26/26 [00:00<00:00, 2311.79it/s]" } }, "a8cfbd3dda5a4b52b8272ce020887878": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "ac9594db97774104b8f55acb742aee59": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "b11c704df3e54f81a42babf632ca7867": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b74a01503f1646de8b315d4cd981e415": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_1787cca271184c8eb27abbaab3be300e", "max": 1.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_811a665ef0d547879390fde0dcfb09a6", "tabbable": null, "tooltip": null, "value": 0.0 } }, "bab4047606d747f384e8c6ec2acbab95": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9714fc698c5e4efb938a20171a231085", "max": 288.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_da6b531b77e94aa1b75701e812e440ad", "tabbable": null, "tooltip": null, "value": 288.0 } }, "c222e63820334a7d979482091c5ce6e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9ab5ad338dca4fdab71c49ddaa53f6eb", "max": 277.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_3fef6f0c0aac4e2691253c90293b44e8", "tabbable": null, "tooltip": null, "value": 277.0 } }, "c27228b2cd7a400db7dc2bad5470a041": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c8d5382ef973447486479e1d23353ee0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_cb61a354221f4e42842df5929ec44af8", "placeholder": "​", "style": "IPY_MODEL_4d09d35673b341a0bc0e678d3d8f1d20", "tabbable": null, "tooltip": null, "value": "Map: 100%" } }, "cb61a354221f4e42842df5929ec44af8": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ccc91ffbb30e4a11886b960be5689dd5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_96880a106242442eac8204c684cde05a", "placeholder": "​", "style": "IPY_MODEL_e0e00e18bebe480dac29339ce898fa17", "tabbable": null, "tooltip": null, "value": "Fetching 26 files: 100%" } }, "ce2e5bd8427a4b97962e5514e57ef68c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "cf3ae4114b35495abf710122faa6f8b4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_6dd9a51230c94066aba8c88d629d0e84", "placeholder": "​", "style": "IPY_MODEL_298674ab9fb74b9baf48a584d2873649", "tabbable": null, "tooltip": null, "value": "Download complete: " } }, "d4ec80130e1f44268ac6c061f6460697": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d57aeef29e2e4a0baf861014d5a87049": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d59e0828c4bd42d291bd68f2605e811d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_ff650267e39645159ccdd1ecc400804a", "placeholder": "​", "style": "IPY_MODEL_234db4f92ffa49ac96e70a22212f3a26", "tabbable": null, "tooltip": null, "value": " 277/277 [00:00<00:00, 30601.65 examples/s]" } }, "da6b531b77e94aa1b75701e812e440ad": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "db59a1505cc9489d96b8d06210933f16": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "dc03651ed3504e5e93ef01bbd86d0d30": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e0e00e18bebe480dac29339ce898fa17": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "e32e0164d469475892dc2e732c9a8f34": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_86a67621c1564f7a8638bf4e45255f48", "IPY_MODEL_bab4047606d747f384e8c6ec2acbab95", "IPY_MODEL_241ad3a7a55c4bcf8f3fcd91981b9e75" ], "layout": "IPY_MODEL_b11c704df3e54f81a42babf632ca7867", "tabbable": null, "tooltip": null } }, "e656151670744d32ae7bf813bad27140": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "e8b212a96bb3427fbe76d143ee8701e3": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e9336ddf350b4839b0976597a4228e93": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "e94f9bd5c6034b359585c34981ad31cc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_3c5a9f4ac1984337a32897668dac3473", "placeholder": "​", "style": "IPY_MODEL_2c33267623f046f9817bff2b51b26023", "tabbable": null, "tooltip": null, "value": " 3000/3000 [00:00<00:00, 173722.74 examples/s]" } }, "e977e167a457478484623e2b9b793486": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f174ce229eb543e49d7b7fdd61f86f1b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_d57aeef29e2e4a0baf861014d5a87049", "placeholder": "​", "style": "IPY_MODEL_40fb358ecfc0467290a7fcb6e7d58121", "tabbable": null, "tooltip": null, "value": " 2490/2490 [00:00<00:00, 11970.31 examples/s]" } }, "f2fe5376461e4ddf896ac48102ce590b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "f607014bc43a4b088f5381a8168ea3f8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_fb1b1b2f018c4a3aa2977152d91ea918", "placeholder": "​", "style": "IPY_MODEL_ce2e5bd8427a4b97962e5514e57ef68c", "tabbable": null, "tooltip": null, "value": " 2490/2490 [00:00<00:00, 146522.31 examples/s]" } }, "f63b1b8d4be246d48e7fb36942dd7867": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f8ffa1c07a4f4b6d9413c519f3cc3dd7": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fa3ef44355854597afe5bbb3ff96b909": { "model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null } }, "fb1b1b2f018c4a3aa2977152d91ea918": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ff650267e39645159ccdd1ecc400804a": { "model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }