interpretune.extensions#

class interpretune.extensions.DebugGeneration[source]#

Give user-provided callbacks with the ability to connect to another user-provided callback.

This resolution logic is provided in order to avoid callback-dependent trainer attributes (e.g. trainer.finetuningscheduler_callback)

Arguments.

DEFAULT_OUTPUT_DATACLS#

alias of GenerateDecoderOnlyOutput

chat_debug_sequences(sequences=None, format=None)[source]#

Render debug sequences through the configured chat template.

Parameters:

  • sequences (List | None, optional) – Debug prompts to render; defaults to the extension’s default sanity-check questions when None.

  • format (str | None, optional) – Model-variant prompt-config pattern (e.g. "llama3-chat").

Returns:

Chat-formatted sequences ready for debug_generate_batch/debug_generate_serial.

Return type:

List

Usage:

# for example, using the llama3 chat format, you want to have input tokenized with sys and inst metadata
# to do so with some reasonable default questions as a sanity check and in batch mode:
self.debug_lm.debug_generate_batch(self.debug_lm.chat_debug_sequences()
# to narrow the problem space, using serial inference (non-batch mode) for a list of strings can be useful
self.debug_lm.debug_generate_serial(self.debug_lm.chat_debug_sequences())
# to override the defaults (both questions and current `max_new_tokens` config)
# you can also specify a specific model variant pattern for a given prompt config, e.g.
# `format="llama3-chat"`
self.debug_lm.debug_generate_batch(self.debug_lm.chat_debug_sequences(format='llama3-chat', sequences=[
'What is the color of a cloudless sky?', 'How many days are in a year?']),
gen_config_override={"max_new_tokens": 25})
collect_prompt_debug_info(raw_sequences, *, rendered_sequences=None, add_special_tokens=False)[source]#

Collect prompt text and tokenization metadata for debugging generation paths.

Return type:

List[Dict[str, Any]]

Parameters:

  • raw_sequences (List | str)

  • rendered_sequences (List | str | None)

  • add_special_tokens (bool)

debug_sequences(sequences=None)[source]#

_summary_

Parameters:

sequences (List | None, optional) – _description_. Defaults to None.

Returns:

_description_

Return type:

List

Usage: `python # one can use this method to probe non-chat fine-tuned models (just the raw sequences, no SYS # or INST metadata) self.debug_lm.debug_generate_batch(self.debug_lm.debug_sequences('My single custom sequence'), gen_config_override={"max_new_tokens": 25}) `

class interpretune.extensions.DebugLMConfig(*, enabled=False, debug_raw_preds=None, debug_raw_labels=None, debug_raw_sequences=None, raw_debug_sequences=<factory>)[source]#
Parameters:

  • enabled (bool)

  • debug_raw_preds (ndarray | None)

  • debug_raw_labels (ndarray | None)

  • debug_raw_sequences (list[str] | None)

  • raw_debug_sequences (list)

class interpretune.extensions.DefaultMemHooks(*values)[source]#
class interpretune.extensions.MemProfilerCfg(*, enabled=False, cuda_allocator_history=False, schedule=<factory>, save_dir=None, enabled_funcs=<factory>, enable_memory_hooks=True, enable_saved_tensors_hooks=True, memory_hooks=<factory>, saved_tensors_funcs=<factory>, save_hook_attrs=<factory>, retain_hooks_for_funcs=<factory>)[source]#
Parameters:
class interpretune.extensions.MemProfilerFuncs(*, cuda=<factory>, cpu=<factory>, cuda_allocator_history=<factory>)[source]#
Parameters:
class interpretune.extensions.MemProfilerHooks(*, pre_forward_hooks=<factory>, post_forward_hooks=<factory>, reset_state_hooks=<factory>)[source]#
Parameters:
class interpretune.extensions.MemProfilerSchedule(*, warmup_steps=0, max_step=None)[source]#
Parameters:

  • warmup_steps (int)

  • max_step (int | None)

interpretune.extensions.NeuronpediaConfig#

alias of object

interpretune.extensions.NeuronpediaIntegration#

alias of object