peal.sparse_dictionaries.msae_decomposition.MSAEDecompositionConfig¶

class peal.sparse_dictionaries.msae_decomposition.MSAEDecompositionConfig[source]¶

Configuration for MSAE (Matryoshka Sparse Autoencoder) decomposition.

Parameters:

clip_model (str, optional) – Encoder the pretrained checkpoint was trained on. Default "clip:ViT-L/14".
ckpt_url (str or None, optional) – URL of the pretrained MSAE weights; None skips the download and leaves the dictionary in trainable mode.
interp_url (str or None, optional) – URL of the concept-interpreter similarity matrix used to name each latent with a vocabulary entry.
local_dir (str, optional) – Download cache directory. Default "/tmp/msae_weights".
vocab_filename (str, optional) – Vocabulary file under peal/dependencies/MSAE/vocab.
n_components (int or None, optional) – Number of latents; overwritten by the checkpoint’s latent_dim when a pretrained model is loaded. Default 6144.
activation (str, optional) – Activation spec for training from scratch. Default "TopKReLU_64".
use_matryoshka (bool, optional) – Train a MatryoshkaAutoencoder instead of a plain Autoencoder.
lr – Hyper-parameters for MSAEDecomposition.fit().
epochs – Hyper-parameters for MSAEDecomposition.fit().
batch_size – Hyper-parameters for MSAEDecomposition.fit().

Methods

Attributes

ckpt_url: str | None = 'https://huggingface.co/WolodjaZ/MSAE/resolve/main/ViT-L_14/centered/6144_768_TopKReLU_64_RW_False_False_0.0_cc3m_ViT-L~14_train_image_2905936_768.pth'¶

interp_url: str | None = 'https://huggingface.co/WolodjaZ/MSAE/resolve/main/ViT-L_14/centered/Concept_Interpreter_6144_768_TopKReLU_64_RW_False_False_0.0_cc3m_ViT-L~14_train_image_2905936_768_disect_ViT-L~14_-1_text_20000_768.npy'¶