peal.sparse_dictionaries.sae_evaluation

Ground-truth evaluation of sparse dictionaries (SAE / MSAE / SVD-filtered).

Given a fitted dictionary with an encode method, activations x and attribute labels y, run_sae_eval() matches every binary attribute to one latent (Hungarian assignment on Pearson correlation, optionally also on the achievable F1), picks a threshold per attribute, and reports coverage, F1, AUC and average precision, in-sample and on an optional held-out split. Continuous (“dense”) factors are scored by linear regression from their matched latent. Results go to sae_evaluation_report.txt, sae_evaluation_results.pkl and matching_matrix_diagonalized.png in base_path and can be forwarded to W&B or TensorBoard. The module also carries the CelebA attribute consistency table used to define the “well-defined” attribute subsets.

Functions

compute_component_f1_scores	Compute F1 scores for the first n_components SAE components against best-matching ground truth binary attributes.
log_evaluation_to_tensorboard	Logs all ground-truth attribute matching metrics and the diagonalized bipartite matching matrix plot to TensorBoard.
log_evaluation_to_wandb	Logs all ground-truth attribute matching metrics, per-attribute F1 scores, and the diagonalized bipartite matching matrix plot to Weights & Biases.
plot_diagonalized_matching_matrix	Plots bipartite correlation matrix ordered by best match on X axis so the primary matches form a clean diagonal.
run_sae_eval	Full SAE evaluation for superposition datasets.