peal.visualization.create_didae_table1.create_table¶

peal.visualization.create_didae_table1.create_table(runs_var_name, data_dict=None)[source]¶

Collect the table cells for one seed directory.

Parameters:

runs_var_name (str) – Environment variable naming the runs directory ("PEAL_RUNS" or "PEAL_RUNS<n>"). If it is unset but PEAL_RUNS is, the directory $PEAL_RUNS<n> is used for PEAL_RUNS1..3.
data_dict (dict, optional) – Dictionary to fill; a new one is created when None.

Returns:

data_dict[dataset][method][column] = (value, from_fallback) for the columns in metrics. Values are percentages except CF/s. Missing configs, base_dir entries or log directories leave the nan placeholders in place.

Return type:

dict

Notes

The tags actually read differ from the metrics mapping: NAFR comes from validation_flip_rate_distilled, NA from validation_non_adversarial_rate, Unbiasedness from validation_unbiasedness (all at step 0) and Gain from gain at step 1. metrics only supplies the column names.