peal.teachers.cluster_teacher.ClusterTeacher

class peal.teachers.cluster_teacher.ClusterTeacher[source]

Bases: TeacherInterface

Teacher that asks a human for one verdict per cluster of counterfactuals.

The constructor wipes and recreates a static directory in the current working directory, starts a Flask app on a daemon thread bound to 0.0.0.0 and the first free port at or above port, and keeps a DataStore that the request handler and get_feedback() share.

Parameters:

  • port (int) – First port to try; incremented until a free one is found.

  • dataset (peal.data.datasets.ImageDataset) – Used to render contrastive collages when tracking_level >= 5.

  • tracking_level (int) – Verbosity/artifact level of the surrounding CFKD run.

  • counterfactual_type (str) – "1sided" marks counterfactuals whose factual was misclassified as "student originally wrong!" instead of a verdict.

Methods

get_feedback

Block until a person has judged every cluster and expand the verdicts.
__init__(port, dataset, tracking_level=0, counterfactual_type='1sided')[source]

Start the feedback web server; see the class docstring.

Raises:

ImportError – If the optional flask dependency is not installed.

get_feedback(num_clusters, **kwargs)[source]

Block until a person has judged every cluster and expand the verdicts.

collage_path_list is split into num_clusters equally sized consecutive chunks, the collages are copied into static/ and served one cluster at a time. The method polls once per second until as many verdicts as clusters have been submitted, then repeats each cluster’s verdict for every counterfactual in that cluster.

Parameters:

  • num_clusters (int) – Number of clusters the counterfactuals are grouped into.

  • **kwargs – The CFKD tracked_values: collage_path_list, x_counterfactual_list, x_list, y_list, y_source_list, y_target_list, y_target_start_confidence_list, y_target_end_confidence_list and base_dir.

Returns:

One entry per counterfactual: "true", "false" or "ood" from the human, overridden by "student originally wrong!" (1-sided mode, factual misclassified) or "student not swapped!" (target confidence below 0.5). At tracking_level >= 5 a contrastive collage is additionally written to base_dir.

Return type:

list of str