peal.teachers.web_feedback_teacher.WebFeedbackTeacher

class peal.teachers.web_feedback_teacher.WebFeedbackTeacher

Bases: TeacherInterface

Teacher that collects human verdicts through JSON files in a job directory.

Used by the web demo’s job worker in place of Human2ModelTeacher. Each call to get_feedback() is one round: the collages that need a verdict are written to feedback_request_<round>.json and the teacher blocks until feedback_response_<round>.json contains a complete set of valid verdicts.

Parameters:

• feedback_dir (str) – Directory shared with the web app; created if missing.

• poll_interval (float) – Seconds between checks for the response file.

• timeout_s (float or None) – Give up (TimeoutError) after this many seconds without a complete response; None waits forever.

• auto_verdict (str or None) – "true", "false" or "ood" answers every request without a person. Falls back to the PEAL_WEB_FEEDBACK_AUTO environment variable when None.

• min_confidence (float) – Counterfactuals whose target-class end confidence is below this value are answered "student not swapped!" without asking anyone.

• tracking_level (int) – Stored for interface compatibility; unused here.

• counterfactual_type (str) – "1sided" asks only about pairs where the student’s prediction on the original equals the label (others get "student incorrect!"); any other value asks about every swapped pair.

Variables:

round (int) – Index of the next feedback round; incremented on every get_feedback() call, including calls that need no verdict.

Raises:

ValueError – If auto_verdict is set to something other than the three verdicts.

Methods

get_feedback	Return one verdict per collage, asking the web app for the open ones.
request_path	Path of the request file the teacher writes for round round_idx.
response_path	Path of the response file the web app writes for round round_idx.

__init__(feedback_dir, poll_interval=2.0, timeout_s=None, auto_verdict=None, min_confidence=0.5, tracking_level=0, counterfactual_type='1sided')

Store the settings, resolve auto_verdict and create feedback_dir.

request_path(round_idx)

Path of the request file the teacher writes for round round_idx.

Parameters:

round_idx (int) – Feedback round.

Returns:

<feedback_dir>/feedback_request_<round_idx>.json.

Return type:

str

response_path(round_idx)

Path of the response file the web app writes for round round_idx.

Parameters:

round_idx (int) – Feedback round.

Returns:

<feedback_dir>/feedback_response_<round_idx>.json.

Return type:

str

get_feedback(collage_path_list, y_target_end_confidence_list, y_source_list, y_list, y_target_list=None, **kwargs)

Return one verdict per collage, asking the web app for the open ones.

Parameters:

• collage_path_list (list of str) – Paths of the contrastive collages, one per counterfactual pair.

• y_target_end_confidence_list (list of float) – Predictor confidence for the target class on each counterfactual.

• y_source_list (list of int) – Student prediction on each original image.

• y_list (list of int) – Ground-truth label of each original image.

• y_target_list (list of int, optional) – Target class of each counterfactual; passed through to the request.

• **kwargs – Ignored; accepted for compatibility with other teachers.

Returns:

Same length and order as collage_path_list. Entries are "true", "false" or "ood" for pairs a person (or auto_verdict) judged, "student incorrect!" when the student was wrong on the original (1-sided mode only) and "student not swapped!" when the end confidence is below min_confidence.

Return type:

list of str

Notes

Writes feedback_request_<round>.json with the items to judge and blocks in _wait_for_response() until the matching response file holds a valid verdict for every requested index. The round counter is advanced even when nothing has to be asked.