peal.teachers.llm2model_teacher.LLM2ModelTeacher

class peal.teachers.llm2model_teacher.LLM2ModelTeacher[source]

Bases: TeacherInterface

Teacher that labels counterfactual collages with a headless Claude call.

Collages are staged into a temporary directory, described in one prompt and sent to claude -p --output-format json; the reply must be a JSON array with one of "true", "false" or "ood" per collage (anything else becomes "false"). get_feedback returns the same sentinel strings the human GUI produces for collages it does not show.

Parameters:

  • dataset (optional) – Kept for interface parity with Human2ModelTeacher; not used.

  • confounder_name (str or None) – Human-readable name of the spurious feature put into the prompt.

  • target_name (str or None) – Human-readable name of the class-defining feature put into the prompt.

  • model (str or None) – Passed as --model to the CLI when set.

  • claude_bin (str) – Executable to run ("claude" on PATH by default).

  • tracking_level (int) – Kept for interface parity; not used.

  • counterfactual_type (str) – Kept for interface parity; not used.

  • batch_size (int) – Collages per CLI call.

  • timeout_s (int) – Seconds before a CLI call is aborted (counts as a failed batch).

  • stage_dir (str or None) – Directory the temporary staging folders are created in; None uses the system temp dir.

Methods

get_feedback

Mirror of Human2ModelTeacher.get_feedback: one verdict per collage.
__init__(dataset=None, confounder_name=None, target_name=None, model=None, claude_bin='claude', tracking_level=0, counterfactual_type='1sided', batch_size=16, timeout_s=300, stage_dir=None)[source]

Store the prompt names, CLI options and batching parameters.

get_feedback(collage_path_list, y_target_end_confidence_list, y_source_list, y_list, **kwargs)[source]

Mirror of Human2ModelTeacher.get_feedback: one verdict per collage.

Only collages whose counterfactual actually swapped the class (confidence >= 0.5) AND where the student was originally correct are shown to the LLM; the rest get the same sentinel labels the human GUI produces.