peal.web.jobs.Worker

class peal.web.jobs.Worker

Bases: Thread

Runs queued jobs one at a time as python run_didae.py –config ….

A daemon thread that polls the store, launches the next queued job as a subprocess with stdout/stderr appended to <job_dir>/log.txt, mirrors the [DiDAE] Step N markers of that log into the job’s stage, honours cancel_requested and finally runs peal.web.summarize.

Parameters:

• store (JobStore) – The store to poll and update.

• python (str, optional) – Interpreter for the subprocess; defaults to $PEAL_WEB_PYTHON or sys.executable.

• poll_interval (float) – Seconds between store polls and between log checks of a running job.

• env (dict, optional) – Extra environment variables for the subprocess, applied after PEAL_RUNS (default <root>/_peal_runs) and PYTHONUNBUFFERED.

Variables:

current (subprocess.Popen or None) – The running job’s process, for cancellation from outside.

Methods

check_accuracy	Run python -m peal.web.analysis --accuracy-only and wait for it.
command	The argv used to launch a job.
run	Thread body: run queued jobs until stop() is called.
run_dfr	Run python -m peal.web.dfr --solver=<solver> <job_dir> and wait.
run_job	Run one job to completion and record its outcome.
start_analysis	Launch python -m peal.web.analysis <job_dir> in the background.
stop	Ask the polling loop to exit after the current job.

Attributes

__init__(store, python=None, poll_interval=3.0, env=None)

stop()

Ask the polling loop to exit after the current job.

run()

Thread body: run queued jobs until stop() is called.

Any exception escaping run_job() marks that job failed with the exception text and the loop continues.

check_accuracy(job_dir, log_path, env=None, timeout_s=1800)

Run python -m peal.web.analysis --accuracy-only and wait for it.

Writes accuracy.json; failures are appended to log_path and do not stop the job.

run_dfr(job_dir, solver, log_path, env=None, timeout_s=3600)

Run python -m peal.web.dfr --solver=<solver> <job_dir> and wait.

Writes dfr.json and the corrected run/model.onnx; a failure is logged and leaves the uncorrected model in place.

start_analysis(job_dir, env=None)

Launch python -m peal.web.analysis <job_dir> in the background.

Writes analysis.log; the page polls for analysis.json. Does nothing when the analysis already ran or is running, or when the sweep has not written its results yet.

Parameters:

• job_dir (str) – Job directory.

• env (dict, optional) – Environment of the subprocess.

Returns:

The started process.

Return type:

subprocess.Popen or None

command(job_dir)

The argv used to launch a job.

Parameters:

job_dir (str) – Job directory holding config.yaml.

Returns:

[python, <project>/run_didae.py, "--config", <job_dir>/config.yaml].

Return type:

list of str

run_job(job_id)

Run one job to completion and record its outcome.

Marks the job running, launches command() from the project directory with the log appended to log.txt, monitors it until it exits or is cancelled, then calls peal.web.summarize.summarize on the job directory (failures are logged, not raised) and sets the final status: finished on exit code 0, failed otherwise, or just the finished timestamp when the job was cancelled.

Parameters:

job_id (str) – Job id.