peal.data.dataset_utils.parse_json

peal.data.dataset_utils.parse_json(data_dir, config, mode, set_negative_to_zero=True)[source]

Parse a json label file of a confounder-controlled dataset.

The json maps sample ids to records with values (feature list), target and has_confounder. Each record becomes a tensor [*values, target].

Parameters:

  • data_dir (str) – Path of the json file.

  • config (DataConfig) – Needs confounding_factors (length 2) and either confounder_probability or full_confounder_config, plus num_samples and split.

  • mode (str) – "train", "val", "test" or anything else for all keys.

  • set_negative_to_zero (bool, optional) – Clamp negative feature values to zero.

Returns:

(data, keys_out) from process_confounder_data_controlled, where data maps string keys (the record index) to tensors. Returns None when the config does not describe a controlled confounder setup (no other branch is implemented).

Return type:

tuple or None