peal.data.dataset_utils.parse_csv

peal.data.dataset_utils.parse_csv(data_dir, config, mode, key_type='idx', set_negative_to_zero=True, delimiter=',', raw_data=None)[source]

Parse a csv label table into per-sample tensors and the keys of one split.

The first row is the header (a leading row with fewer than two fields, e.g. a sample count, is skipped). Every other row is converted to a float tensor; fields that are not valid floats become -1.0. Three selection mechanisms follow, in order:

  1. If config.confounding_factors has two entries and a confounder probability or full_confounder_config is set, rows are subsampled by process_confounder_data_controlled.

  2. Otherwise all rows are kept (rows with has_mask != 1 are dropped when config.has_hints) and split either by config.split fractions after a seeded shuffle, or, if split has not exactly two entries, by a split column holding 0/1/2 for train/val/test.

  3. If config.spray_label_file is set, its SprayLabel column overwrites the confounder column; samples whose spray label is not 0 or 1 are dropped, and with config.spray_groups_balanced the four (target, confounder) groups are truncated to the smallest group size.

Parameters:

  • data_dir (str) – Path of the csv file (ignored when raw_data is given).

  • config (DataConfig) – Uses x_selection, confounding_factors, confounder_probability, full_confounder_config, has_hints, split, spray_label_file and spray_groups_balanced.

  • mode (str) – "train", "val" or "test"; any other value keeps all keys.

  • key_type (str, optional) – "idx" keys samples by row number; "name" keys them by the column named config.x_selection (or column 0) and drops that and all preceding columns from the tensor and from attributes.

  • set_negative_to_zero (bool, optional) – Clamp negative values to zero.

  • delimiter (str, optional) – Field separator.

  • raw_data (list of str, optional) – Already-read lines to use instead of opening data_dir.

Returns:

  • attributes (list of str) – Column names of the returned tensors.

  • data (dict) – Mapping key -> 1-D float tensor with one entry per attribute.

  • keys_out (list of str) – Keys belonging to mode, shuffled with random.seed(0).