peal.data.dataset_factory.get_datasets

peal.data.dataset_factory.get_datasets(config, base_dir=None, task_config=None, return_dict=False, test_config=None, data_dir=None)[source]

Instantiate the train, validation and test datasets of a data config.

Parameters:

  • config (DataConfig or str) – Data config object or path of its yaml (resolved by load_yaml_config). Consumed keys: dataset_path, input_type, output_type, dataset_class, invariances, crop_size, input_size, normalization, split, diffusion_augmented, batch_wise_augmentation and the diffusion augmentation keys generator, sampling_time_fraction, num_discretization_steps.

  • base_dir (str, optional) – Root directory of the dataset; defaults to config.dataset_path.

  • task_config (TaskConfig, optional) – Stored on every returned dataset as task_config.

  • return_dict (bool, default False) – Passed through to the dataset constructors.

  • test_config (DataConfig or str, optional) – Separate config for the test split; defaults to config.

  • data_dir (str, optional) – Passed through to the train / val dataset constructors.

Returns:

(train_data, val_data, test_data). When test_config.split is [x, 1.0] the validation dataset object is reused as test set. Each dataset gets normalization and task_config attributes.

Return type:

tuple of PealDataset

Raises:

ValueError – If dataset_class is unknown and the input_type / output_type combination has no default dataset class.

Notes

Augmentations are selected by name from config.invariances: circular_cut, rotation, rotation10, hflipping, vflipping, random_resize10/20/50, color_jitter, sharpness, blur, crop, horizontalflip. An empty normalization list is filled in place with the per-channel mean and std of the training split.