peal.data.dataloaders

Dataloader construction and mixing for PEAL training loops.

get_dataloader and create_dataloaders_from_datasource turn a config plus a dataset path, dataset tuple or dataloader tuple into the train/val/test dataloaders used by trainers and adaptors. DataloaderMixer is the training-side wrapper that samples batches from several dataloaders with priorities (e.g. original data plus counterfactuals appended by CFKD), supports concatenated class-balanced batches and epochs of a fixed number of steps; DataStack and create_class_ordered_batch build small class-ordered batches for visualizations.

Functions

create_class_ordered_batch	Build a small batch with two consecutive samples of every class.
create_dataloaders_from_datasource	Create the train/val/test dataloaders for a config from a datasource.
get_dataloader	Wrap a PEAL dataset in a DataLoader (and a mixer for training).
resolve_num_workers	Worker processes for the loaders get_dataloader builds.

Classes

DataIterator	Iterator over a DataloaderMixer yielding a fixed number of batches.
DataStack	Per-class FIFO buffer of samples drawn from a dataset or dataloader.
DataloaderMixer	Dataloader that samples batches from several dataloaders.
WeightedDataloaderList	A list of DataLoader objects with associated sampling weights.