peal.teachers.virelay_teacher.create_dataset

peal.teachers.virelay_teacher.create_dataset(dataset_file_path, samples_shape, number_of_samples, dataloader)[source]

Creates a new dataset HDF5 file.

The file gets a float32 data dataset of shape (number_of_samples, *samples_shape) and a uint16 label dataset, both filled by iterating once over dataloader.

Parameters:

  • dataset_file_path (str) – The path to the dataset HDF5 file that is to be created.

  • samples_shape (tuple) – The shape of one sample (without the batch axis).

  • number_of_samples (int) – The number of samples in the dataset.

  • dataloader (torch.utils.data.DataLoader) – Yields (X, y) batches that are written in order.

Returns:

The still open file handle of the dataset database.

Return type:

h5py.File