Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Use Epochs to load CIFAR10 #495

Merged
merged 1 commit into from
May 7, 2020
Merged

Conversation

sgugger
Copy link
Contributor

@sgugger sgugger commented May 7, 2020

This PR starts the work of removing Batcher and use Epochs instead.

To avoid one gigantic PR, I have created new protocol named ImageClassificationData that I use and kept the old cifar model in OldCifar10 (to use it in the benchmarks). The protocol ImageClassificationData can be renamed once everything is done and the current ImageClassificationDataset is removed.
The OldCifar10 file can be removed when MNIST and the benchmarks have been converted to use Epochs.

New with this implementation:

  • the shuffle of the training set can be seeded;
  • the pixel values of the tensors are always scaled to [0, 1] even if normalize is false;
  • the tensors are created on the fly from the raw UInt8 data to avoid launching several kernels of Tf-runtime

@sgugger sgugger requested a review from BradLarson May 7, 2020 16:28
Copy link
Contributor

@BradLarson BradLarson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense to transition parts over. Once MNIST is done, we should be able to migrate the benchmarks across and see how this impacts performance.

@sgugger sgugger merged commit e275188 into tensorflow:master May 7, 2020
@sgugger sgugger deleted the cifar_epochs branch May 7, 2020 19:35
@sgugger sgugger mentioned this pull request May 7, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants