Skip to content

Commit 2b198ed

Browse files
authored
Lower learning rate for TF cifar integ test (#265)
We've been seeing a lot of errors that say failure due to NaN loss. One suggestion from https://stackoverflow.com/questions/40050397/deep-learning-nan-loss-reasons is to use a lower number for learning rate.
1 parent 7dd06ca commit 2b198ed

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

tests/data/cifar_10/source/resnet_cifar_10.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,8 @@
3333
BATCH_SIZE = 1
3434

3535
# Scale the learning rate linearly with the batch size. When the batch size is
36-
# 128, the learning rate should be 0.1.
37-
_INITIAL_LEARNING_RATE = 0.1 * BATCH_SIZE / 128
36+
# 128, the learning rate should be 0.05.
37+
_INITIAL_LEARNING_RATE = 0.05 * BATCH_SIZE / 128
3838
_MOMENTUM = 0.9
3939

4040
# We use a weight decay of 0.0002, which performs better than the 0.0001 that

0 commit comments

Comments
 (0)