Does truncated_bptt_steps have any effect on the eval/test phases?

[colllin]

I’m using the auto_scale_batch_size feature of the trainer, and specifying the flag truncated_bptt_steps=32. What I’m seeing is that that batch size scaler settles on batch_size=16, but then during the training phase, I’m seeing GPU memory usage at ~2.2GB on a device with ~12GB of GPU memory. My dataset’s sequence len (time dimension) is currently configured to 256. It seems like the selected batch_size might be governed by a non-truncated sequence in the validation loop.

Am I right about what’s going on? If so, then would it be reasonable to also implement truncated_bptt_steps to take effect in the evaluation loop? If so... should I contribute this and copy the training-side implementation and tests?