Why optimizer_zero_grad() is behind backward()

[AliceNEET]

In PL, the whole hook is in https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_module.html?#hooks.
When I see code template about pytorch, optimizer_zero_grad() is in front of backward(). Why ?

[akihironitta]

@AliceNEET Hi, we recently changed the order from backward, step, zero_grad to zero_grad, backward, step in #6147, so if you're using 1.2.2 or later, zero_grad is called before backward. I'll update the docs in no time to reflect the change.

[AliceNEET]

Thanks !!