Fix RNN gradient accumulation.

[Shashi456]

partial fix for #518.

[rxwei]

Could you please add a unit test?

[Shashi456]

@rxwei what would a unit test for gradient accumulation look like?

[rxwei]

Just add a unit test for RNN. It's possible that we don't have one yet.

[eaplatanios]

You can check whether the gradient of the update matrix is non-zero for time steps before the last one I believe. The bug should make these gradients zero and this should be fixed now.

…

On Tue, Oct 1, 2019 at 1:50 PM Richard Wei ***@***.***> wrote: Just add a unit test for RNN. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#519?email_source=notifications&email_token=AAJ4EXAENQTUOKHXC2CH6JTQMOEWFA5CNFSM4I4ME2QKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEACETLA#issuecomment-537151916>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJ4EXALNDIODMI4UNFHZ53QMOEWFANCNFSM4I4ME2QA> .

[rxwei]

@Shashi456 Did you have a chance to update this PR?

[Shashi456]

@rxwei sorry for the delay, I'll write the rnn test in a day or 2.

@eaplatanios if you think these changes are better added together in #522, Then i'll close this PR.

[saeta]

Hi @Shashi456 ! Just following up here; do you think you can add a test or two? Thanks so much for this PR! -Brennan

[Shashi456]

@saeta, when we say unit test we mean an rnn gradient test? Sorry for the confusion

[marcrasi]

Hi, we'll merge this now to unblock #522. We're still interested in having a test, so I'll create a github issue for that and we can discuss details about the test there.