Skip to content
This repository was archived by the owner on Jul 1, 2023. It is now read-only.

Normalization layers fix (fixes #384 and #426). #428

Merged
merged 6 commits into from
Aug 9, 2019

Conversation

eaplatanios
Copy link
Contributor

This PR fixes the shape related issues in the normalization layers (i.e., fixes #384 and #426). The role of LayerNorm is unclear to me currently as I believe it's not really the same as the layer norm in the Python API. In either case, this PR fixes shape issues in a principled way. We can deal with what LayerNorm is supposed to be later.

cc @rxwei @BradLarson @mikowals

@eaplatanios
Copy link
Contributor Author

@BradLarson could you please verify if your use cases works fine with these changes?

Copy link
Contributor

@rxwei rxwei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's wait on @BradLarson's review too.

Copy link
Contributor

@BradLarson BradLarson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This passes tests on my end. We should still put in inference tests for BatchNorm at some point, but I can work on that.

@eaplatanios
Copy link
Contributor Author

Great, thanks @BradLarson!

@eaplatanios eaplatanios merged commit f218a34 into tensorflow:master Aug 9, 2019
@mikowals
Copy link
Contributor

mikowals commented Aug 9, 2019

@BradLarson I have an test that is part of #429. I will make it part of a separate pull request later today.

@BradLarson
Copy link
Contributor

@mikowals - Excellent, thanks. Better test coverage overall is definitely something we're interested in.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants