-
Notifications
You must be signed in to change notification settings - Fork 10.5k
[AutoDiff] Initial Forward Mode AD Support #26057
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AutoDiff] Initial Forward Mode AD Support #26057
Conversation
04be590
to
0ca031b
Compare
40b3680
to
652a6d7
Compare
9835cb0
to
966c470
Compare
3c01ab8
to
5883668
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please resolve merge conflicts and rebase into a single commit?
EDIT: "Squash and merge" instead of rebasing is fine as well.
@swift-ci please test tensorflow |
1 similar comment
@swift-ci please test tensorflow |
051a924
to
4e60732
Compare
4e60732
to
1883ac1
Compare
For time going to run tests now to get this merged in as soon as possible assuming there are only NFC changes that need to be made |
@swift-ci please test tensorflow |
1 similar comment
@swift-ci please test tensorflow |
6007902
to
129a3d5
Compare
@swift-ci please test tensorflow |
1 similar comment
@swift-ci please test tensorflow |
|
||
ForwardModeTests.test("Protocols") { | ||
let inst = Linear(m: 5, b: -2) | ||
let (y1, diff1) = valueWithDifferential(at: 5) { x in inst.foo(x: x) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not testing differentiation through protocols at all because you calling a method with concrete Self
type Linear
.
To differentiate through a protocol requirement, define a generic function that takes an instance bound to a generic parameter that conforms to the protocol, and differentiate that.
func genericFoo<T: DiffReq>(_ t: T, _ x: Float) -> Float {
t.foo(x)
}
...
valueWithDifferential(at: 5) { x in genericFoo(inst, x) }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
discussed offline, will address this in the next PR which will support everything else
@swift-ci please test tensorflow |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Congrats on going through all the hurdles to finish this important step!
This PR extends on the previous work here on the
JVPEmitter
(#25954) to make it be able to generate SIL code for thedifferential
function, as well as more correctly implementing the body of the JVP.'generates a valid JVP and differential. I have yet to add support for control flow and generics. This PR also renames
PullbackInfo
toLinearMap
info since it has a lot of tangent mapping helpers thatJVPEmitter
can use when creating the differential.This is still early, so the JVP/Differential emission only runs with the flag
-Xllvm -run-jvp-generation
when running Swift. With it, you can differentiate basic Float functions with+ - / *
with novar
s like: