Skip to content

[AutoDiff] add some @derivative(of:) tests #28785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 18, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions include/swift/AST/AutoDiff.h
Original file line number Diff line number Diff line change
Expand Up @@ -491,12 +491,18 @@ template<typename T> struct DenseMapInfo;
template<> struct DenseMapInfo<AutoDiffConfig> {
static AutoDiffConfig getEmptyKey() {
auto *ptr = llvm::DenseMapInfo<void *>::getEmptyKey();
// The `derivativeGenericSignature` component must be `nullptr` so that
// `getHashValue` and `isEqual` do not try to `getCanonicalSignature()` on
// an invalid pointer.
return {static_cast<IndexSubset *>(ptr), static_cast<IndexSubset *>(ptr),
nullptr};
}

static AutoDiffConfig getTombstoneKey() {
auto *ptr = llvm::DenseMapInfo<void *>::getTombstoneKey();
// The `derivativeGenericSignature` component must be `nullptr` so that
// `getHashValue` and `isEqual` do not try to `getCanonicalSignature()` on
// an invalid pointer.
return {static_cast<IndexSubset *>(ptr), static_cast<IndexSubset *>(ptr),
nullptr};
}
Expand Down
27 changes: 27 additions & 0 deletions test/AutoDiff/Serialization/derivative_attr.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// RUN: %empty-directory(%t)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: #28781 added a a file of the same name (test/AutoDiff/Serialization/derivative_attr.swift) to master branch. It covers the functionality added in this test: @derivative attribute serialization for init and subscript.


So far, AutoDiff tests on master branch and tensorflow branch have been kept separate:

  • master branch tests are all categorized in test/AutoDiff/XXX subdirectories, like test/AutoDiff/Parse.
  • tensorflow branch tests exist in test/AutoDiff at the top-level. Some multi-file tests include files in test/AutoDiff/Inputs and/or their own subdirectories, like test/AutoDiff/silgen_thunking/main.swift and swift/test/AutoDiff/Inputs/silgen_thunking_other_module.swift.

I think separation is nice because it makes it clear which tests have been upstreamed and prevents merge conflicts.

How about moving all non-upstreamed tensorflow branch tests to a test/AutoDiff/tensorflow subdirectory? cc @rxwei @bgogul

As AutoDiff code is upstreamed, tests can be polished and moved to test/AutoDiff/xxx, so eventually test/AutoDiff/tensorflow becomes empty. On tensorflow branch, tests should never be added to test/AutoDiff outside of test/AutoDiff/tensorflow. On master branch, test/AutoDiff/tensorflow does not exist and tests should be added to test/AutoDiff/XXX.

I'm happy to start this after this PR is merged!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A separate folder sounds good, but perhaps instead put upstreamed ones in test/AutoDiff/upstreamed/?

// RUN: %target-swift-frontend %s -emit-module -parse-as-library -o %t
// RUN: llvm-bcanalyzer %t/derivative_attr.swiftmodule | %FileCheck %s -check-prefix=BCANALYZER
// RUN: %target-sil-opt -disable-sil-linking -enable-sil-verify-all %t/derivative_attr.swiftmodule -o - | %FileCheck %s

// BCANALYZER-NOT: UnknownCode

struct DerivativeOfSpecialMethods: Differentiable {
var x: Float

init(_ x: Float) { self.x = x }
subscript(_ i: Int) -> Float { x }

// CHECK: @derivative(of: init, wrt: x)
// CHECK-NEXT: static func dInit(_ x: Float)
@derivative(of: init(_:))
static func dInit(_ x: Float) -> (value: Self, pullback: (TangentVector) -> Float) {
fatalError()
}

// CHECK: @derivative(of: subscript, wrt: self)
// CHECK-NEXT: func dSubscript(_ i: Int)
@derivative(of: subscript(_:))
func dSubscript(_ i: Int) -> (value: Float, pullback: (Float) -> TangentVector) {
fatalError()
}
}
30 changes: 30 additions & 0 deletions test/AutoDiff/derivative_attr_type_checking.swift
Original file line number Diff line number Diff line change
Expand Up @@ -532,3 +532,33 @@ struct StoredProperty: Differentiable {
(stored, { _ in .zero })
}
}

// When the generic signature was not considered while calculating the actual pullback type, the
// typechecker did not realize that `T.TangentVector == Float`, and therefore it complained that
// "'pullback' does not have expected type '(Float) -> (Float)'".
// Users were able to work around this by setting the pullback type to `(Float) -> Float`.
func genericSignatureConsidered<T>(_ x: T) -> T { fatalError() }
@derivative(of: genericSignatureConsidered)
func dGenericSignatureConsidered<T>(_ x: T)
-> (value: T, pullback: (T.TangentVector) -> T.TangentVector)
where T: Differentiable, T.TangentVector == Float
{
fatalError()
}

// When the generic signature was not considered while calculating the actual pullback type,
// the typechecker complained that the pullback type was not correct, and there was no pullback type
// that users could specify to satisfy the typechecker.
struct Wrapper<T: AdditiveArithmetic & Equatable>: AdditiveArithmetic, Equatable {
var t: T
init(_ t: T) { self.t = t }
}
extension Wrapper: Differentiable where T: Differentiable, T == T.TangentVector {
typealias TangentVector = Wrapper<T.TangentVector>
}
extension Wrapper where T: Differentiable, T == T.TangentVector {
@derivative(of: init(_:))
static func dInit(_ t: T) -> (value: Self, pullback: (Wrapper<T>.TangentVector) -> (T)) {
fatalError()
}
}
20 changes: 20 additions & 0 deletions test/AutoDiff/derivative_registration.swift
Original file line number Diff line number Diff line change
Expand Up @@ -156,4 +156,24 @@ DerivativeRegistrationTests.testWithLeakChecking("DerivativeGenericSignature") {
expectEqual(1000, dx)
}

// When non-canonicalized generic signatures are used to compare derivative configurations, the
// `@differentiable` and `@derivative` attributes create separate derivatives, and we get a
// duplicate symbol error in TBDGen.
public protocol RefinesDifferentiable: Differentiable {}
extension Float: RefinesDifferentiable {}
@differentiable(where T: Differentiable, T: RefinesDifferentiable)
public func nonCanonicalizedGenSigComparison<T>(_ t: T) -> T { t }
@derivative(of: nonCanonicalizedGenSigComparison)
public func dNonCanonicalizedGenSigComparison<T: RefinesDifferentiable>(_ t: T)
-> (value: T, pullback: (T.TangentVector) -> T.TangentVector)
{
(t, { _ in T.TangentVector.zero })
}
DerivativeRegistrationTests.testWithLeakChecking("NonCanonicalizedGenericSignatureComparison") {
let dx = gradient(at: Float(0), in: nonCanonicalizedGenSigComparison)
// Expect that we use the custom registered derivative, not a generated derivative (which would
// give a gradient of 1).
expectEqual(0, dx)
}

runAllTests()