Skip to content

test that various pointer types hit UTF8 decoding fast path #21743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 12, 2019

Conversation

weissi
Copy link
Contributor

@weissi weissi commented Jan 9, 2019

In #21706 I wanted to add a bunch of benchmarks to show that String(decoding:as:) is fast for all Unsafe(Mutable)(Raw)BufferPointer(<UInt8>) variants.
But @milseman & @eeckstein had a better idea: Why not add a test to show that the code-gen emits good code for all those cases.

@weissi
Copy link
Contributor Author

weissi commented Jan 9, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

swift-ci commented Jan 9, 2019

Build failed
Swift Test Linux Platform
Git Sha - 1854ab59a4461ec728d0dc8eb591c1c1f766328b

Copy link
Member

@milseman milseman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good so far

// CHECK: {{.*}} = call swiftcc {{.*}} @"$sSS18_fromUTF8Repairing{{.*}}
// CHECK-NOT: {{.*}} = call swiftcc {{.*}}_fromCodeUnits{{.*}}
// CHECK: ret

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. @eeckstein, is there a more robust way to do this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I wasn't happy with this either but thought better than nothing... @eeckstein ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can make the CHECK-NOT more robust by just using:

// CHECK-NOT: _fromCodeUnits

// CHECK: ret
public func decodeUBPSliceAsUTF8(_ ptr: Slice<UnsafeBufferPointer<UInt8>>) -> String {
return String(decoding: ptr, as: Unicode.UTF8.self)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you file a bug and directly link it in the comment? Unless the test is XFAIL, I think it's better to disable the check than to test the undesired behavior. You can say something like:

// CHECK-LABEL: define swiftcc {{.*}}decodeUBPSliceAsUTF8{{.*}}
//
// ... comment linking and explaining bug, to be reenabled when fixed
// xCHECK-NOT: {{.*}} = call swiftcc {{.*}}_fromCodeUnits{{.*}}
// xCHECK: {{.*}} = call swiftcc {{.*}} @"$sSS18_fromUTF8Repairing{{.*}}
//
// CHECK: ret

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say checking the undesired behavior is better because the test'll tell you if it changes for the better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@milseman up to you which way, I did file a bug

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll go with @jrose-apple's suggestion, but make it a little more explicit that you're checking against current behavior rather than desired behavior. Something like:

// FIXME(https://bugs.swift.org/browse/SR-9700):  _fromUTF8Repairing instead of _fromCodeUnits.
// Currently, Slice<UBP> doesn't actually work. Swap checked function below when resolved.

And then probably link to this specific test in the bug itself, so the resolver knows where to expect a change.

@milseman
Copy link
Member

Not sure why, but that test fails on Linux:

/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/swift/test/SILOptimizer/utf8_decoding_fastpath.swift:13:17: error: CHECK-LABEL: expected string not found in input
13:22:03 // CHECK-LABEL: define swiftcc {{.*}}s22utf8_decoding_fastpath15decodeUBPAsUTF8ySSSRys5UInt8VGF{{.*}}
13:22:03                 ^
13:22:03 <stdin>:1:1: note: scanning from here
13:22:03 ; ModuleID = '-'
13:22:03 ^
13:22:03 <stdin>:58:75: note: possible intended match here
13:22:03 @llvm.used = appending global [8 x i8*] [i8* bitcast ({ i64, %swift.bridge* } (i64, i64)* @"$s22utf8_decoding_fastpath15decodeUBPAsUTF8ySSSRys5UInt8VGF" to i8*), i8* bitcast ({ i64, %swift.bridge* } (i64, i64)* @"$s22utf8_decoding_fastpath16decodeUMBPAsUTF8ySSSrys5UInt8VGF" to i8*), i8* bitcast ({ i64, %swift.bridge* } (i64, i64)* @"$s22utf8_decoding_fastpath16decodeURBPAsUTF8ySSSWF" to i8*), i8* bitcast ({ i64, %swift.bridge* } (%Ts28__ContiguousArrayStorageBaseC*)* @"$s22utf8_decoding_fastpath17decodeArrayAsUTF8ySSSays5UInt8VGF" to i8*), i8* bitcast ({ i64, %swift.bridge* } (i64, i64)* @"$s22utf8_decoding_fastpath17decodeUMRBPAsUTF8ySSSwF" to i8*), i8* bitcast ({ i64, %swift.bridge* } (i64, i64, i64, i64)* @"$s22utf8_decoding_fastpath20decodeUBPSliceAsUTF8ySSs5SliceVySRys5UInt8VGGF" to i8*), i8* bitcast (i16* @__swift_reflection_version to i8*), i8* getelementptr inbounds ([12 x i8], [12 x i8]* @_swift1_autolink_entries, i32 0, i32 0)], section "llvm.metadata"
13:22:03       

@jrose-apple
Copy link
Contributor

jrose-apple commented Jan 10, 2019

I think there's an additional protected on Linux, because of hidden-visibility-by-default or something.

@weissi
Copy link
Contributor Author

weissi commented Jan 18, 2019

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 1854ab59a4461ec728d0dc8eb591c1c1f766328b

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 1854ab59a4461ec728d0dc8eb591c1c1f766328b

@weissi
Copy link
Contributor Author

weissi commented Jan 18, 2019

Thanks @jrose-apple / @milseman , tests are now passing.

@weissi
Copy link
Contributor Author

weissi commented Feb 1, 2019

@swift-ci please test

@swift-ci
Copy link
Contributor

swift-ci commented Feb 1, 2019

Build failed
Swift Test OS X Platform
Git Sha - 8a9e9f66189cf5bb61192f78eadad59c65e53dc2

@weissi
Copy link
Contributor Author

weissi commented Feb 1, 2019

so I think this failure is suprious:

09:15:41 FAIL: Swift(iphonesimulator-i386) :: stdlib/KVOKeyPaths.swift (7437 of 11712)
09:15:41 
******************** TEST 'Swift(iphonesimulator-i386) :: stdlib/KVOKeyPaths.swift' FAILED ********************
09:15:41 Script:
09:15:41 --
09:15:41 : 'RUN: at line 1';   rm -rf "/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/KVOKeyPaths.swift.tmp" && mkdir -p "/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/KVOKeyPaths.swift.tmp" && xcrun --toolchain default --sdk '/Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk' /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/bin/swiftc -target i386-apple-ios7.0  -module-cache-path '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/swift-test-results/i386-apple-ios7.0/clang-module-cache' -F '/Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks'  -swift-version 4   -module-cache-path '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/swift-test-results/i386-apple-ios7.0/clang-module-cache' /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/stdlib/KVOKeyPaths.swift -o /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/KVOKeyPaths.swift.tmp/a.out -module-name main && codesign -f -s - /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/KVOKeyPaths.swift.tmp/a.out &&xcrun --toolchain default --sdk '/Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk' simctl spawn 'iPhone 5'  /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/KVOKeyPaths.swift.tmp/a.out | '/usr/bin/python' '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/utils/PathSanitizingFileCheck' --sanitize BUILD_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/swift-macosx-x86_64' --sanitize SOURCE_DIR='/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift' --use-filecheck '/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/buildbot_incremental/llvm-macosx-x86_64/bin/FileCheck' /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/stdlib/KVOKeyPaths.swift
09:15:41 --
09:15:41 Exit Code: 1
09:15:41 
09:15:41 Command Output (stderr):
09:15:41 --
09:15:41 2019-02-01 07:15:07.852 a.out[36193:2194238] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'An instance 0x79e3f5e0 of class _TtCC4main7Target25Dummy was deallocated while key value observers were still registered with it. Current observation info: <NSKeyValueObservationInfo 0x79e3fd20> (
09:15:41 <NSKeyValueObservance 0x79e3fd00: Observer: 0x79e3f9f0, Key path: name, Options: <New: NO, Old: NO, Prior: NO> Context: 0x0, Property: 0x79e3fbc0>
09:15:41 )'
09:15:41 *** First throw call stack:
09:15:41 (
09:15:41 	0   CoreFoundation                      0x00278ded __exceptionPreprocess + 189
09:15:41 	1   libobjc.A.dylib                     0x0064ad6f objc_exception_throw + 49
09:15:41 	2   CoreFoundation                      0x002e37d8 +[NSException raise:format:] + 104
09:15:41 	3   Foundation                          0x00b02a3b NSKVODeallocate + 348
09:15:41 	4   libobjc.A.dylib                     0x0065f068 _ZN11objc_object17sidetable_releaseEb + 206
09:15:41 	5   libobjc.A.dylib                     0x0065e396 objc_release + 38
09:15:41 	6   a.out                               0x000416b4 $s4main7Target2C22keyPathsAffectingValue3forShys10AnyKeyPathCGAG_tFZ + 868
09:15:41 	7   a.out                               0x00042168 $s4main7Target2C10Foundation32NSKeyValueObservingCustomizationAadEP017keyPathsAffectingE03forShys10AnyKeyPathCGAJ_tFZTW + 24
09:15:41 	8   libswiftFoundation.dylib            0x0177ab90 $s10Foundation27__KVOKeyPathBridgeMachinery33_6DA0945A07226B3278459E9368612FF4LLC31keyPathsForValuesAffectingValue6forKeyShySSGSSSg_tFZ + 144
09:15:41 	9   libswiftFoundation.dylib            0x0177b3b4 $s10Foundation27__KVOKeyPathBridgeMachinery33_6DA0945A07226B3278459E9368612FF4LLC31keyPathsForValuesAffectingValue6forKeyShySSGSSSg_tFZTo + 84
09:15:41 	10  Foundation                          0x00aa9465 -[NSKeyValueUnnestedProperty _givenPropertiesBeingInitialized:getAffectingProperties:] + 122
09:15:41 	11  Foundation                          0x00aa9160 -[NSKeyValueUnnestedProperty _initWithContainerClass:key:propertiesBeingInitialized:] + 179
09:15:41 	12  Foundation                          0x00aa9025 NSKeyValuePropertyForIsaAndKeyPathInner + 295
09:15:41 	13  Foundation                          0x00aa8cbb NSKeyValuePropertyForIsaAndKeyPath + 151
09:15:41 	14  Foundation                          0x00aa8a67 -[NSObject(NSKeyValueObserverRegistration) addObserver:forKeyPath:options:context:] + 69
09:15:41 	15  libswiftFoundation.dylib            0x0177bf25 $s10Foundation27_KeyValueCodingAndObservingPAAE7observe_7options13changeHandlerAA05NSKeyC11ObservationCs0B4PathCyxqd__G_So0kcF7OptionsVyx_AA0kC14ObservedChangeVyqd__GtctlF + 1301
09:15:41 	16  a.out                               0x00042268 $s4mainyAA7Target2CXEfU_ + 216
09:15:41 	17  a.out                               0x00042305 $s4main7Target2Cs5Error_pIggzo_ACytsAD_pIegnrzo_TR + 37
09:15:41 	18  a.out                               0x00042380 $s4main7Target2Cs5Error_pIggzo_ACytsAD_pIegnrzo_TRTA + 48
09:15:41 	19  libswiftCore.dylib                  0x010b11d5 $ss20withExtendedLifetimeyq_x_q_xKXEtKr0_lF + 21
09:15:41 	20  a.out                               0x0003ce0f main + 6063
09:15:41 	21  libdyld.dylib                       0x01f40779 start + 1
09:15:41 )
09:15:41 libc++abi.dylib: terminating with uncaught exception of type NSException
09:15:41 Child process terminated with signal 6: Abort trap
09:15:41 
/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/test/stdlib/KVOKeyPaths.swift:171:17: error: CHECK-LABEL: expected string not found in input
09:15:41 // CHECK-LABEL: creating NSSortDescriptor

@jrose-apple
Copy link
Contributor

Disabled in #22294, apparently.

Copy link
Contributor

@eeckstein eeckstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basically lgtm.
See my comments

// CHECK: {{.*}} = call swiftcc {{.*}} @"$sSS18_fromUTF8Repairing{{.*}}
// CHECK-NOT: {{.*}} = call swiftcc {{.*}}_fromCodeUnits{{.*}}
// CHECK: ret

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can make the CHECK-NOT more robust by just using:

// CHECK-NOT: _fromCodeUnits

return String(decoding: ptr, as: Unicode.UTF8.self)
}

// UnsafeMutableBufferPointer<UInt8>: we check that we call the merged function
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To simplify the checking, you can disable function merging by setting the option -Xllvm swiftmergefunc-threshold=0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, that's much much better

@weissi weissi force-pushed the jw-utf8-decode-fastpath-test branch from 8a9e9f6 to 2ec49e0 Compare February 1, 2019 17:50
@weissi weissi force-pushed the jw-utf8-decode-fastpath-test branch from 2ec49e0 to 7a447bd Compare March 8, 2019 19:08
@weissi
Copy link
Contributor Author

weissi commented Mar 8, 2019

@swift-ci please test

@weissi weissi requested review from milseman and eeckstein March 8, 2019 19:31
@swift-ci
Copy link
Contributor

swift-ci commented Mar 8, 2019

Build failed
Swift Test Linux Platform
Git Sha - 7a447bd40e802612670c31df6671d43d68811f88

@swift-ci
Copy link
Contributor

swift-ci commented Mar 8, 2019

Build failed
Swift Test OS X Platform
Git Sha - 7a447bd40e802612670c31df6671d43d68811f88

@milseman
Copy link
Member

test/SILOptimizer/utf8_decoding_fastpath.swift:75:11: error: CHECK: expected string not found in input
// CHECK: {{.*}} = call swiftcc {{.*}}_fromCodeUnits{{.*}}

@weissi
Copy link
Contributor Author

weissi commented Mar 10, 2019

test/SILOptimizer/utf8_decoding_fastpath.swift:75:11: error: CHECK: expected string not found in input
// CHECK: {{.}} = call swiftcc {{.}}_fromCodeUnits{{.*}}

Thanks @milseman ! That's probably good news and means that by now, String.UTF8View actually does hit the fast path. Will rebase my Swift checkout and run again tomorrow.

@weissi weissi force-pushed the jw-utf8-decode-fastpath-test branch from 7a447bd to 61d4d0a Compare March 11, 2019 17:29
@weissi
Copy link
Contributor Author

weissi commented Mar 11, 2019

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 7a447bd40e802612670c31df6671d43d68811f88

@weissi weissi force-pushed the jw-utf8-decode-fastpath-test branch from 61d4d0a to 8fd42eb Compare March 11, 2019 17:31
@weissi
Copy link
Contributor Author

weissi commented Mar 11, 2019

@milseman good news, they all seem to now work (on my local build) so I'd expect the CI to pass now

@weissi
Copy link
Contributor Author

weissi commented Mar 11, 2019

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 61d4d0a26fb7cdd222848baf1faa2fba6b4e1184

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 61d4d0a26fb7cdd222848baf1faa2fba6b4e1184

@weissi
Copy link
Contributor Author

weissi commented Mar 11, 2019

@milseman the Linux Test failed because of

17:55:39 
llvm-config: error: component libraries and shared library
17:55:39 
17:55:39 llvm-config: error: missing: /home/buildnode/jenkins/workspace/swift-PR-Linux@2/branch-master/buildbot_linux/llvm-linux-x86_64/lib/libLLVMTestingSupport.a

but the utf8 fastpath test I added here passed:

18:25:14 PASS: Swift(linux-x86_64) :: SILOptimizer/utf8_decoding_fastpath.swift (8555 of 11751)

@milseman
Copy link
Member

@swift-ci please test linux

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 8fd42eb

@weissi
Copy link
Contributor Author

weissi commented Mar 12, 2019

hmm

05:31:43 Testing Time: 89.08s
05:31:43 ********************
05:31:43 Failing Tests (3):
05:31:43     LLDB :: Reproducer/Functionalities/TestDataFormatter.test
05:31:43     LLDB :: Reproducer/Functionalities/TestImageList.test
05:31:43     LLDB :: Reproducer/Functionalities/TestStepping.test
05:31:43 

@weissi
Copy link
Contributor Author

weissi commented Mar 12, 2019

@swift-ci please test linux

Copy link
Member

@milseman milseman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@weissi weissi merged commit a2322b4 into swiftlang:master Mar 12, 2019
@weissi weissi deleted the jw-utf8-decode-fastpath-test branch March 12, 2019 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants