Remove stdlib and runtime dependencies on Foundation and CF #26630

Catfish-Man · 2019-08-13T01:47:46Z

No description provided.

stdlib/public/stubs/OptionalBridgingHelper.mm

Catfish-Man · 2019-08-13T01:52:20Z

stdlib/public/stubs/Availability.mm

-    return [[NSProcessInfo processInfo] operatingSystemVersion];
-  } else {
-    // Otherwise load and parse from SystemVersion dictionary.
-    return operatingSystemVersionFromPlist();


Now that we're in the OS, we no longer need this fallback path

stdlib/public/runtime/SwiftObject.mm

Catfish-Man · 2019-08-13T01:53:34Z

stdlib/public/runtime/SwiftObject.mm

-NSString *swift::getDescription(OpaqueValue *value, const Metadata *type) {
-  auto result = swift_stdlib_getDescription(value, type);
+SWIFT_CC(swift) SWIFT_RUNTIME_STDLIB_API
+id swift_stdlib_boxUTF8CString(const char *cstr, int len);


How do you make an NSString without going through ObjC? Ask Swift to do it ;D

Catfish-Man · 2019-08-13T01:54:01Z

stdlib/public/core/ShadowProtocols.swift

@@ -171,6 +171,10 @@ internal protocol _NSSetCore: _NSCopying, _NSFastEnumeration {
 /// supplies.
 @unsafe_no_objc_tagged_pointer @objc
 internal protocol _NSSet: _NSSetCore {
+  @objc(getObjects:count:) func getObjects(


This should be a small speedup relative to the CF call :)

Catfish-Man · 2019-08-13T01:54:38Z

stdlib/public/stubs/OptionalBridgingHelper.mm

@@ -58,7 +75,7 @@ - (NSString*)description {
 static id getSentinelForDepth(unsigned depth) {
  // For unnested optionals, use NSNull.
  if (depth == 1)
-    return id_const_cast(kCFNull);
+    return SWIFT_LAZY_CONSTANT(id_const_cast([objc_getClass("NSNull") null]));


This burns a tiny bit of memory, which makes me sad. We could possibly not cache it, I'm not sure how hot this code path is.

The cache is probably a good idea. I don't think you need id_const_cast if you're going through Objective-C, by the way.

This code path should only come up in cases that shouldn't be that common to begin with (bridging a nil Optional to a non-nullable id, such as when turning a [T?] into NSArray), so it may be sufficiently cold that the cache isn't needed.

test/api-digester/Outputs/stability-stdlib-abi.asserts.additional.swift.expected

Catfish-Man · 2019-08-13T01:55:35Z

stdlib/public/stubs/Availability.mm

+#include <os/system_version.h>
+static NSOperatingSystemVersion getOSVersion() {
+  struct os_system_version_s vers;
+  if (os_system_version_get_current_version(&vers)) {


I believe this should make availability checking significantly more efficient :)

Catfish-Man · 2019-08-13T02:01:15Z

@swift-ci please smoke test

Catfish-Man · 2019-08-13T02:01:20Z

@swift-ci please smoke benchmark

swift-ci · 2019-08-13T02:53:11Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
LessSubstringSubstring	40	44	+10.0%	0.91x
EqualStringSubstring	40	44	+10.0%	0.91x (?)
EqualSubstringSubstringGenericEquatable	40	44	+10.0%	0.91x (?)
EqualSubstringString	40	44	+10.0%	0.91x (?)
LessSubstringSubstringGenericComparable	40	44	+10.0%	0.91x
RandomShuffleLCG2	704	768	+9.1%	0.92x (?)
FlattenListLoop	3981	4333	+8.8%	0.92x (?)
Array2D	6912	7520	+8.8%	0.92x (?)
RemoveWhereSwapInts	62	67	+8.1%	0.93x
MapReduce	369	398	+7.9%	0.93x (?)
MapReduceAnyCollection	370	398	+7.6%	0.93x (?)

Improvement	OLD	NEW	DELTA	RATIO
FloatingPointPrinting_Float_description_small	5724	5292	-7.5%	1.08x (?)

Code size: -O

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
EqualSubstringSubstring	40	44	+10.0%	0.91x
EqualSubstringSubstringGenericEquatable	40	44	+10.0%	0.91x
EqualSubstringString	41	45	+9.8%	0.91x (?)
RemoveWhereMoveInts	34	37	+8.8%	0.92x (?)
Array2D	6928	7520	+8.5%	0.92x
RemoveWhereSwapInts	60	65	+8.3%	0.92x (?)
FlattenListLoop	4906	5277	+7.6%	0.93x (?)

Improvement	OLD	NEW	DELTA	RATIO
DictionaryBridgeToObjC_Access	982	896	-8.8%	1.10x (?)
InsertCharacterEndIndex	165	151	-8.5%	1.09x (?)

Code size: -Osize

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
LessSubstringSubstringGenericComparable	45	50	+11.1%	0.90x (?)
StringBuilderWithLongSubstring	3010	3270	+8.6%	0.92x (?)

Improvement	OLD	NEW	DELTA	RATIO
Set.subtracting.Seq.Int100	9069	8050	-11.2%	1.13x (?)
Set.subtracting.Seq.Int50	7983	7402	-7.3%	1.08x (?)
ArrayOfPOD	1148	1065	-7.2%	1.08x (?)

Code size: -swiftlibs

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

Catfish-Man · 2019-08-13T02:54:50Z

Looks pretty much in the noise. The tests that maybe-regressed are all ones I recognize as maybe-progressions from other PRs, and should be unrelated to this code.

stdlib/public/runtime/SwiftObject.mm

stdlib/public/stubs/Availability.mm

stdlib/public/runtime/SwiftObject.h

stdlib/public/core/StringBridge.swift

stdlib/public/runtime/ErrorObject.mm

Catfish-Man · 2019-08-14T22:31:50Z

@swift-ci please smoke test

Catfish-Man · 2019-08-14T22:34:11Z

@swift-ci please smoke test

Catfish-Man · 2019-08-14T22:34:18Z

@swift-ci please smoke benchmark

Catfish-Man · 2019-08-14T23:15:02Z

Test failure is just the ABI one, which I mis-merged. So tests look good! That's progress.

Still investigating the mystery of why the F/CF dependency in the cmake file is needed even though nothing references them anymore.

stdlib/public/core/ReflectionMirror.swift

swift-ci · 2019-08-14T23:25:23Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubFromNSStringRef	155	186	+20.0%	0.83x (?)
IterateData	1319	1454	+10.2%	0.91x (?)
String.data.LargeUnicode	89	97	+9.0%	0.92x (?)
String.replaceSubrange.RepChar.Small	383	412	+7.6%	0.93x (?)

Improvement	OLD	NEW	DELTA	RATIO
FlattenListFlatMap	7969	7272	-8.7%	1.10x (?)
Data.hash.Empty	74	68	-8.1%	1.09x (?)

Code size: -O

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
IterateData	1319	1458	+10.5%	0.90x (?)

Improvement	OLD	NEW	DELTA	RATIO
Data.hash.Empty	77	71	-7.8%	1.08x (?)

Code size: -Osize

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
StringUTF16SubstringBuilder	18980	20920	+10.2%	0.91x (?)
StringMatch	54300	59400	+9.4%	0.91x (?)
StringBuilderWithLongSubstring	3060	3320	+8.5%	0.92x (?)

Improvement	OLD	NEW	DELTA	RATIO
DictionaryBridgeToObjC_Access	1147	1019	-11.2%	1.13x (?)
Set.subtracting.Seq.Int100	9085	8087	-11.0%	1.12x (?)

Code size: -swiftlibs

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

Catfish-Man · 2019-08-14T23:25:58Z

big sigh of relief I was NOT sure that benchmark run would be ok

stdlib/public/runtime/SwiftObject.mm

Catfish-Man · 2019-08-15T23:02:27Z

@swift-ci please test

Catfish-Man · 2019-08-15T23:02:34Z

@swift-ci please smoke benchmark

swift-ci · 2019-08-15T23:04:40Z

Build failed
Swift Test Linux Platform
Git Sha - c3419e180484e464dcff0d17895e0703a2210681

swift-ci · 2019-08-15T23:06:16Z

Build failed
Swift Test OS X Platform
Git Sha - c3419e180484e464dcff0d17895e0703a2210681

Catfish-Man · 2019-08-15T23:18:38Z

@swift-ci please test

Catfish-Man · 2019-08-15T23:18:44Z

@swift-ci please smoke benchmark

stdlib/public/runtime/ErrorObject.mm

jrose-apple · 2019-08-23T15:44:59Z

stdlib/public/runtime/ErrorObject.mm

@@ -153,16 +167,29 @@ - (BOOL)isEqual:(id)other {
 @end

 Class swift::getNSErrorClass() {
-  return SWIFT_LAZY_CONSTANT([NSError class]);
+  return SWIFT_LAZY_CONSTANT(objc_lookUpClass("NSError"));


Is there still a difference between this and objc_getClass in the modern runtime?

One small difference: they pass different values to the third, unused parameter of the internal function that implements the functionality. Don't ask me why.

stdlib/public/runtime/ErrorObject.mm

stdlib/public/runtime/SwiftValue.mm

stdlib/public/stubs/OptionalBridgingHelper.mm

test/stdlib/SmallString.swift

jrose-apple · 2019-08-23T15:52:38Z

test/stdlib/SmallString.swift

-// RUN: %target-build-swift -Xfrontend -disable-access-control -module-name a %s -o %t.out -O
+// RUN: %empty-directory(%t)
+// RUN: %gyb %s > %t/main.swift
+// RUN: %target-build-swift -Xfrontend -disable-access-control -module-name a %t/main.swift %S/../Inputs/SmallStringTestUtilities.swift -o %t.out -O


Do you still need -disable-access-control? Because -disable-access-control is basically not guaranteed to work, especially with -O.

Well, that's not new, just moved, but yes. This uses _SmallString

I guess as long as it's just @usableFromInline things you use it should be okay…

Maybe we should have a different frontend flag for that.

jrose-apple · 2019-08-23T15:53:11Z

test/stdlib/StringBridge.swift

-// RUN: %target-run-simple-swift
+// RUN: %empty-directory(%t)
+// RUN: %gyb %s > %t/main.swift
+// RUN: %target-build-swift -Xfrontend -disable-access-control -module-name a %t/main.swift %S/../Inputs/SmallStringTestUtilities.swift -o %t.out -O


Also not gyb, also hopefully doesn't need -disable-access-control!

Catfish-Man · 2019-08-23T20:48:53Z

@swift-ci please test

swift-ci · 2019-08-23T20:49:24Z

Build failed
Swift Test OS X Platform
Git Sha - 97e6b532193db16179bd492f3a6700783b1137bc

swift-ci · 2019-08-23T20:51:08Z

Build failed
Swift Test Linux Platform
Git Sha - 97e6b532193db16179bd492f3a6700783b1137bc

Catfish-Man · 2019-08-23T21:04:59Z

@swift-ci please test

swift-ci · 2019-08-23T21:07:25Z

Build failed
Swift Test Linux Platform
Git Sha - ed19f1deb92a52ea56dd34f5bcc3e680f928282c

swift-ci · 2019-08-23T21:09:42Z

Build failed
Swift Test OS X Platform
Git Sha - ed19f1deb92a52ea56dd34f5bcc3e680f928282c

milseman

LGTM

Catfish-Man · 2019-08-23T21:26:36Z

@swift-ci please test and merge

Catfish-Man · 2019-08-23T21:29:25Z

@swift-ci please test and merge

Catfish-Man commented Aug 13, 2019

View reviewed changes

Catfish-Man marked this pull request as ready for review August 13, 2019 02:56

Catfish-Man self-assigned this Aug 13, 2019

Catfish-Man requested review from milseman, mikeash and lorentey August 13, 2019 02:56

mikeash approved these changes Aug 13, 2019

View reviewed changes

stdlib/public/runtime/SwiftObject.mm Show resolved Hide resolved

stdlib/public/stubs/Availability.mm Outdated Show resolved Hide resolved

jrose-apple reviewed Aug 13, 2019

View reviewed changes

stdlib/public/runtime/SwiftObject.h Outdated Show resolved Hide resolved

jrose-apple reviewed Aug 13, 2019

View reviewed changes

stdlib/public/core/StringBridge.swift Outdated Show resolved Hide resolved

jrose-apple reviewed Aug 14, 2019

View reviewed changes

stdlib/public/runtime/ErrorObject.mm Outdated Show resolved Hide resolved

Catfish-Man changed the title ~~Remove most of the remaining link-time references to Foundation and CF~~ [WIP] Remove remaining link-time references to Foundation and CF Aug 14, 2019

Catfish-Man force-pushed the shim-shim-sher-ee branch from 6fdac98 to edb478d Compare August 14, 2019 22:30

jrose-apple reviewed Aug 14, 2019

View reviewed changes

stdlib/public/core/ReflectionMirror.swift Outdated Show resolved Hide resolved

jrose-apple reviewed Aug 15, 2019

View reviewed changes

stdlib/public/runtime/SwiftObject.mm Outdated Show resolved Hide resolved