[stdlib] properly promote stack buffer to heap buffer when necessary #20585

lancep · 2018-11-14T23:35:04Z

rdar://problem/45216800
https://bugs.swift.org/browse/SR-8939

lancep · 2018-11-14T23:35:12Z

@swift-ci please test

lancep · 2018-11-14T23:35:20Z

@swift-ci please benchmark

milseman

Some code cleanup and testing suggestions

stdlib/public/core/NormalizedCodeUnitIterator.swift

milseman · 2018-11-15T00:18:23Z

stdlib/public/core/NormalizedCodeUnitIterator.swift

      // Our source of code units to normalize is empty and our buffers from
      // previous normalizations are also empty.
      return nil
    }
-    if segmentBufferCount == 0 && overflowBufferCount == 0 {
+    if segmentBufferCount == 0 {
      //time to fill a buffer if possible. Otherwise we are done, return nil


We don't return nil anymore, do we? What's this comment describing?

milseman · 2018-11-15T00:20:47Z

validation-test/stdlib/String.swift

+
+  expectTrue(set.contains(str))
+}
+


Do we have any expectation on what this should normalize to? Can we write a test for that?

stdlib/public/core/NormalizedCodeUnitIterator.swift

milseman · 2018-11-15T00:25:18Z

stdlib/public/core/NormalizedCodeUnitIterator.swift

+        ) {
+          segmentBufferCount = count
+        } else {
+          segmentBufferCount = normalizeWithHeapBuffers()


So along this path, we have segment input inside normalizationBuffer, but nothing in segmentBuffer. In the code path below, we don't have input in normalizationBuffer. Is that important?

milseman · 2018-11-15T00:26:28Z

stdlib/public/core/NormalizedCodeUnitIterator.swift

+        return segmentHeapBuffer![index]
+      } else {
+        let index = segmentBufferIndex
+        segmentBufferIndex += 1


Common lines can be hoisted

milseman · 2018-11-15T00:26:53Z

stdlib/public/core/NormalizedCodeUnitIterator.swift

    } else {
-        return nil
+      return nil


We can simplify this code by putting the return nil as an early return.

E.g.:

guard segmentBufferIndex < segmentBufferCount else { return nil } // .. now the rest of the function has less indentation and control flow

lancep · 2018-11-15T22:17:36Z

@swift-ci please test

lancep · 2018-11-15T22:17:42Z

@swift-ci please benchmark

swift-ci · 2018-11-15T22:19:38Z

Build failed
Swift Test Linux Platform
Git Sha - b4bc47ca5fd057c91567559091ab15f6d8b30b26

swift-ci · 2018-11-15T22:21:31Z

Build failed
Swift Test OS X Platform
Git Sha - b4bc47ca5fd057c91567559091ab15f6d8b30b26

stdlib/public/core/NormalizedCodeUnitIterator.swift

milseman · 2018-11-15T22:39:11Z

stdlib/public/core/NormalizedCodeUnitIterator.swift

    } else {
-        return nil
+      return nil


E.g.:

guard segmentBufferIndex < segmentBufferCount else { return nil } // .. now the rest of the function has less indentation and control flow

stdlib/public/core/NormalizedCodeUnitIterator.swift

milseman · 2018-11-15T22:50:00Z

validation-test/stdlib/String.swift

+  let strNFC = "\u{0336}\u{0308}\u{0301}\u{0357}\u{0313}\u{0314}\u{0351}\u{0300}\u{0300}\u{0300}\u{0314}\u{0357}\u{0301}\u{0308}\u{0301}\u{0315}\u{0360}\u{0061}"
+  expectEqual(str, strNFC)
+}
+


Were you able to test the case where the previous approach would've skipped over the long segment?

Yes, this new test caught it.

The concern was that a corner case would cause us to skip a portion of the String. Unfortunately, a test case expressed in terms of string comparison is only testing behavioral consistency but not the actual result. We have _nfcCodeUnits and _withNFCCodeUnits to check exactly this.

swift-ci · 2018-11-15T23:01:23Z

Build comment file:

Performance: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
StringEqualPointerComparison	600	657	+9.5%	0.91x
Hanoi	3486	3787	+8.6%	0.92x (?)

Performance: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
CStringLongAscii	423	528	+24.8%	0.80x
StringEqualPointerComparison	588	646	+9.9%	0.91x

Performance: -Onone

TEST	OLD	NEW	DELTA	RATIO
Improvement
ArrayOfPOD	859	775	-9.8%	1.11x (?)
StringWordBuilderReservingCapacity	2357	2154	-8.6%	1.09x (?)

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

--------------

lancep · 2018-11-16T18:17:46Z

@swift-ci please test

lancep · 2018-11-16T18:17:54Z

@swift-ci please benchmark

swift-ci · 2018-11-16T18:18:14Z

Build failed
Swift Test OS X Platform
Git Sha - 2534e9b9923048374d653b169e0676c0345b63e8

swift-ci · 2018-11-16T18:20:03Z

Build failed
Swift Test Linux Platform
Git Sha - 2534e9b9923048374d653b169e0676c0345b63e8

swift-ci · 2018-11-16T19:01:12Z

Build comment file:

Performance: -Onone

TEST	OLD	NEW	DELTA	RATIO
Improvement
StringUTF16SubstringBuilder	13813	11735	-15.0%	1.18x

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB

--------------

lancep · 2018-11-16T19:09:37Z

@swift-ci please test

swift-ci · 2018-11-16T19:10:53Z

Build failed
Swift Test OS X Platform
Git Sha - a29b11138ab35ceb259de02115a74b99b11021ff

lancep · 2018-11-16T19:11:23Z

@swift-ci please benchmark

lancep · 2018-11-16T19:15:23Z

@swift-ci please test

lancep · 2018-11-16T19:15:34Z

@swift-ci please benchmark

milseman · 2018-11-16T19:21:22Z

Approved with request for follow PR that tests the actual normalized code units rather than String comparison

swift-ci · 2018-11-16T19:29:00Z

Build failed
Swift Test Linux Platform
Git Sha - a29b11138ab35ceb259de02115a74b99b11021ff

swift-ci · 2018-11-16T20:13:57Z

Build comment file:

Performance: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
DropWhileAnySeqCntRange	165	188	+13.9%	0.88x (?)
InsertCharacterEndIndex	151	164	+8.6%	0.92x

Performance: -Onone

TEST	OLD	NEW	DELTA	RATIO
Improvement
AnyHashableWithAClass	144000	105500	-26.7%	1.36x

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

--------------

lancep · 2018-11-16T21:27:35Z

@swift-ci please test Linux platform

swift-ci · 2018-11-16T22:40:02Z

Build failed
Swift Test Linux Platform
Git Sha - fc3341e

lancep · 2018-11-16T22:44:38Z

@swift-ci please test

swift-ci · 2018-11-16T22:46:29Z

Build failed
Swift Test OS X Platform
Git Sha - fc3341e

swift-ci · 2018-11-16T22:46:38Z

Build failed
Swift Test Linux Platform
Git Sha - fc3341e

lancep requested a review from milseman November 14, 2018 23:54

This comment has been minimized.

Sign in to view

milseman requested changes Nov 15, 2018

View reviewed changes

milseman reviewed Nov 15, 2018

View reviewed changes

properly promote stack buffer to heap buffer when necessary

0009b21

Lance Parker added 2 commits November 16, 2018 10:20

Michael's feedback

12bf297

Moar feedback

f9aa787

fixup rebase

fc3341e

lancep force-pushed the spotFixForStringCompCrash branch from a29b111 to fc3341e Compare November 16, 2018 19:07

milseman approved these changes Nov 16, 2018

View reviewed changes

Make the NormalizationCheck test compare code units

1718734

lancep merged commit 8306ef3 into swiftlang:master Nov 17, 2018

lancep deleted the spotFixForStringCompCrash branch November 17, 2018 03:05

lancep mentioned this pull request Nov 26, 2018

[stdlib]Properly promote stack buffer to heap buffer when necessary (5.0 cherry pick) #20668

Merged

swift-ci mentioned this pull request Dec 12, 2018

[SR-8939] "this should not happen" panic when putting unicode in a Map #51444

Closed

[stdlib] properly promote stack buffer to heap buffer when necessary #20585

[stdlib] properly promote stack buffer to heap buffer when necessary #20585

Uh oh!

Conversation

lancep commented Nov 14, 2018

Uh oh!

lancep commented Nov 14, 2018

Uh oh!

lancep commented Nov 14, 2018

Uh oh!

This comment has been minimized.

milseman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lancep commented Nov 15, 2018

Uh oh!

lancep commented Nov 15, 2018

Uh oh!

swift-ci commented Nov 15, 2018

Uh oh!

swift-ci commented Nov 15, 2018

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swift-ci commented Nov 15, 2018

Build comment file:

Performance: -O

Performance: -Osize

Performance: -Onone

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018

Build comment file:

Performance: -Onone

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

lancep commented Nov 16, 2018

Uh oh!

milseman commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018

Uh oh!

swift-ci commented Nov 16, 2018