Skip to content

Implement a custom Data.Iterator #3831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 29, 2016
Merged

Conversation

lilyball
Copy link
Contributor

What's in this pull request?

Implement a custom Data.Iterator instead of using the default IndexingIterator. This custom iterator uses an internal 32-byte buffer so it doesn't have to call copyBytes(to:count:) as often. This results in an approximate 6x speedup on my computer.

With the included benchmark, I get the following numbers with IndexingIterator:

SAMPLES MIN(μs) MAX(μs) MEAN(μs) SD(μs) MEDIAN(μs) MAX_RSS(B)
3,      14680,  15039,  14867,   0,     14867,     4666709

And the following numbers with the custom iterator:

SAMPLES MIN(μs) MAX(μs) MEAN(μs) SD(μs) MEDIAN(μs) MAX_RSS(B)
3,      2436,   2508,   2482,    0,     2482,      4642133

Before merging this pull request to apple/swift repository:

  • Test pull request on Swift continuous integration.

Triggering Swift CI

The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:

Smoke Testing

Platform Comment
All supported platforms @swift-ci Please smoke test
All supported platforms @swift-ci Please smoke test and merge
OS X platform @swift-ci Please smoke test OS X platform
Linux platform @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

  1. Builds the compiler incrementally.
  2. Builds the standard library only for macOS. Simulator standard libraries and
    device standard libraries are not built.
  3. lldb is not built.
  4. The test and validation-test targets are run only for macOS. The optimized
    version of these tests are not run.

A smoke test on Linux does the following:

  1. Builds the compiler incrementally.
  2. Builds the standard library incrementally.
  3. lldb is built incrementally.
  4. The swift test and validation-test targets are run. The optimized version of these
    tests are not run.
  5. lldb is tested.

Validation Testing

Platform Comment
All supported platforms @swift-ci Please test
All supported platforms @swift-ci Please test and merge
OS X platform @swift-ci Please test OS X platform
OS X platform @swift-ci Please benchmark
Linux platform @swift-ci Please test Linux platform

Lint Testing

Language Comment
Python @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

ptr[i] = UInt8(i % 23)
}
}
checkSequence(data, (0..<65535).lazy.map({ UInt8($0 % 23) }))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use two spaces for indentation unless the file already uses 4.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@gribozavr
Copy link
Contributor

@swift-ci Please test

@gribozavr
Copy link
Contributor

LGTM, but deferring to @parkera to approve once CI passes.

ptr[i] = UInt8(i % 23)
}
}
checkSequence(data, (0..<65535).lazy.map({ UInt8($0 % 23) }))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing: the expected value is the first parameter, sequence under test is the second.

public func checkSequence<
  ${genericParam}, S : Sequence
>(
  _ expected: ${Expected},
  _ sequence: S,

We should probably add argument labels to this function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. That's a pretty important distinction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@parkera
Copy link
Contributor

parkera commented Jul 28, 2016

Great idea, thanks @kballard .

This iterator uses an inline 32-byte buffer so it doesn't have to call
copyBytes(to:count:) for every single byte. It results in an approximate
6x speedup on my computer.
@gribozavr
Copy link
Contributor

@swift-ci Please test

@gribozavr
Copy link
Contributor

Tests passed, CI failed for other reasons.

@gribozavr gribozavr merged commit 69255f0 into swiftlang:master Jul 29, 2016
@tkremenek
Copy link
Member

Reverted in #3848:

/swift/stdlib/public/SDK/Foundation/Data.swift:650:16: error: 'Data.Iterator' initializer is inaccessible due to 'private' protection level
        return Iterator(_data: self)
               ^
/Users/buildnode/jenkins/workspace/oss-swift-incremental-RA-osx/swift/stdlib/public/SDK/Foundation/Data.swift:663:17: note: 'init' declared here
        private init(_data: Data) {
                ^
/Users/buildnode/jenkins/workspace/oss-swift-incremental-RA-osx/swift/stdlib/public/SDK/Foundation/Notification.swift:134:16: warning: explicitly specified type '[String : Any]?' adds an additional level of optional to the initializer, making the optional check always succeed
            if let info : [String : Any]? = _NSUserInfoDictionary.bridgeReference(from: userInfo) {
               ^          ~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                          [String : Any]

@gribozavr
Copy link
Contributor

@kballard Could you resubmit with this fix applied? #3846

@lilyball
Copy link
Contributor Author

D'oh. Why didn't that fail with my local build?

@gribozavr
Copy link
Contributor

@kballard Because we just changed what private means, right after the CI run started.

@lilyball lilyball mentioned this pull request Jul 29, 2016
1 task
@lilyball
Copy link
Contributor Author

Re-submitted as #3849.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants