Back swift_task_enqueueGlobal with public Dispatch API #34997

rjmccall · 2020-12-08T10:37:09Z

We expect to iterate on this quite a bit, both publicly and internally, but this is a fine starting-point.

I've renamed runAsync to runAsyncAndBlock to underline very clearly what it does and why it's not long for this world. I've also had to give it a radically different implementation in an effort to make it continue to work given an actor implementation that is no longer just running all work synchronously.

The major remaining bit of actor-scheduling work is to make swift_task_enqueue actually do something sensible based on the executor it's been given; currently it's expecting a flag that IRGen simply doesn't know to set. But we may want to express this as a more general actor-to-executor projection step.

rjmccall · 2020-12-08T10:37:17Z

@swift-ci Please test

swift-ci · 2020-12-08T10:54:10Z

Build failed
Swift Test Linux Platform
Git Sha - 2661455384886b96a061b9673e74afcc4f7e2792

swift-ci · 2020-12-08T12:01:35Z

Build failed
Swift Test OS X Platform
Git Sha - 2661455384886b96a061b9673e74afcc4f7e2792

stdlib/public/Concurrency/Actor.cpp

stdlib/public/Concurrency/Task.swift

DougGregor · 2020-12-08T21:50:16Z

stdlib/public/Concurrency/Task.swift

@@ -308,6 +306,9 @@ extension Task {
    // Create the asynchronous task future.
    let (task, _) = Builtin.createAsyncTaskFuture(flags.bits, nil, operation)

+    // Enqueue the resulting job.
+    _enqueueJobGlobal(Builtin.convertTaskToJob(task))


We'll want to do this for _runChildTask as well.

DougGregor · 2020-12-08T21:52:42Z

CMakeLists.txt

@@ -941,7 +941,7 @@ if (LLVM_ENABLE_DOXYGEN)
  message(STATUS "Doxygen: enabled")
 endif()

-if(SWIFT_BUILD_SYNTAXPARSERLIB OR SWIFT_BUILD_SOURCEKIT)
+if(SWIFT_BUILD_SYNTAXPARSERLIB OR SWIFT_BUILD_SOURCEKIT OR SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY)


I'm guessing you'll need to teach _Concurrency to link against the (C) lib dispatch on Linux and Windows.

rjmccall · 2020-12-09T04:20:50Z

@swift-ci Please test

DougGregor · 2020-12-09T05:07:01Z

Windows failure:

/Fostdlib\public\Concurrency\CMakeFiles\swift_Concurrency-windows-x86_64.dir\Actor.cpp.obj /Fdstdlib\public\Concurrency\CMakeFiles\swift_Concurrency-windows-x86_64.dir\ /FS -c S:\jenkins\workspace\swift-PR-windows\swift\stdlib\public\Concurrency\Actor.cpp
S:\jenkins\workspace\swift-PR-windows\swift\stdlib\public\Concurrency\Actor.cpp(1419,43): error: use of undeclared identifier 'qos_class_t'
  auto queue = dispatch_get_global_queue((qos_class_t) priority,
                                          ^

DougGregor · 2020-12-09T05:08:11Z

Okay, I think the Windows failure is because you need to use dispatch_qos_class_t instead of qos_class_t. qos_class_t seems like the Darwin one.

include/swift/AST/Builtins.def

stdlib/public/Concurrency/Actor.cpp

swift-ci · 2020-12-09T05:21:24Z

Build failed
Swift Test Linux Platform
Git Sha - 5b2db3dbb91166bab98cf09e2f6fec2101f0748a

rjmccall · 2020-12-09T06:42:58Z

@swift-ci Please test

swift-ci · 2020-12-09T08:09:10Z

Build failed
Swift Test Linux Platform
Git Sha - 4d3d562947747da1f8417e8ad68e5e6e8dd36ac4

rjmccall · 2020-12-09T09:01:58Z

@swift-ci Please test

swift-ci · 2020-12-09T09:45:50Z

Build failed
Swift Test Linux Platform
Git Sha - f79c87aaee1d61321d99b51b9971b593a798979f

rjmccall · 2020-12-09T18:06:24Z

@swift-ci Please test

swift-ci · 2020-12-09T18:49:05Z

Build failed
Swift Test Linux Platform
Git Sha - f79c87aaee1d61321d99b51b9971b593a798979f

rjmccall · 2020-12-09T18:57:21Z

@swift-ci Please test

swift-ci · 2020-12-09T19:23:43Z

Build failed
Swift Test OS X Platform
Git Sha - f96a6bf3233cfbd783efb1c60957b21e50532a0f

swift-ci · 2020-12-09T19:56:28Z

Build failed
Swift Test Linux Platform
Git Sha - f96a6bf3233cfbd783efb1c60957b21e50532a0f

rjmccall · 2020-12-09T21:26:22Z

@swift-ci Please test

swift-ci · 2020-12-09T22:09:50Z

Build failed
Swift Test Linux Platform
Git Sha - 3cbb4b8ad6ceba6a28d05640f89a597a071abbb7

rjmccall · 2020-12-09T23:36:40Z

@swift-ci Please test

swift-ci · 2020-12-10T06:24:49Z

Build failed
Swift Test OS X Platform
Git Sha - 9f877852156f23b7d6591cc495a0cecd0d5d3cc4

swift-ci · 2020-12-10T06:38:28Z

Build failed
Swift Test Linux Platform
Git Sha - 9f877852156f23b7d6591cc495a0cecd0d5d3cc4

rjmccall · 2020-12-10T07:39:31Z

@swift-ci Please test

rjmccall · 2020-12-10T08:34:14Z

@swift-ci Please test

rjmccall · 2020-12-10T08:37:46Z

Okay, there's a remaining test failure on Windows, but a bunch of work is blocking on this, so I marked the test as unsupported and filed rdar://72171028 to fix it post-commit.

swift-ci · 2020-12-10T10:03:10Z

Build failed
Swift Test Linux Platform
Git Sha - 0d368ecfe565a53d74f14b1b0940b6a96bdbaab6

rjmccall · 2020-12-10T18:35:37Z

@swift-ci Please test

rjmccall · 2020-12-10T19:28:46Z

@swift-ci Please test

swift-ci · 2020-12-10T20:04:44Z

Build failed
Swift Test Linux Platform
Git Sha - 2f7636ad3a30089a0db6d3bd3f7d028c754ab967

rjmccall · 2020-12-10T20:28:01Z

@swift-ci Please test

swift-ci · 2020-12-10T20:31:07Z

Build failed
Swift Test Linux Platform
Git Sha - 2f7636ad3a30089a0db6d3bd3f7d028c754ab967

swift-ci · 2020-12-10T20:32:00Z

Build failed
Swift Test OS X Platform
Git Sha - 2f7636ad3a30089a0db6d3bd3f7d028c754ab967

rjmccall · 2020-12-10T22:05:17Z

@swift-ci Please test Windows

I'm going to use this in Builtins.cpp.

rjmccall · 2020-12-10T22:20:44Z

@swift-ci Please test

We expect to iterate on this quite a bit, both publicly and internally, but this is a fine starting-point. I've renamed runAsync to runAsyncAndBlock to underline very clearly what it does and why it's not long for this world. I've also had to give it a radically different implementation in an effort to make it continue to work given an actor implementation that is no longer just running all work synchronously. The major remaining bit of actor-scheduling work is to make swift_task_enqueue actually do something sensible based on the executor it's been given; currently it's expecting a flag that IRGen simply doesn't know to set.

It would be more abstractly correct if this got DI support so that we destroy the member if the constructor terminates abnormally, but we can get to that later.

Currently, the only thing in the system that donates a thread to run it is swift_runAndBlockThread, but we'll probably need others. Nothing in the concurrency runtime should block via a semaphore in this configuration. As an outrageous hack, work around the layering problems with using libdispatch from the concurrency library on non-Darwin systems by making those systems use the cooperative global executor. This is only acceptable as a temporary solution for landing this change and setting things onto the right long-term design.

rjmccall · 2020-12-11T00:19:23Z

@swift-ci Please test

DougGregor reviewed Dec 8, 2020

View reviewed changes

rjmccall force-pushed the global-actor-dispatch branch 2 times, most recently from df7bd87 to 5b2db3d Compare December 9, 2020 04:20

rjmccall marked this pull request as ready for review December 9, 2020 04:47

DougGregor approved these changes Dec 9, 2020

View reviewed changes

include/swift/AST/Builtins.def Show resolved Hide resolved

stdlib/public/Concurrency/Actor.cpp Outdated Show resolved Hide resolved

stdlib/public/Concurrency/Actor.cpp Outdated Show resolved Hide resolved

rjmccall force-pushed the global-actor-dispatch branch from 5b2db3d to 4d3d562 Compare December 9, 2020 06:42

rjmccall force-pushed the global-actor-dispatch branch from 4d3d562 to f79c87a Compare December 9, 2020 09:01

rjmccall force-pushed the global-actor-dispatch branch from f79c87a to f96a6bf Compare December 9, 2020 18:57

rjmccall force-pushed the global-actor-dispatch branch from f96a6bf to 3cbb4b8 Compare December 9, 2020 21:26

rjmccall force-pushed the global-actor-dispatch branch from 0070c27 to b14af23 Compare December 10, 2020 00:10

rjmccall force-pushed the global-actor-dispatch branch from 9f87785 to be5a66d Compare December 10, 2020 07:39

rjmccall force-pushed the global-actor-dispatch branch from be5a66d to 0d368ec Compare December 10, 2020 08:33

ktoso mentioned this pull request Dec 10, 2020

[Concurrency] Initial implementation of TaskGroups #34999

Merged

rjmccall force-pushed the global-actor-dispatch branch from 0d368ec to 79f0ad3 Compare December 10, 2020 18:35

rjmccall force-pushed the global-actor-dispatch branch from 79f0ad3 to 2f7636a Compare December 10, 2020 19:28

rjmccall force-pushed the global-actor-dispatch branch from 2f7636a to c5fcaef Compare December 10, 2020 20:28

rjmccall added 2 commits December 10, 2020 17:06

[NFC] Add a header library to simplify generating synthetic types

10e0dfc

I'm going to use this in Builtins.cpp.

Add a builtin to convert a Task* to a Job*.

b22407e

rjmccall force-pushed the global-actor-dispatch branch from c5fcaef to f788445 Compare December 10, 2020 22:09

rjmccall added 3 commits December 10, 2020 19:18

Add builtins to initialize and destroy a default-actor member.

d874479

It would be more abstractly correct if this got DI support so that we destroy the member if the constructor terminates abnormally, but we can get to that later.

rjmccall force-pushed the global-actor-dispatch branch from f788445 to 7cc63f9 Compare December 11, 2020 00:19

rjmccall merged commit cce0600 into swiftlang:main Dec 11, 2020

rjmccall deleted the global-actor-dispatch branch December 11, 2020 04:40

Back swift_task_enqueueGlobal with public Dispatch API #34997

Back swift_task_enqueueGlobal with public Dispatch API #34997

Uh oh!

Conversation

rjmccall commented Dec 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rjmccall commented Dec 8, 2020

Uh oh!

swift-ci commented Dec 8, 2020

Uh oh!

swift-ci commented Dec 8, 2020

Uh oh!

Uh oh!

Uh oh!

DougGregor Dec 8, 2020

Choose a reason for hiding this comment

Uh oh!

DougGregor Dec 8, 2020

Choose a reason for hiding this comment

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

DougGregor commented Dec 9, 2020

Uh oh!

DougGregor commented Dec 9, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 9, 2020

Uh oh!

rjmccall commented Dec 9, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 10, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

swift-ci commented Dec 10, 2020

Uh oh!

rjmccall commented Dec 8, 2020 •

edited

Loading