|
| 1 | +# Global objects in DPC++ runtime |
| 2 | + |
| 3 | +## Intro |
| 4 | + |
| 5 | +C++ standard does not specify the order in which global objects are constructed |
| 6 | +or destroyed. If global objects somehow interact with each other, there's a |
| 7 | +chance, that one of the objects has not been initialized or has been destroyed |
| 8 | +by the time of interaction. This problem is also refered to as |
| 9 | +[static initialization order fiasco]. |
| 10 | + |
| 11 | +The only two things C++ guarantees is that global objects are constructed before |
| 12 | +program enters `main` and within one translation unit objects will be |
| 13 | +constructed in the same order as they occur in code. Initialization order |
| 14 | +between translation units is undefined. |
| 15 | + |
| 16 | +At the same time, SYCL users may want to construct some SYCL objects globally, |
| 17 | +like in example below: |
| 18 | + |
| 19 | +``` |
| 20 | +#include <CL/sycl.hpp> |
| 21 | +
|
| 22 | +sycl::queue Queue; |
| 23 | +
|
| 24 | +int main() { |
| 25 | + Queue = sycl::queue{sycl::default_selector{}.select_device()}; |
| 26 | +
|
| 27 | + return 0; |
| 28 | +} |
| 29 | +``` |
| 30 | + |
| 31 | +While the above piece of code is syntactically correct, it is still an undefined |
| 32 | +behavior from C++ standard point of view. There are a few places in the runtime, |
| 33 | +where global objects arise: scheduler, program manager, plugins, low-level |
| 34 | +runtimes. To prevent crashes in such scenarios, the DPC++ runtime must ensure |
| 35 | +global objects lifetime is long enough. |
| 36 | + |
| 37 | +## DPC++ runtime |
| 38 | + |
| 39 | +### General idea |
| 40 | + |
| 41 | +Different platforms may handle global initialization and deinitialization |
| 42 | +differently (for example, see [Itanium ABI]). So, handling global objects |
| 43 | +lifetime is platform-dependent. However, there's a common idea behind those |
| 44 | +approaches. |
| 45 | + |
| 46 | +DPC++ wraps all complex global objects in a special structure, called |
| 47 | +`GlobalHandler`. The runtime stores a global pointer to that structure, and |
| 48 | +initializes it on first call to `GlobalHandler::instance()` method (singleton |
| 49 | +pattern). The `GlobalHandler` provides getter methods to access different |
| 50 | +objects. Those objects are stored in `std::unique_ptr`s, that are initialized |
| 51 | +on first call to getter member function. This way DPC++ runtime ensures, that |
| 52 | +no unwanted initialization happens before object is requested. |
| 53 | + |
| 54 | +Deinitialization is platform-specific. Upon application shutdown, the DPC++ |
| 55 | +runtime frees memory pointed by `GlobalHandler` global pointer, which triggers |
| 56 | +destruction of nested `std::unique_ptr`s. |
| 57 | + |
| 58 | +### Linux |
| 59 | + |
| 60 | +On Linux DPC++ runtime uses `__attribute__((destructor))` property with maximum |
| 61 | +possible priority value 65535. This approach does not guarantee, that |
| 62 | +`GlobalHandler` destructor is the last thing to run, as user code may contain |
| 63 | +a similar function with the same priority value. |
| 64 | + |
| 65 | +Another approach would be to leak global objects. This would guarantee user, |
| 66 | +that global objects live long enough. But some global objects allocate heap |
| 67 | +memory. If user application uses `dlopen` and `dlclose` on `libsycl.so` many |
| 68 | +times, the memory leak may impact code performance. |
| 69 | + |
| 70 | +### Windows |
| 71 | + |
| 72 | +To identify shutdown moment on Windows, DPC++ runtime uses default `DllMain` |
| 73 | +function with `DLL_PROCESS_DETACH` reason. This guarantees, that global objects |
| 74 | +deinitialization happens right before `sycl.dll` is unloaded from process |
| 75 | +address space. |
| 76 | + |
| 77 | +### Recommendations for DPC++ runtime developers |
| 78 | + |
| 79 | +There are a few things to keep in mind, when developing DPC++ runtime: |
| 80 | + |
| 81 | +- It is fine to have global objects with trivial constructor and destructor. |
| 82 | +These objects can be zero initialized, and there's no deinitialization procedure |
| 83 | +for such objects. This is why `int`, `bool`, and other objects of trivial types |
| 84 | +are not wrapped with `GlobalHandler`. |
| 85 | +- `std::mutex` is not guaranteed to be trivial. Either wrap it with |
| 86 | +`GlobalHandler` or consider using `sycl::detail::SpinLock`, which has trivial |
| 87 | +constructor and destructor. |
| 88 | + |
| 89 | +## Plugins |
| 90 | + |
| 91 | +TBD |
| 92 | + |
| 93 | +## Low-level runtimes |
| 94 | + |
| 95 | +Generally, DPC++ runtime has no control over its dependencies. Such libraries |
| 96 | +can have global objects of their own. If you observe problems with dependency |
| 97 | +library, please, report it to library maintainers. |
| 98 | + |
| 99 | +[static initialization order fiasco]: https://isocpp.org/wiki/faq/ctors#static-init-order |
| 100 | +[Itanium ABI]: https://itanium-cxx-abi.github.io/cxx-abi/abi.html#dso-dtor |
0 commit comments