Merge changes for nightly tests in CI (#1190)

* dpnp_take_c uses SYCL kernel, no need to use no_sycl parameter in adapter

The reason this crashed with CPU device and gave incorrect results on
Windows was deeper.

   1. Adapter call allocated USM-shared buffer and copies data into it
   2. Kernel is submitted to work on USM-shared pointer
   3. dpnp_take_c returns kernel submission even
   4. Adapter class goes out of scope and frees USM allocation
      without making sure that the kernel that works on it has completed
      its execution
   5. If kernel execution was in progress we got a crash on CPU, or
      incorrect result on GPU

      If kernel execution was complete it worked as expected.

This change fixes the problem because it removes creation of unprotected
USM-shared temporary.

* Change to DPNPC_adapter to set/use events upon which deallocation must depend

The deallocation routine simply calls sycl::event::wait on the stored vector.

* Used DPNPC_ptr_adapter::depends_on

Also applied DPCTLEvent_Delete in legacy interfaces to avoid memory leak.

* Get rid of "Improper Null Termination" issue

Add a null-terminated symbol at the end of char array to avoid "Improper Null Termination" issue reported by Checkmarx scan.

* implemented PR feedback

* Reworked solution with a pointer on void

* Update dpnp/backend/kernels/dpnp_krnl_random.cpp

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Update dpnp/backend/kernels/dpnp_krnl_random.cpp

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Skip for two more tests till waiting fix (#1171)

* Skip for two more tests till waiting fix

tests/third_party/cupy/statistics_tests/test_order.py::TestOrder::test_ptp_all_nan
tests/third_party/cupy/statistics_tests/test_order.py::TestOrder::test_ptp_nan

Need to skip them because CI does not work due to this.

* The same tests skip for gpu

* dpnp_take failed on Windows due to memory corruption (#1172)

* dpnp_take failed on Windows due to memory corruption

* Add more tests

* Integer indexes types with different types of input data

* Add trailing empty line to .gitignore

* Add workflow for Win

Fix typo

Relax a strict pinning for numpy & cmake

Update run command for conda build on Win

Fix declaring DPLROOT env

Fix DPLROOT source

Fix DPLROOT for Win

Add missing double quotes

Try conda-incubator for Linux

Setup conda-incubator for Linux

Update caching

Exclude python 3.8

Strickly pin on 3.8.13

Change channel order

Fix artifcat uploading

Replace to single quotes

Add missing backslash

Corect backslash

* Attempt to fix workflow

* attempt to fix upload steps of the workflow on Linux

* Another attempt to fix upload step of conda-package workflow

* Set default shell in upload actions (#1180)

* Use pin_compatible for run-time dependency generation on numpy, restrict numpy version bracket for host section

* Reorder channels in conda-build (#1182)

* Reorder channels in conda-build

* Remove conda-build script for Linux

* Add tests running as a part of github actions (#1184)

* [Build] setuptools 63.4.1 breaks build for Windows (#1185)

* [SAT-5366] setuptools 63.4.1 breaks build for Windows

* Add TODO note as suggested in review comment

* Add extra information on how to build dpdnp from source.

* Fix import dpnp on Win & python 3.10 (#1189)

* Fix import dpnp on Win & python 3.10

* PATH is used by python till 3.10

* Update dpnp/__init__.py

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Update dpnp/__init__.py

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Update dpnp/__init__.py

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Update dpnp/__init__.py

Co-authored-by: Oleksandr Pavlyk <[email protected]>

* Apply review comments

Co-authored-by: Oleksandr Pavlyk <[email protected]>

Co-authored-by: Oleksandr Pavlyk <[email protected]>
Co-authored-by: Alexander Rybkin <[email protected]>
Co-authored-by: Diptorup Deb <[email protected]>

Author: 4 people

README.md

@@ -20,6 +20,35 @@ DPNP_QUEUE_GPU=1 python examples/example1.py
 ```
 
 ## Build from source:
+ Ensure you have the following prerequisite packages installed:
+
+- `mkl-devel-dpcpp`
+- `dpcpp_linux-64` or `dpcpp_win-64` (depending on your OS)
+- `tbb-devel`
+- `dpctl`
+
+In addition, you need oneDPL installed on your system. There are two ways to do
+so:
+
+1. Install oneAPI and run the oneDPL activation script. E.g., on linux:
+
+   ```bash
+   source /opt/intel/oneapi/dpl/latest/env/vars.sh
+   ```
+
+2. Clone dpl from https://github.com/oneapi-src/oneDPL and set the `DPL_ROOT`
+   environment variable to point to the `include` directory in the repository.
+
+   E.g., on linux
+
+   ```bash
+   git clone https://github.com/oneapi-src/oneDPL
+   export DPL_ROOT=$(pwd)/oneDPL/include
+   ```
+
+After these steps, `dpnp` can be built in debug mode as follows:
+
+
 ```bash
 git clone https://github.com/IntelPython/dpnp
 cd dpnp

dpnp/__init__.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # *****************************************************************************
-# Copyright (c) 2016-2020, Intel Corporation
+# Copyright (c) 2016-2022, Intel Corporation
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -30,7 +30,16 @@
 import dpctl
 dpctlpath = os.path.dirname(dpctl.__file__)
 
-os.environ["PATH"] += os.pathsep + mypath + os.pathsep + dpctlpath
+# For Windows OS with Python >= 3.7, it is required to explicitly define a path
+# where to search for DLLs towards both DPNP backend and DPCTL Sycl interface,
+# otherwise DPNP import will be failing. This is because the libraries
+# are not installed under any of default paths where Python is searching.
+from platform import system
+if system() == 'Windows':
+    if hasattr(os, "add_dll_directory"):
+        os.add_dll_directory(mypath)
+        os.add_dll_directory(dpctlpath)
+    os.environ["PATH"] = os.pathsep.join([os.getenv("PATH", ""), mypath, dpctlpath])
 
 
 from dpnp.dpnp_array import dpnp_array as ndarray