Skip to content

Backport gh-2166, gh-2057, gh-2211 #2214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/build-sphinx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/conda-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -197,7 +196,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -327,7 +325,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -469,7 +466,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -505,7 +501,6 @@ jobs:
steps:
- uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/generate_coverage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,23 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.16.1] - 12/XX/2024

This is a bug-fix release.

### Changed

* Changed to use `Miniforge` installer in GutHub actions [#2057](https://github.com/IntelPython/dpnp/pull/2057)
* Updated `README.md` to reflect current installation requirements and available options [#2166](https://github.com/IntelPython/dpnp/pull/2166)

### Fixed

* Resolved a compilation error when building with DPC++ 2025.1 compiler [#2211](https://github.com/IntelPython/dpnp/pull/2211)


## [0.16.0] - 10/14/2024

This release reaches an important milestone by making offloading fully asynchronous. Calls to `dpnp` submit tasks for execution to DPC++ runtime and return without waiting for execution of these tasks to finish. The sequential semantics a user comes to expect from execution of Python script is preserved though.
Expand Down
129 changes: 44 additions & 85 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,117 +6,76 @@
[![Build Sphinx](https://github.com/IntelPython/dpnp/workflows/Build%20Sphinx/badge.svg)](https://intelpython.github.io/dpnp)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/IntelPython/dpnp/badge)](https://securityscorecards.dev/viewer/?uri=github.com/IntelPython/dpnp)

<img align="left" src="https://spec.oneapi.io/oneapi-logo-white-scaled.jpg" alt="oneAPI logo" width="75"/>

# DPNP - Data Parallel Extension for NumPy*

Data Parallel Extension for NumPy* or `dpnp` is a Python library that
implements a subset of NumPy* that can be executed on any data parallel device.
The subset is a drop-in replacement of core NumPy* functions and numerical data types.

[API coverage summary](https://intelpython.github.io/dpnp/reference/comparison.html#summary)

[Full documentation](https://intelpython.github.io/dpnp/)

[DPNP C++ backend documentation](https://intelpython.github.io/dpnp/backend_doc/)
`Dpnp` is the core part of a larger family of [data-parallel Python libraries and tools](https://www.intel.com/content/www/us/en/developer/tools/oneapi/distribution-for-python.html)
to program on XPUs.

## Build from source:
Ensure you have the following prerequisite packages installed:

- `cython`
- `cmake >=3.21`
- `dpcpp_linux-64` or `dpcpp_win-64` (depending on your OS)
- `dpctl`
- `mkl-devel-dpcpp`
- `onedpl-devel`
- `ninja`
- `numpy >=1.19,<1.25a0`
- `python`
- `scikit-build`
- `setuptools`
- `sysroot_linux-64 >=2.28` (only on Linux OS)
- `tbb-devel`
# Installing

After these steps, `dpnp` can be built in debug mode as follows:
You can install the library using `conda`, `mamba` or [pip](https://pypi.org/project/dpnp/)
package managers. It is also available as part of the [Intel(R) Distribution for Python](https://www.intel.com/content/www/us/en/developer/tools/oneapi/distribution-for-python.html)
(IDP).

```bash
git clone https://github.com/IntelPython/dpnp
cd dpnp
python scripts/build_locally.py
```
## Intel(R) Distribution for Python

## Install Wheel Package via pip
Install DPNP
```cmd
python -m pip install --index-url https://software.repos.intel.com/python/pypi dpnp
```
You can find the most recent release of `dpnp` every quarter as part of the IDP
releases.

Set path to Performance Libraries in case of using venv or system Python:
```cmd
export LD_LIBRARY_PATH=<path_to_your_env>/lib
```
To get the library from the latest release, follow the instructions from
[Get Started With Intel® Distribution for Python](https://www.intel.com/content/www/us/en/developer/articles/technical/get-started-with-intel-distribution-for-python.html).

It is also required to set following environment variables:
```cmd
export OCL_ICD_FILENAMES_RESET=1
export OCL_ICD_FILENAMES=libintelocl.so
```
## Conda

## Run test
```bash
pytest
# or
pytest tests/test_matmul.py -s -v
# or
python -m unittest tests/test_mixins.py
```
To install `dpnp` from the Intel(R) conda channel, use the following command:

## Run numpy external test
```bash
. ./0.env.sh
python -m tests.third_party.numpy_ext
# or
python -m tests.third_party.numpy_ext core/tests/test_umath.py
# or
python -m tests.third_party.numpy_ext core/tests/test_umath.py::TestHypot::test_simple
conda install dpnp -c https://software.repos.intel.com/python/conda/ -c conda-forge
```

### Building documentation:
## Pip

The `dpnp` can be installed using `pip` obtaining wheel packages either from
PyPi or from Intel(R) channel. To install `dpnp` wheel package from Intel(R)
channel, run the following command:

```bash
Prerequisites:
$ conda install sphinx sphinx_rtd_theme
Building:
1. Install dpnp into your python environment
2. $ cd doc && make html
3. The documentation will be in doc/_build/html
python -m pip install --index-url https://software.repos.intel.com/python/pypi dpnp
```

## Packaging:
## Installing the bleeding edge

To try out the latest features, install `dpnp` using our development channel on
Anaconda cloud:

```bash
. ./0.env.sh
conda-build conda-recipe/
conda install dpnp -c dppy/label/dev -c https://software.repos.intel.com/python/conda/ -c conda-forge
```

## Run benchmark:
```bash
cd benchmarks/

asv run --python=python --bench <filename without .py>
# example:
asv run --python=python --bench bench_elementwise
# Building

# or
Refer to our [Documentation](https://intelpython.github.io/dpnp/quick_start_guide.html)
for more information on setting up a development environment and building `dpnp`
from the source.

asv run --python=python --bench <class>.<bench>
# example:
asv run --python=python --bench Elementwise.time_square

# add --quick option to run every case once but looks like first execution has additional overheads and takes a lot of time (need to be investigated)
```
# Running Tests

Tests are located in folder [dpnp/tests](dpnp/tests).

## Tests matrix:
| # |Name |OS |distributive|interpreter|python used from|SYCL queue manager|build commands set |forced environment |
|---|------------------------------------|-----|------------|-----------|:--------------:|:----------------:|------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
|1 |Ubuntu 20.04 Python37 |Linux|Ubuntu 20.04|Python 3.7 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|2 |Ubuntu 20.04 Python38 |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|3 |Ubuntu 20.04 Python39 |Linux|Ubuntu 20.04|Python 3.9 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|4 |Ubuntu 20.04 External Tests Python37|Linux|Ubuntu 20.04|Python 3.7 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|5 |Ubuntu 20.04 External Tests Python38|Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|6 |Ubuntu 20.04 External Tests Python39|Linux|Ubuntu 20.04|Python 3.9 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|7 |Code style |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |python ./setup.py style |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis, conda-verify, pycodestyle, autopep8, black |
|8 |Valgrind |Linux|Ubuntu 20.04| | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|9 |Code coverage |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis, conda-verify, pycodestyle, autopep8, pytest-cov|
To run the tests, use:
```bash
python -m pytest --pyargs dpnp
```
21 changes: 19 additions & 2 deletions dpnp/backend/kernels/dpnp_krnl_common.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,23 @@
#include "queue_sycl.hpp"
#include <dpnp_iface.hpp>

/**
* Version of SYCL DPC++ 2025.1 compiler where support of
* sycl::ext::oneapi::experimental::properties was added.
*/
#ifndef __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
#define __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT 20241129
#endif

namespace mkl_blas = oneapi::mkl::blas;
namespace mkl_blas_cm = oneapi::mkl::blas::column_major;
namespace mkl_blas_rm = oneapi::mkl::blas::row_major;
namespace mkl_lapack = oneapi::mkl::lapack;

#if __SYCL_COMPILER_VERSION >= __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
namespace syclex = sycl::ext::oneapi::experimental;
#endif

template <typename _KernelNameSpecialization1,
typename _KernelNameSpecialization2,
typename _KernelNameSpecialization3>
Expand Down Expand Up @@ -76,8 +88,13 @@ sycl::event dot(sycl::queue &queue,
cgh.parallel_for(
sycl::range<1>{size},
sycl::reduction(
result_out, std::plus<_DataType_output>(),
sycl::property::reduction::initialize_to_identity{}),
result_out, sycl::plus<_DataType_output>(),
#if __SYCL_COMPILER_VERSION >= __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
syclex::properties(syclex::initialize_to_identity)
#else
sycl::property::reduction::initialize_to_identity {}
#endif
),
[=](sycl::id<1> idx, auto &sum) {
sum += static_cast<_DataType_output>(
input1_in[idx * input1_strides]) *
Expand Down
2 changes: 1 addition & 1 deletion tests/third_party/cupy/creation_tests/test_ranges.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ def test_linspace_float_overflow(self, xp):
dtype = cupy.default_float_type()
return xp.linspace(0.0, xp.finfo(dtype).max / 5, 10, dtype=dtype)

@testing.numpy_cupy_allclose()
@testing.numpy_cupy_allclose(rtol={numpy.float32: 1e-6, "default": 1e-7})
def test_linspace_float_underflow(self, xp):
# find minimum subnormal number
dtype = cupy.default_float_type()
Expand Down
Loading