Update dlpack to v1.0rc by ndgrigorian · Pull Request #1667 · IntelPython/dpctl

ndgrigorian · 2024-05-09T04:33:22Z

This PR bumps the version of dlpack by bringing in the new dlpack.h from v1.0rc.

As a result of changes introduced in v1.0rc

get_build_dlpack_version now returns a tuple giving the major and minor DLPack version dpctl was built with
support for the new DLManagedTensorVersioned struct has been added, including full support of the new flags attribute
__dlpack__ method of usm_ndarray has been updated to choose between DLManagedTensorVersioned and DLManagedTensor paths based on the new max_version keyword argument, and also now has dl_device and copy keyword arguments
from_dlpack method has been updated with device and copy keyword arguments and now has a path for arrays supporting the new __dlpack__ interface and DLManagedTensorVersioned. from_dlpack also honors the read-only bitmask that can now be present on a DLManagedTensorVersioned object
an enumerator class for the DLPack device types has been added to _usmarray.pyx

This pull request also slips in a minor change to usm_ndarray's constructor to honor the writable flag when buffer is another usm_ndarray

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
If this PR is a work in progress, are you opening the PR as a draft?

github-actions · 2024-05-09T05:10:13Z

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

github-actions · 2024-05-09T05:11:03Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_336 ran successfully.
Passed: 884
Failed: 8
Skipped: 92

coveralls · 2024-05-09T05:11:24Z

coverage: 87.948% (+0.06%) from 87.892%
when pulling f0ced4e on feature/dlpack-1-support
into d3e124a on master.

ndgrigorian · 2024-05-09T07:06:39Z

@oleksandr-pavlyk
The commit from the memory work branch seems to have broken the dlpack tests on Windows.

github-actions · 2024-05-09T07:37:46Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_337 ran successfully.
Passed: 883
Failed: 9
Skipped: 92

github-actions · 2024-05-09T16:33:58Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_339 ran successfully.
Passed: 884
Failed: 8
Skipped: 92

github-actions · 2024-05-09T17:57:59Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_340 ran successfully.
Passed: 883
Failed: 9
Skipped: 92

Use `ary_base` rather than `usm_ary` as manager_ctx in DLManagedTensor struct. Tests for DLPack use fixture to provide all devices to take advantage of caching.

…read-only array as `buffer` argument

* Uses `dlpack.h` as per DLPack v1.0 rc * Implements `device` and `copy` keywords into `__dlpack__` and `from_dlpack` * Implements `max_version` argument into `__dlpack__` * Adds `DLDeviceType` enumerator to `_usmarray.pyx` for representing device types as enumerators * Reimplements `dpt._dlpack.get_build_dlpack_version` to return a tuple of the major version and the minor version

…ck_versioned_capsule` Also adjusts some error messages in _dlpack.pyx

These tweaks are based on similar changes made for the managed tensor during memory work

github-actions · 2024-05-10T03:12:21Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_340 ran successfully.
Passed: 884
Failed: 8
Skipped: 92

oleksandr-pavlyk · 2024-05-10T14:08:42Z

The check need tweaking:

In [9]: x.__dlpack__(max_version=1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[9], line 1
----> 1 x.__dlpack__(max_version=1)

File ~/repos/dpctl_tmp/dpctl/tensor/_usmarray.pyx:1131, in dpctl.tensor._usmarray.usm_ndarray.__dlpack__()
   1129 else:
   1130     dpctl_dlpack_version = get_build_dlpack_version()
-> 1131     if max_version >= dpctl_dlpack_version or max_version[0] == dpctl_dlpack_version[0]:
   1132         # DLManagedTensorVersioned path
   1133         # TODO: add logic for targeting a device

TypeError: '>=' not supported between instances of 'int' and 'tuple'

The check should check that max_version is a 2-tuple of integers. But looking at this check, what is the purpose of overriding clause max_version[0] == dpctl_dlpack_version[0]?

oleksandr-pavlyk · 2024-05-10T14:38:03Z

Please update dpctl/tensor/include/dlpack/README.md with version and SHA of the vendored dlpack.h

ndgrigorian · 2024-05-10T15:34:24Z

The check need tweaking:

In [9]: x.__dlpack__(max_version=1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[9], line 1
----> 1 x.__dlpack__(max_version=1)

File ~/repos/dpctl_tmp/dpctl/tensor/_usmarray.pyx:1131, in dpctl.tensor._usmarray.usm_ndarray.__dlpack__()
   1129 else:
   1130     dpctl_dlpack_version = get_build_dlpack_version()
-> 1131     if max_version >= dpctl_dlpack_version or max_version[0] == dpctl_dlpack_version[0]:
   1132         # DLManagedTensorVersioned path
   1133         # TODO: add logic for targeting a device

TypeError: '>=' not supported between instances of 'int' and 'tuple'

The check should check that max_version is a 2-tuple of integers. But looking at this check, what is the purpose of overriding clause max_version[0] == dpctl_dlpack_version[0]?

max_version[0] >= dpctl_dlpack_version[0] should be enough.

The spec recommends that if the major versions match, it's a strong enough condition to return a capsule with our own max_version.

DLPack version updated and new SHA for `dlpack.h` added

github-actions · 2024-05-10T19:17:21Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_344 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

* Fixture all_root_devices is to keep only 2 devices from each platform This is done to speed-up test suite execution on multi-GPU system, like Aurora. * Add tests for legacy/versions capsule import-export * Expand tests more to improve coverage Use F-contiguous array to both legacy and versioned capsule. Add tests for read-only array and for use of copy=True keyword for __dlpack__ call

github-actions · 2024-05-10T22:33:44Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_345 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

github-actions · 2024-05-10T23:10:01Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_346 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

Also adds a test checking for the writable flag of a usm_ndarray constructed with a read-only usm_ndarray as the buffer Removes some commented out `flags` checks from `test_meshgrid2`. These checks were malformed, as `dpt.meshgrid` returns a non-contiguous view in the test.

github-actions · 2024-05-10T23:47:29Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_346 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

oleksandr-pavlyk · 2024-05-11T00:24:33Z

It would be good to factor out common code from to_dlpack_capsule and to_dlpack_versioned_capsule into a function.

Reuse that in both capsule producers to avoid code duplication

…ation

github-actions · 2024-05-11T18:07:48Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_348 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

* Reused get_parent_device_ordinal_id routine * test_legacy_dlpack_capsule uses 4 kinds of dtype Added test to use non-default copy keyword, and non-default device keyword argument.

github-actions · 2024-05-12T23:11:30Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_349 ran successfully.
Passed: 889
Failed: 16
Skipped: 91

github-actions · 2024-05-12T23:30:33Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_351 ran successfully.
Passed: 888
Failed: 17
Skipped: 91

github-actions · 2024-05-13T00:17:20Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_352 ran successfully.
Passed: 889
Failed: 16
Skipped: 91

oleksandr-pavlyk

LGTM! Thank you @ndgrigorian

ndgrigorian requested a review from oleksandr-pavlyk May 9, 2024 04:33

oleksandr-pavlyk reviewed May 9, 2024

View reviewed changes

Comment thread dpctl/tensor/_dlpack.pyx Outdated

ndgrigorian added 8 commits May 9, 2024 18:33

Implements DLPack changes necessitated by memory object work

83241ee

Use `ary_base` rather than `usm_ary` as manager_ctx in DLManagedTensor struct. Tests for DLPack use fixture to provide all devices to take advantage of caching.

usm_ndarray constructor now honors the writable flag when passed a …

da0f04f

…read-only array as `buffer` argument

Adds check for major version of versioned dlpack tensor to `from_dlpa…

8697c83

…ck_versioned_capsule` Also adjusts some error messages in _dlpack.pyx

Add missing device_type enumerators from dlpack.h

e9a3396

usm_ary -> ary_base in to_dlpack_versioned_capsule

a113eaa

Changes to exception messages throughout _dlpack.pyx

a7d255a

Versioned DLpack tensor deleter and owner tweaked

c4b34e5

These tweaks are based on similar changes made for the managed tensor during memory work

ndgrigorian force-pushed the feature/dlpack-1-support branch from 6920790 to c4b34e5 Compare May 10, 2024 02:34

oleksandr-pavlyk mentioned this pull request May 10, 2024

Add more dlpack tests #1670

Merged

6 tasks

ndgrigorian added 2 commits May 10, 2024 09:14

Updates dpctl/tensor/include/dlpack/README.md

c54a90a

DLPack version updated and new SHA for `dlpack.h` added

Adds check that max_version is a 2-tuple to __dlpack__

9afae01

ndgrigorian marked this pull request as ready for review May 10, 2024 19:04

ndgrigorian force-pushed the feature/dlpack-1-support branch from 99f03ce to 85f12d0 Compare May 10, 2024 23:10

oleksandr-pavlyk added 2 commits May 10, 2024 22:02

Factor out device-id computation

470bb7c

Reuse that in both capsule producers to avoid code duplication

Merge pull request #1671 from IntelPython/factor-out-device-id-comput…

1205b0b

…ation

oleksandr-pavlyk and others added 3 commits May 12, 2024 15:32

Reuse get parent device ordinal id routine (#1672)

3e3ab03

* Reused get_parent_device_ordinal_id routine * test_legacy_dlpack_capsule uses 4 kinds of dtype Added test to use non-default copy keyword, and non-default device keyword argument.

Adds more DLPack tests to improve coverage

7a0e4d1

Adds a test for boolean arrays in test_legacy_dlpack_capsule

bff131a

DLPack tests for copy and stream kwargs in __dlpack__

f0ced4e

ndgrigorian requested a review from oleksandr-pavlyk May 12, 2024 23:41

oleksandr-pavlyk approved these changes May 13, 2024

View reviewed changes

ndgrigorian merged commit 8b42313 into master May 13, 2024

ndgrigorian deleted the feature/dlpack-1-support branch May 13, 2024 04:40

antonwolfy mentioned this pull request Aug 14, 2024

Align with dpctl changes for DLPack v1.0 support IntelPython/dpnp#1980

Merged

6 tasks

Conversation

ndgrigorian commented May 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 9, 2024

Uh oh!

coveralls commented May 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ndgrigorian commented May 9, 2024

Uh oh!

github-actions Bot commented May 9, 2024

Uh oh!

Uh oh!

github-actions Bot commented May 9, 2024

Uh oh!

github-actions Bot commented May 9, 2024

Uh oh!

github-actions Bot commented May 10, 2024

Uh oh!

oleksandr-pavlyk commented May 10, 2024

Uh oh!

oleksandr-pavlyk commented May 10, 2024

Uh oh!

ndgrigorian commented May 10, 2024

Uh oh!

github-actions Bot commented May 10, 2024

Uh oh!

github-actions Bot commented May 10, 2024

Uh oh!

github-actions Bot commented May 10, 2024

Uh oh!

github-actions Bot commented May 10, 2024

Uh oh!

oleksandr-pavlyk commented May 11, 2024

Uh oh!

github-actions Bot commented May 11, 2024

Uh oh!

github-actions Bot commented May 12, 2024

Uh oh!

github-actions Bot commented May 12, 2024

Uh oh!

github-actions Bot commented May 13, 2024

Uh oh!

oleksandr-pavlyk left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ndgrigorian commented May 9, 2024 •

edited

Loading

github-actions Bot commented May 9, 2024 •

edited

Loading

coveralls commented May 9, 2024 •

edited

Loading