Skip to content

Use cached default device in tensor#1568

Merged
oleksandr-pavlyk merged 1 commit intomasterfrom
use-cached-default-device-in-tensor
Feb 28, 2024
Merged

Use cached default device in tensor#1568
oleksandr-pavlyk merged 1 commit intomasterfrom
use-cached-default-device-in-tensor

Conversation

@oleksandr-pavlyk
Copy link
Copy Markdown
Contributor

@oleksandr-pavlyk oleksandr-pavlyk commented Feb 28, 2024 •
edited
Loading

Profiling of an example of GPU Max revealed that repeated calls to dpctl.select_default_device() took 200x longer than actual computatations.

Introduced dpctl._sycl_device_factory._cached_default_device() and deployed it in tensor (in "_usmarray.pyx" and in "_device.py").

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you opening the PR as a draft?

@oleksandr-pavlyk
Use cached default device in tensor
fb47db4 
Profiling of an example of GPU Max revealed that repeated calls to
dpctl.select_default_device() took 200x longer than actual computatations.

Introduced dpctl._sycl_device_factory._cached_default_device() and deployed
it in tensor (in _usmarray and in _device).
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 28, 2024 •
edited
Loading

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

@coveralls
Copy link
Copy Markdown
Collaborator

coveralls commented Feb 28, 2024

Coverage Status

coverage: 91.072% (-0.03%) from 91.099%
when pulling fb47db4 on use-cached-default-device-in-tensor
into be4a01c on master.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 28, 2024

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_33 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

@oleksandr-pavlyk oleksandr-pavlyk marked this pull request as ready for review February 28, 2024 19:32
@oleksandr-pavlyk
Copy link
Copy Markdown
Contributor Author

oleksandr-pavlyk commented Feb 28, 2024

Verifying on a machine with two GPU Max cards:

In [3]: %timeit foo_dpt(1_000_000, None)
218 µs ± 55.5 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [4]: %timeit foo_dpt(1_000_000, None)
129 µs ± 2.35 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

In [5]: q = dpctl.SyclQueue()

In [6]: %timeit foo_dpt(1_000_000, q)
154 µs ± 29.1 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [7]: %timeit foo_dpt(1_000_000, q)
118 µs ± 15 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

Previously, the timing was

In [3]: %timeit foo_dpt(1_000_000, None)
6.87 ms ± 99.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [4]: %timeit foo_dpt(1_000_000, None)
7.07 ms ± 18.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [5]: q = dpctl.SyclQueue()

In [6]: %timeit foo_dpt(1_000_000, q)
280 µs ± 81.3 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [7]: %timeit foo_dpt(1_000_000, q)
248 µs ± 3.45 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

ndgrigorian
ndgrigorian approved these changes Feb 28, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@ndgrigorian ndgrigorian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is a good change, thank you @oleksandr-pavlyk

@oleksandr-pavlyk oleksandr-pavlyk merged commit ea40d71 into master Feb 28, 2024
@oleksandr-pavlyk oleksandr-pavlyk deleted the use-cached-default-device-in-tensor branch February 28, 2024 23:41
@oleksandr-pavlyk oleksandr-pavlyk mentioned this pull request Mar 26, 2024
Merged
6 tasks
oleksandr-pavlyk added a commit that referenced this pull request Mar 27, 2024
@oleksandr-pavlyk
Merge pull request #1606 from IntelPython/backport-gh-1568
fc26a64 
Backport gh-1568 to 0.16.x maintenance branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ndgrigorian ndgrigorian ndgrigorian approved these changes

@diptorupd diptorupd Awaiting requested review from diptorupd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@oleksandr-pavlyk @coveralls @ndgrigorian