Overhaul of dpctl documentation by oleksandr-pavlyk · Pull Request #1619 · IntelPython/dpctl

oleksandr-pavlyk · 2024-03-28T15:00:39Z

This PR is a rewrite of dpctl's documentation. Here is the link to rendered docs based on sources in this PR.

It introduces Beginner's guide, User Guide, Reference Guide and contributor's guide
It does away with generate_rst.py and AutoAutosummary class, but uses Sphinx's autosummary extension to generate individual pages for classes, and array API functions.
It changes theme from read-the-docs to Furo
It modifies docstrings to fix sphinx issues
It modifies array API function signatures by using positional-only, keyword-only delimiters as done in the standard.
This required test changes and code changes.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
If this PR is a work in progress, are you opening the PR as a draft?

diptorupd · 2024-03-28T15:15:35Z

@oleksandr-pavlyk the PR will need to be split up. I suggest all docstring changes to be merged separately.

github-actions · 2024-03-28T15:39:47Z

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

github-actions · 2024-03-28T15:41:13Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T15:57:42Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T16:33:32Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

coveralls · 2024-03-28T16:56:55Z

coverage: 88.049%. remained the same
when pulling 1abd650 on docstring-edits
into 10b831c on master.

github-actions · 2024-03-28T17:03:53Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_182 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T19:04:56Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_186 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-28T19:45:22Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_187 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-28T21:00:56Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_189 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-31T03:01:34Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_191 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

oleksandr-pavlyk · 2024-04-01T13:24:07Z

@oleksandr-pavlyk the PR will need to be split up. I suggest all docstring changes to be merged separately.

I split the PR into gh-1625, and remaining docs-only changes in this PR opened into branch behind gh-1625.

github-actions · 2024-04-01T13:57:33Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_190 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-01T14:01:49Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_191 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-01T18:03:05Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_192 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T01:40:51Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_199 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T02:10:20Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_200 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T13:34:16Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_201 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T16:15:08Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_202 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-16T15:41:42Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_285 ran successfully.
Passed: 870
Failed: 8
Skipped: 92

ndgrigorian · 2024-04-16T22:13:33Z

+
+.. note::
+    Two instances :class:`dpctl.SyclQueue` may target the same ``sycl::device`` and be using the same ``sycl::context``, but correspond
+    to different scheduling enties, and hence be in violation of the compute-follows-data requirement. One common example of this are


Nit, but was this meant to say "entries"?

ndgrigorian · 2024-04-16T22:17:55Z

+If assignign USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,
+the USM-type of the output array has the smallest score of all its inputs.
+
+.. currentmodule:: dpctl.utils
+
+The convenience function :py:func:`get_coerced_usm_type` is a convenience function to determine the USM-type
+following this convention:


Suggested change

If assignign USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,

the USM-type of the output array has the smallest score of all its inputs.

.. currentmodule:: dpctl.utils

The convenience function :py:func:`get_coerced_usm_type` is a convenience function to determine the USM-type

following this convention:

If assigning USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,

the USM-type of the output array has the smallest score of all its inputs.

.. currentmodule:: dpctl.utils

The convenience function :py:func:`get_coerced_usm_type` determines the USM-type following this convention:

ndgrigorian · 2024-04-16T22:21:21Z

+    # r3 has value "host"
+    r3 = get_coerced_usm_type(["host", "host", "host"])
+
+Sharing data between devices and Python


It may be worthwhile to add an example demonstrating what happens when migrating a Numpy array of a data type not supported by the destination device.

Can be done later though.

ndgrigorian · 2024-04-16T22:23:14Z

+-------------
+
+* History of ``"dpctl"`` :ref:`name <beginners_guide_why_dpctl>`
+* Frequenty asked questions


Suggested change

* Frequenty asked questions

* Frequently asked questions

ndgrigorian · 2024-04-16T22:25:33Z

@@ -0,0 +1,69 @@
+.. _beginners_guide_why_dpctl:
+
+History of ``dpctl`` name


I wonder if this page should either be renamed or moved to the beginner's guides index page in the future. It stands out when looking at the list.

ndgrigorian · 2024-04-16T22:28:30Z

+   :class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``
+   could not be combined in a call to the same function that implementes
+   :ref:`compute-followed-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.


Suggested change

:class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``

could not be combined in a call to the same function that implementes

:ref:`compute-followed-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.

:class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``

could not be combined in a call to the same function that implements

:ref:`compute-follows-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.

ndgrigorian · 2024-04-16T22:34:32Z

+        x1 : dpctl.tensor.usm_ndarray,
+        x2 : dpctl.tensor.usm_ndarray
+    ):
+        exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))


Suggested change

exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))

exec_q = dpctl.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))

ndgrigorian · 2024-04-16T22:35:53Z

+        ...
+
+In order to ensure that compute-follows-data works seemlessly out-of-the-box, :mod:`dpctl` maintains
+a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.


Suggested change

a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.

a cache with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.

ndgrigorian · 2024-04-16T22:43:21Z

+    const auto &all_root_devs = sycl::device::get_devices();
+    auto beg = std::begin(all_root_devs);
+    auto end = std::end(all_root_devs);
+    auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {


Suggested change

auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {

auto selector_fn = [parent_root_device](const sycl::device &root_d) -> bool {

ndgrigorian · 2024-04-16T22:51:54Z

+    select_cpu_device
+    select_gpu_device
+    select_accelerator_device
+    select_device_with_aspects


This function could use a proper docstring later.

This was referring specifically to select_device_with_aspects, which has a description that isn't a true docstring (no real info about what types are permitted for a required_aspect, for instance).

ndgrigorian · 2024-04-16T23:03:58Z

+    SyclProgram
+    SyclKernel


Empty __init__() methods are picked up in the documentation of these, and the methods don't have pages or proper docstrings.

ndgrigorian · 2024-04-16T23:24:31Z

+:py:mod:`dpctl.tensor` is written using C++ and :sycl_spec_2020:`SYCL <>`
+and oneAPI extensions implemented in :dpcpp_compiler:`Intel(R) oneAPI DPC++ compiler <>`.
+
+This module contains:


Some undocumented things:

~~all constants (pi, e, nan, inf, and newaxis)~~

asnumpy and to_numpy

dtype itself

in elementwise functions, no documentation is rendered for nin, nout, types, etc. special attributes

print utilities do not have rendered documentation

~~flags object on usm_ndarray is mentioned under usm_ndarray.flags but has no rendered documentation~~

order keyword in general/memory layouts (C-contiguous vs. F-contiguous) may be worth a quick note

ndgrigorian · 2024-04-16T23:37:26Z

@oleksandr-pavlyk
In the future it may also be nice to change the libsyclinterface documentation rendering so that the global functions and so on are listed directly under the "on this page" section, with shortcuts. That way you don't have to scroll and click on them in the overview.

Some found in review, some identified by spell-checker tool.

…look at

Also added few workd to the list is known-to-be-correctly-spelled

github-actions · 2024-04-17T15:48:46Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_291 ran successfully.
Passed: 871
Failed: 7
Skipped: 92

ndgrigorian

Changes requested in my review have now either been committed or will be deferred to a later PR.

This LGTM, thank you @oleksandr-pavlyk! Very exciting to see these documentation improvements finally go in.

diptorupd

Let's merge it as-is for now, the changes in the PR are already a big improvement over status quo.

Also, the PR has become too large and all further change should be done incrementally.

oleksandr-pavlyk force-pushed the docstring-edits branch from f58650f to 0567168 Compare March 28, 2024 15:19

oleksandr-pavlyk force-pushed the docstring-edits branch from 0567168 to 2348853 Compare March 28, 2024 15:59

oleksandr-pavlyk marked this pull request as draft March 28, 2024 17:14

oleksandr-pavlyk mentioned this pull request Apr 1, 2024

Docstring edits only changes in documentation overhaul #1626

Merged

6 tasks

oleksandr-pavlyk force-pushed the docstring-edits branch from a597ccc to 4da37bc Compare April 1, 2024 13:23

oleksandr-pavlyk changed the base branch from master to docstring-edits-only April 1, 2024 13:23

oleksandr-pavlyk force-pushed the docstring-edits-only branch from 891a1c5 to 7ac9ff2 Compare April 1, 2024 17:25

oleksandr-pavlyk force-pushed the docstring-edits branch from 4840c5f to 6ef7cc5 Compare April 1, 2024 17:25

vtavana reviewed Apr 1, 2024

View reviewed changes

Comment thread docs/doc_sources/beginners_guides/index.rst

Comment thread docs/doc_sources/beginners_guides/parallelisms.rst Outdated

Comment thread docs/doc_sources/beginners_guides/device_info.rst Outdated

Comment thread docs/doc_sources/beginners_guides/environment_variables.rst Outdated

vtavana reviewed Apr 1, 2024

View reviewed changes

Comment thread docs/doc_sources/beginners_guides/environment_variables.rst Outdated

oleksandr-pavlyk force-pushed the docstring-edits branch from 4463a84 to 9bc0a86 Compare April 2, 2024 01:36

ndgrigorian reviewed Apr 16, 2024

View reviewed changes

oleksandr-pavlyk changed the title ~~WIP: overhaul of dpctl documentation~~ Overhaul of dpctl documentation Apr 17, 2024

oleksandr-pavlyk added 6 commits April 17, 2024 08:38

Using sphinxcontrib.spelling as extension

ee22e70

Spelling fixes

175eba8

Some found in review, some identified by spell-checker tool.

Reworded no longer accurate statement that spell-checker prompted to …

a1093d9

…look at

Typos found by @ndigrigorian

b7d9515

Also added few workd to the list is known-to-be-correctly-spelled

Writeable->writable

b9fb29f

Fixed typo, added more known-as-spelled-correctly words

1abd650

ndgrigorian approved these changes Apr 17, 2024

View reviewed changes

diptorupd approved these changes Apr 17, 2024

View reviewed changes

oleksandr-pavlyk merged commit 182aede into master Apr 17, 2024

oleksandr-pavlyk deleted the docstring-edits branch April 17, 2024 19:10

ndgrigorian mentioned this pull request Apr 29, 2024

Render documentation for elementwise functions #1297

Closed

oleksandr-pavlyk mentioned this pull request Jul 19, 2024

Removed use of deprecated SYCL_DEVICE_FILTER #1740

Merged

6 tasks

ndgrigorian mentioned this pull request Feb 27, 2025

Add get_devices method to dpctl.SyclPlatform class #1992

Merged

8 tasks

ndgrigorian mentioned this pull request Apr 9, 2025

Update docs/requirements.txt #2045

Open

		@@ -0,0 +1,69 @@
		.. _beginners_guide_why_dpctl:

		History of ``dpctl`` name

	exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))
	exec_q = dpctl.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))

	a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.
	a cache with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.

	auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {
	auto selector_fn = [parent_root_device](const sycl::device &root_d) -> bool {

Conversation

oleksandr-pavlyk commented Mar 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diptorupd commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

coveralls commented Mar 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 28, 2024

Uh oh!

github-actions Bot commented Mar 31, 2024

Uh oh!

oleksandr-pavlyk commented Apr 1, 2024

Uh oh!

github-actions Bot commented Apr 1, 2024

Uh oh!

github-actions Bot commented Apr 1, 2024

Uh oh!

github-actions Bot commented Apr 1, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Apr 2, 2024

Uh oh!

github-actions Bot commented Apr 2, 2024

Uh oh!

github-actions Bot commented Apr 2, 2024

Uh oh!

github-actions Bot commented Apr 2, 2024

Uh oh!

github-actions Bot commented Apr 16, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndgrigorian Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndgrigorian Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

oleksandr-pavlyk commented Mar 28, 2024 •

edited

Loading

github-actions Bot commented Mar 28, 2024 •

edited

Loading

coveralls commented Mar 28, 2024 •

edited

Loading

ndgrigorian Apr 16, 2024 •

edited

Loading

ndgrigorian Apr 16, 2024 •

edited

Loading

ndgrigorian commented Apr 16, 2024 •

edited

Loading