Skip to content

sycl: add GGML_SYCL_USE_ASYNC_MEM_OP env toggle#22153

Open
aicss-genai wants to merge 2 commits intoggml-org:masterfrom
aicss-genai:aicss-genai/sycl-bmg-upstream-pr-6
Open

sycl: add GGML_SYCL_USE_ASYNC_MEM_OP env toggle#22153
aicss-genai wants to merge 2 commits intoggml-org:masterfrom
aicss-genai:aicss-genai/sycl-bmg-upstream-pr-6

Conversation

@aicss-genai
Copy link
Copy Markdown

@aicss-genai aicss-genai commented Apr 20, 2026 •
edited
Loading

Overview

Authors

Exposes async USM allocation/free as an independent env knob.

Previously g_ggml_sycl_use_async_mem_op was implicitly tied to whether graphs were enabled (= !g_ggml_sycl_disable_graph). The reorder staging path benefits from async alloc/free even without graphs — it avoids host waits while preserving queue ordering.

Adds GGML_SYCL_USE_ASYNC_MEM_OP (default 1) read at init, logged alongside the other SYCL env vars. The SYCL extension availability check is unchanged.

Additional information

Split from #22066 per reviewer request for independent review.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: Yes. This work was partially produced with an agentic engineering approach: agents surface issues and explore experiments while engineers identify and reject candidates using domain knowledge. Human feedback involved.

@aicss-genai aicss-genai requested a review from a team as a code owner April 20, 2026 07:05
@ggml-gh-bot
Copy link
Copy Markdown

ggml-gh-bot Bot commented Apr 20, 2026

Hi @aicss-genai, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 7 open PRs.

  • AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Apr 20, 2026
NeoZhangJianyu
NeoZhangJianyu reviewed Apr 22, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@NeoZhangJianyu NeoZhangJianyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Async Mem is used for SYCL graph.
Following code is mandatory:

g_ggml_sycl_use_async_mem_op = !g_ggml_sycl_disable_graph

I don't see the value of new global parameter.
Without SYCL graph, only use async memory copy has not benefit.

Is there any test result to show the benefit of this PR?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu left review comments

Reviewers whose approvals may not affect merge requirements

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aicss-genai @NeoZhangJianyu @ctao456