Skip to content

Add unit tests and type-safe overloads for CUDA kernel functions#800

Open
saschiwy wants to merge 1 commit into
taskflow:masterfrom
saschiwy:feature/typesafe-kernel-deduction
Open

Add unit tests and type-safe overloads for CUDA kernel functions#800
saschiwy wants to merge 1 commit into
taskflow:masterfrom
saschiwy:feature/typesafe-kernel-deduction

Conversation

@saschiwy

@saschiwy saschiwy commented Jul 6, 2026

Copy link
Copy Markdown

Add type-safe kernel() overloads to cudaGraphBase and cudaGraphExecBase


Acknowledgement

First and foremost: a huge thank you to Dr. Tsung-Wei Huang and all contributors for building
and maintaining Taskflow. It is one of the most elegant and well-engineered C++ frameworks I
have encountered, and I rely on it in production code daily.


Background

I use Taskflow in a production pipeline that runs on both Linux and Windows. Over time I ran
into a platform-dependent crash that took a while to track down. The root cause turned out to
be a completely silent type mismatch in a cudaGraph::kernel() call:

static constexpr auto DATA_SIZE = 256; // deduced as int
graph.kernel(grid, block, 0, my_kernel, d_ptr, DATA_SIZE); // kernel expects size_t

On Linux this worked fine — GCC's stack layout happens to zero the upper 4 bytes of the
8-byte size_t slot, so the driver reads the correct value by accident. On Windows the upper
bytes contain garbage, and the CUDA driver interprets the argument as something like
0x????????00000100, causing the kernel to access memory far out of bounds and fail with
cudaErrorIllegalAddress.

To be clear: this is not a bug in Taskflow. The existing API is intentionally generic and
fully correct within the C++ rules. The mismatch lives entirely at the call site.

That said, I thought others might benefit from having these errors surface at compile time
rather than as hard-to-reproduce runtime crashes — especially in cross-platform codebases
where the Linux/Windows ABI difference means the bug is invisible during development and only
shows up in production.


What this PR adds

A new overload of kernel() for both cudaGraphBase and cudaGraphExecBase that accepts a
typed __global__ function pointer (void(*)(Params...)) instead of the generic
template<typename F> callable. Because the function pointer carries the exact parameter
types in its type, the compiler can enforce correct argument types before anything is packed
into the void* array.

New helper: tf::detail::kernelArgCast

namespace detail {

template<typename TParam, typename TArg>
constexpr TParam kernelArgCast(TArg&& arg) {
    if constexpr (std::is_pointer_v<TParam>) {
        return reinterpret_cast<TParam>(arg);   // pointer: handles T*→const T*, typedef aliases
    } else {
        return static_cast<TParam>(std::forward<TArg>(arg));  // scalar: width mismatch → compile error
    }
}

} // namespace detail

Two cast strategies are used deliberately:

  • Scalars → static_cast: A narrower type (e.g. int) being passed where a wider type
    (e.g. size_t) is expected results in a widened, correct value. A genuinely incompatible
    type (a struct, an unrelated enum) produces a compile error.
  • Pointers → reinterpret_cast: Required to handle two common real-world cases that
    static_cast rejects in CUDA's augmented type system:
    • T*const T* const-qualification (nvcc rejects this via static_cast)
    • Typedef aliases like typedef bool boolean_T vs __nv_bool* that code generators produce

New overload in cudaGraphBase (cuda_graph.hpp)

template <typename... Params, typename... ArgsT>
cudaTask kernel(dim3 g, dim3 b, size_t s, void(*f)(Params...), ArgsT&&... args) {
    static_assert(sizeof...(Params) == sizeof...(ArgsT),
        "kernel: argument count does not match kernel parameter count");

    auto castedArgs = std::make_tuple(
        detail::kernelArgCast<Params>(std::forward<ArgsT>(args))...
    );

    cudaGraphNode_t node;
    cudaKernelNodeParams p;

    void* arguments[sizeof...(Params)];
    [&]<std::size_t... Is>(std::index_sequence<Is...>) {
        ((arguments[Is] = static_cast<void*>(&std::get<Is>(castedArgs))), ...);
    }(std::make_index_sequence<sizeof...(Params)>{});

    p.func           = reinterpret_cast<void*>(f);
    p.gridDim        = g;
    p.blockDim       = b;
    p.sharedMemBytes = s;
    p.kernelParams   = arguments;
    p.extra          = nullptr;

    TF_CHECK_CUDA(
        cudaGraphAddKernelNode(&node, this->get(), nullptr, 0, &p),
        "failed to create a kernel task"
    );
    return cudaTask(this->get(), node);
}

The cast values are stored in a std::tuple — not addressed directly from the function
parameter pack — to guarantee that the void* pointers remain valid for the duration of
the cudaGraphAddKernelNode call. This avoids a subtle dangling-address issue that the
original naive pattern (void* arguments[] = { (void*)(&args)... }) is susceptible to when
a conversion creates a temporary.

New overload in cudaGraphExecBase (cuda_graph_exec.hpp)

Identical pattern, calling cudaGraphExecKernelNodeSetParams instead.


Overload resolution — no ambiguity, no breaking changes

The two overloads coexist cleanly:

// Overload A — generic (existing, unchanged)
template <typename F, typename... ArgsT>
cudaTask kernel(dim3, dim3, size_t, F f, ArgsT... args);

// Overload B — typed pointer (new)
template <typename... Params, typename... ArgsT>
cudaTask kernel(dim3, dim3, size_t, void(*f)(Params...), ArgsT&&... args);

When the caller passes a named __global__ function, its type is void(*)(Params...)
Overload B matches more specifically via partial ordering ([temp.func.order]) and is
preferred. When the caller passes a lambda or functor, deduction for Overload B fails and
only Overload A is viable. No existing call site needs to change.


What this catches (and what it doesn't)

Scenario Before After
int passed where size_t expected Silent UB — crashes on Windows ✗ Compile error
unsigned int passed where size_t expected Silent UB ✗ Compile error
Wrong argument count Silent UB ✗ Compile error (static_assert)
float* passed where const float* expected Works (no mismatch in this case) ✓ Compiles cleanly
boolean_T* (typedef) passed where __nv_bool* expected Works on matching platforms ✓ Compiles cleanly
Lambda / functor passed to kernel() ✓ Works ✓ Still works (generic overload)
float* passed where int* expected Works (pointer width matches) ✓ Compiles — not caught (outside scope)

Pointer-to-incompatible-type mismatches (float* for int*) are intentionally outside
the protection boundary; they require compute-sanitizer or code review to detect.


Tests

A new test file unittests/cuda/test_cuda_type_safe_kernel.cu is added with 11 test cases:

  • Property 1kernelArgCast dispatch: scalar path uses static_cast, pointer path uses reinterpret_cast (100 iterations each)
  • Property 2 — Argument delivery, creation path: int passed for size_t N, GPU output verified across 100+ size values
  • Property 2 — Argument delivery, update path: same, via cudaGraphExecBase::kernel()
  • Property 3 — Overload equivalence: typed overload produces identical GPU output to a second identical typed call (100 iterations)
  • Property 4 — Zero-cost: graph-creation time of typed path vs. identical typed path ≤ 1% deviation over 5 × 1000 iterations
  • Functional unit tests: const-qualification, argument count match, backward-compat with named function pointers, exec-path variants

Two negative-compile test files (test_compile_fail_arg_count.cu,
test_compile_fail_scalar_mismatch.cu) are wired into CMakeLists.txt via try_compile to
verify the static_assert and ill-formed static_cast paths actually reject bad code.


Files changed

File Change
taskflow/cuda/cuda_graph.hpp Add tf::detail::kernelArgCast; add typed kernel() overload to cudaGraphBase
taskflow/cuda/cuda_graph_exec.hpp Add typed kernel() overload to cudaGraphExecBase
unittests/cuda/test_cuda_type_safe_kernel.cu New — 11 test cases (property + unit)
unittests/cuda/test_compile_fail_arg_count.cu New — negative compile test
unittests/cuda/test_compile_fail_scalar_mismatch.cu New — negative compile test
unittests/cuda/CMakeLists.txt Register new test + try_compile negative tests
doxygen/releases/release-4.2.0.dox Add GPU Tasking subsection under New Features

@saschiwy saschiwy force-pushed the feature/typesafe-kernel-deduction branch from 1d72ba5 to 47bedd8 Compare July 6, 2026 11:35
Includes tests for compile-time argument checks, scalar widening, pointer qualification, and backward compatibility. Negative compile tests validate incompatible arguments fail correctly. Updated CMakeLists to include new tests.
@saschiwy saschiwy force-pushed the feature/typesafe-kernel-deduction branch from 47bedd8 to 0adfa62 Compare July 6, 2026 11:37
@saschiwy

saschiwy commented Jul 6, 2026

Copy link
Copy Markdown
Author

Sorry, the commits went out with my work email — I'm rewriting the history to use my personal address and will force-push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant