Skip to content

[BUG] Thread local caches not reset when exiting evalNodes through an exception #3354

Description

@cschreib-ibex

The evalNodes function uses several thread_local variables as caches, to prevent recurring memory allocations. These variables are reset when the function exits normally, but not if an exception is thrown.

Description

See #3353 for the full description of the problem that got me there. After scratching my head at the problem for a while, I decided to recompile ArrayFire in debug so I could peer into the code and find out where the hard-crash was coming from. If you don't want to read all of #3353, the gist is I am using a build of ArrayFire with CUDA 11.8 and I try to run an application on a machine where the Nvidia drivers are too old to work with CUDA 11.8. This causes kernels to fail compilation/link inside ArrayFire.

In this case, it appears that evalNodes failed once, an exception was raised, and caught by our application. Our application then tried running another calculation, which caused evalNodes to be called again. In this second call, the cache variable full_nodes is not empty at the start of the function, and contains nullptr nodes from the last failed call. This causes node->clone() to crash, by de-referencing a null pointer.

System Information

See #3353

Checklist

  • Using the latest available ArrayFire release
  • GPU drivers are up to date

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions