Skip to content

[BUG] Setting AF_CUDA_MAX_JIT_LEN to values < 2 produces SIGSEGV #3065

Description

@cschreib-ibex

Setting the environment variable AF_CUDA_MAX_JIT_LEN to any value less than 2 makes any ArrayFire code produce SIGSEGV (Segmentation Fault).

Description

  • The problem also occurs with the OpenCL backend and AF_OPENCL_MAX_JIT_LEN.
  • The problem does not occur with the CPU backend and AF_CPU_MAX_JIT_LEN.
  • Using a debugger, I can see the error appears to be some form of infinite recursion:
Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffcfa6ae179 in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
(gdb) bt
#0  0x00007ffcfa6ae179 in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
#1  0x00007ffcfa6a9b2b in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
#2  0x00007ffcfa6a9c1c in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
#3  0x00007ffcf9516e1d in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
#4  0x00007ffcf9517009 in afcuda!??5af@@YA?AVarray@0@AEBV10@AEBE@Z () from C:\Program Files\ArrayFire\v3\lib\afcuda.dll
...
  • Did you build ArrayFire yourself or did you use the official installers: Official installers
  • Which backend is experiencing this issue: CUDA and OpenCL
  • Do you have a workaround: No (besides the obvious "don't do that")
  • Can the bug be reproduced reliably on your system: Yes
  • A clear and concise description of what you expected to happen: I expect it to either disable the JIT entirely, or if this is not possible, at least throw an exception to signal an invalid environment variable
  • Run your executable with AF_TRACE=all and AF_PRINT_ERRORS=1 environment
    variables set.
  • Screenshot or terminal output of the results:
[platform][1608204098][023044] [ ..\src\backend\common\DependencyModule.cpp(99) ] Attempting to load: forge.dll
[platform][1608204098][023044] [ ..\src\backend\common\DependencyModule.cpp(102) ] Found: forge.dll
[platform][1608204098][023044] [ ..\src\backend\cuda\device_manager.cpp(428) ] CUDA Driver supports up to CUDA 10.1 ArrayFire CUDA Runtime 10.1
[platform][1608204098][023044] [ ..\src\backend\cuda\device_manager.cpp(495) ] Found 1 CUDA devices
[platform][1608204098][023044] [ ..\src\backend\cuda\device_manager.cpp(521) ] Found device: GeForce GTX 1070 (8 GB | ~6256.71 GFLOPs | 15 SMs)
[platform][1608204098][023044] [ ..\src\backend\cuda\device_manager.cpp(556) ] AF_CUDA_DEFAULT_DEVICE:
[platform][1608204098][023044] [ ..\src\backend\cuda\device_manager.cpp(575) ] Default device: 0(GeForce GTX 1070)
[mem][1608204099][023044] [ ..\src\backend\cuda\memory.cpp(158) ] nativeAlloc:    1 KB 0xb04e00000
[jit][1608204099][023044] [ ..\src\backend\cuda\compile_module.cpp(430) ] {18389128331537752529           : loaded from C:\Users\CORENT~1.SCH\AppData\Local\Temp\\ArrayFire\KER18389128331537752529_CU_61_AF_37.cubin for GeForce GTX 1070 }

Reproducible Code and/or Steps

Compile the following with afcuda and run the following code with the environment variable AF_CUDA_MAX_JIT_LEN set to 1

#include <arrayfire.h>
#include <iostream>

int main() {
    try {
        af::array a = af::constant(1, 10);
        af::array b = af::constant(2, 10);
        af::array c = a * b;

        std::cout << af::sum<double>(c) << std::endl;
    } catch (const af::exception& e) {
        std::cerr << e.what() << std::endl;
    } catch (const std::exception& e) {
        std::cerr << e.what() << std::endl;
    } catch (...) {
        std::cerr << "Unknown error" << std::endl;
    }

    return 1;
}

System Information

Please provide the following information:

  1. ArrayFire version: 3.7.2
  2. Devices installed on the system: Nvidia GeForce GTX 1070 (8GB)
  3. (optional) Output from the af::info() function if applicable.
ArrayFire v3.7.2 (CUDA, 64-bit Windows, build 218dd2c)
Platform: CUDA Runtime 10.1, Driver: 10010
[0] GeForce GTX 1070, 8192 MB, CUDA Compute 6.1
  1. Output from the following scripts:
    Windows: nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 432.00       Driver Version: 432.00       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070   WDDM  | 00000000:65:00.0 Off |                  N/A |
|  0%   49C    P8    14W / 151W |    559MiB /  8192MiB |      2%      Default |
+-------------------------------+----------------------+----------------------+

Checklist

  • Using the latest available ArrayFire release
  • GPU drivers are up to date

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions