Skip to content

AF bytesToString crash when trying to alloc a giant array #2826

Description

@WilliamTambellini

Bad crash in afcpu 3.7.1 when trying to alloc a giant array :

size_t nr = nc = 1000000000
af::array a(nr, nc);

Expected behavior: bad alloc exception, no crash.

1 __strlen_avx2 strlen-avx2.S 62 0x7fffe6a4f5a1
2 fmt::v5::internal::result_of<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>> (int)>::type fmt::v5::visit<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>(fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>&&, fmt::v5::basic_format_arg<fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>) 0x7fffdb94e64e
3 void fmt::v5::internal::parse_format_string<fmt::v5::internal::null_terminating_iterator, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>&>(fmt::v5::internal::null_terminating_iterator, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>&) 0x7fffdb9502f6
4 fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>::iterator fmt::v5::vformat_to<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>(fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer>>::range, fmt::v5::basic_string_view, fmt::v5::basic_format_args<fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer>, char>>) 0x7fffdb950569
5 fmt::v5::vformat(fmt::v5::basic_string_view, fmt::v5::format_args) 0x7fffdc6786f2
6 common::bytesToString(unsigned long) 0x7fffdc6712dc
7 cpu::Allocator::nativeAlloc(unsigned long) 0x7fffdb9429f7
8 common::DefaultMemoryManager::alloc(bool, unsigned int, long long *, unsigned int) 0x7fffdc66e5d5
9 std::unique_ptr<float [], std::function<void (float *)>> cpu::memAlloc(unsigned long const&) 0x7fffdb943bc6
10 cpu::Array::Array(af::dim4) 0x7fffdb5b6421
11 cpu::Array cpu::createEmptyArray(af::dim4 const&) 0x7fffdb5b651c
12 createHandle(af::dim4 const&, af_dtype) 0x7fffdc10b93c
13 af_create_handle 0x7fffdc10ecce
14 af_create_handle 0x7fffe792e08b
15 (anonymous namespace)::initEmptyArray(af_dtype, long long, long long, long long, long long) 0x7fffe7a38e5c
16 af::array::array(long long, long long, long long, af_dtype) 0x7fffe7a3a522

Seems to crash in nativeAlloc()
because of a log using bytesToString() :
void *Allocator::nativeAlloc(const size_t bytes) {
void *ptr = malloc(bytes);
AF_TRACE("nativeAlloc: {:>7} {}", bytesToString(bytes), ptr);
if (!ptr) AF_ERROR("Unable to allocate memory", AF_ERR_NO_MEM);
return ptr;
}

in Logger.cpp:
string bytesToString(size_t bytes) {
static array<const char *, 5> units{{"B", "KB", "MB", "GB", "TB"}};
size_t count = 0;
double fbytes = static_cast(bytes);
size_t num_units = units.size();
for (count = 0; count < num_units && fbytes > 1000.0f; count++) {
fbytes *= (1.0f / 1024.0f);
}
return fmt::format("{:.3g} {}", fbytes, units[count]);
}

Cannot upgrade to 3.7.1 even if disabling logs at runtime I guess.
@umar456 should be easy to fix, would you like me to ?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions