Skip to content

PERF: avoid temporary boolean array in np.array_equal for contiguous integer arrays #32465

Description

@lesshaste

Proposed new feature or change

np.array_equal() currently implements its normal comparison path as:

builtins.bool((asanyarray(a1 == a2)).all())

This materializes an elementwise boolean array and then reduces it in a second pass. For same-shape, same-dtype, contiguous arrays of NumPy's built-in signed/unsigned integer dtypes, equality can instead be fused, potentially with a bytewise comparison. This would avoid the temporary allocation and allow an early exit on mismatches.

A conservative fast path would require both converted operands to be exact ndarrays, equal shapes and identical built-in signed/unsigned integer dtypes, and the same contiguous traversal order (both C-contiguous or both F-contiguous). Unsupported cases would retain the current implementation.

A raw-byte path should not initially include bool (distinct nonzero bytes can both mean true), floating-point (for example, 0.0 == -0.0), object, structured, flexible, or user-defined dtypes. For built-in integers, equal_nan does not affect the result.

Conceptually:

if (same_builtin_integer_dtype &&
        same_shape &&
        compatible_contiguous_layout) {
    return memcmp(data1, data2, nbytes) == 0;
}

The implementation need not literally use memcmp; a NumPy SIMD-aware fused compare/early-exit helper may be preferable. An identity check can also make np.array_equal(a, a) constant-time for these dtypes.

Prototype benchmark

I tested a narrow C prototype on NumPy main commit d2cf21758a32029b4c614fc90c2fc5dcbcf07a9c (2026-08-31). The helper uses memcmp only for identical built-in integer dtypes and compatible C/F-contiguous layouts, and otherwise returns NotImplemented to the existing path.

Following the benchmark style used in recent NumPy performance PRs, I added parameterized ASV time_ benchmarks with setup outside the timed method and ran:

cd benchmarks
asv continuous d2cf21758a32029b4c614fc90c2fc5dcbcf07a9c HEAD \
    -b "bench_array_equal" --interleave-rounds --show-stderr

Machine: AMD Ryzen 5 3400G, Linux x86-64, Python 3.12.

case current main prototype ratio
10M int64, equal, C 7.49 ms 5.67 ms 0.76
10M int64, mismatch in middle, C 7.30 ms 2.90 ms 0.40
10M int64, mismatch first, C 6.93 ms 11.9 µs 0.0017
10M int64, mismatch last, C 7.56 ms 5.61 ms 0.74
10M int8, equal, C 2.21 ms 785 µs 0.36
10M int8, mismatch first, C 1.51 ms 9.74 µs 0.0065
10M int8, mismatch in middle, C 1.92 ms 399 µs 0.21
10M int8, mismatch last, C 2.23 ms 790 µs 0.35

F-contiguous results were similar. At one million elements, all eight built-in signed/unsigned integer dtypes improved. For first-element mismatches:

dtype current main prototype
int8 63.0 µs 4.75 µs
uint8 64.8 µs 4.71 µs
int16 274 µs 7.65 µs
uint16 267 µs 7.44 µs
int32 546 µs 8.56 µs
uint32 567 µs 9.64 µs
int64 708 µs 9.78 µs
uint64 713 µs 10.5 µs

Short contiguous integer arrays (10 and 1,000 elements) did not regress in this run. Explicit fallback controls also showed no measured regression:

unsupported path current main prototype
1M float64, equal 871 µs 824 µs
1M bool, equal 107 µs 90.2 µs
1M strided int64, equal 1.48 ms 1.45 ms

Those small fallback differences should be treated as noise; the important result is that the added dispatch was not measurably slower.

The core ASV benchmark shape was:

class ArrayEqualContiguousInteger:
    params = [
        [10, 1_000, 10_000_000],
        ["int8", "int64"],
        ["equal", "early", "middle", "late"],
        ["C", "F"],
    ]
    param_names = ["size", "dtype", "difference", "order"]

    def setup(self, size, dtype, difference, order):
        shape = (size // 10, 10)
        self.a = np.arange(size, dtype=dtype).reshape(shape, order=order)
        self.b = self.a.copy(order=order)
        if difference != "equal":
            index = {
                "early": 0,
                "middle": size // 2,
                "late": size - 1,
            }[difference]
            self.b.ravel(order=order)[index] ^= 1

    def time_array_equal(self, size, dtype, difference, order):
        np.array_equal(self.a, self.b)

Temporary memory

I also measured process peak RSS in fresh subprocesses after constructing two 10M-element int64 arrays. The current (a == b).all() path increased peak RSS by 9,816 KiB; the prototype increased it by 0 KiB. The result was the same for equal arrays and a first-element mismatch. This is consistent with eliminating the 10M-byte boolean temporary.

Correctness checks

The prototype passed focused checks for all eight integer dtypes, C- and F-contiguous multidimensional arrays, equal and unequal values, identity, empty arrays, and equal_nan=True. Tests also verified that bool (including distinct nonzero true representations), float, mixed byte order, mixed dtype, and strided arrays use the existing fallback.

Scope

This issue proposes only the built-in contiguous integer case. If a fused equality primitive proves maintainable, dtype-specific loops could later cover strided integers, floating-point semantics, or other fixed-width dtypes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions