NumPy · Module 12 of 13 · 7 concepts · about 8 min

Memory, Views & Performance

Replacing Python-level loops with whole-array operations, and telling a constant-factor problem from an algorithmic one.

At work: Keeping numeric jobs fast and inside memory as data grows, so they finish on time and don't need bigger, pricier machines.

The vectorisation mindset

What it is

Express the computation as operations on whole arrays. Every Python-level iteration over elements — a for, a comprehension, map — pays interpreter overhead per element; a ufunc pays it once.

Why it matters

The gap is one to two orders of magnitude, and it is the single reason NumPy exists. An interview answer that loops over rows is usually the answer that gets asked to try again.

In the real world: Rewriting a nightly scoring loop as array operations took it from 2 hours to 2 minutes, so results are ready before the business day starts.

# loop: one Python call per element
out = np.array([x * 2 + 1 for x in arr])
# vectorised: one compiled pass
out = arr * 2 + 1

# the same for conditionals
out = np.where(arr > 0, arr, 0.0)

Believing the loop is fine because the array is small. It usually is — the problem is that the small array becomes a batch, and the loop is by then three layers down inside a training step.

Practise it: A Confusion Matrix in One Pass, Class Weights From Label Counts, Cosine Similarity Without Loops, One-Hot Encode a Label Vector, Running Share of a Total, Group Sums Without a GroupBy, Intersection Over Union for Every Box Pair, Moving Average Without a Loop, Softmax That Does Not Overflow, Top-K Retrieval Without Sorting Everything, Fill the Gaps in Every Row

np.vectorize is not vectorisation

What it is

np.vectorize(f) wraps a scalar Python function so it accepts arrays. It still calls f once per element; it only provides broadcasting and shape handling.

Why it matters

The name is actively misleading, and reaching for it is how a 'vectorised' rewrite ends up no faster than the loop it replaced.

In the real world: Wrapping a slow per-row function in np.vectorize looks faster but isn't, so the batch job still misses its deadline.

slow = np.vectorize(lambda v: v * 2 + 1)   # a Python call per element
fast = lambda v: v * 2 + 1                  # already works on arrays
# When the logic is genuinely branchy, compose masks instead:
out = np.where(x < 0, 0.0, np.where(x > 1, 1.0, x))

Using it on a hot path and reporting the rewrite as an optimisation. Time it: np.vectorize is typically within a factor of two of the original loop.

Constant factor versus asymptotic cost

What it is

Two separate problems. Vectorising a loop removes a constant factor of 10-100x; replacing an O(n * w) window scan with a prefix sum, or a full sort with a partition, changes the exponent.

Why it matters

Vectorising a quadratic algorithm gives you a fast quadratic algorithm, which still fails at the next order of magnitude. The diagnosis decides which fix is worth writing.

In the real world: Vectorising a 30-day moving average speeds it up, but computing it from running totals is what makes it scale to years of data.

# constant factor: same algorithm, no Python loop
np.abs(x).sum()
# asymptotic: O(n * w) -> O(n)
c = np.concatenate(([0.0], np.cumsum(x)))
windows = c[w:] - c[:-w]
# asymptotic: O(n log n) -> O(n)
top = np.argpartition(scores, -k)[-k:]

Optimising the constant factor of an operation whose *result* is the problem. A 3-GB broadcast intermediate is not made acceptable by computing it quickly; it needs the algebraic identity that never builds it.

Practise it: Moving Average Without a Loop

Memory: dtype, temporaries and out=

What it is

An array's footprint is size * itemsize, and every intermediate in a chained expression is a full-size allocation. float32 halves the bill, in-place operators and out= remove the temporaries.

Why it matters

Most 'NumPy is slow' complaints at scale are memory bandwidth: the arithmetic is trivial and the machine spends its time moving bytes. Fewer and narrower arrays is the whole optimisation.

In the real world: Choosing 32-bit numbers and reusing arrays lets a 50 GB feature job fit on one machine instead of a costly cluster.

X.nbytes / 1e9                     # check before allocating another one
Z = (X - mu) / sd                  # two temporaries the size of X
Z = X - mu; Z /= sd                # one, then in place
np.subtract(X, mu, out=X)          # none, if destroying X is acceptable

Casting to float32 for memory and then multiplying by a NumPy float64 scalar, which promotes the whole array back. Keep the scalars np.float32, or use out= to pin the output dtype.

Practise it: One-Hot Encode a Label Vector, Pairwise Euclidean Distances, Moving Average Without a Loop

Contiguity and the axis you iterate

What it is

C-order arrays are laid out row by row, so a reduction along the last axis reads adjacent memory while one along the first strides through it. x.flags['C_CONTIGUOUS'] tells you the layout; np.ascontiguousarray fixes it.

Why it matters

The same reduction can differ severalfold in wall time depending on which axis it walks, and a transposed array passed into BLAS may be copied behind your back.

In the real world: Summing an image batch along the right axis runs several times faster, which adds up across millions of images a day.

x = np.zeros((10_000, 10_000))
x.sum(axis=1)     # walks along rows: cache-friendly
x.sum(axis=0)     # strides between rows: slower on the same data
x.T.flags['C_CONTIGUOUS']          # False — a transposed view
np.ascontiguousarray(x.T).shape    # an explicit copy, if it is worth it

Storing features as (d, n) because the maths is written that way, then reducing per sample. Lay the array out in the direction you will read it — usually one row per sample.

Measuring, not guessing

What it is

Time the candidates on realistic sizes: timeit for microbenchmarks, time.perf_counter around a real call, and arr.nbytes for the memory question. Then check the answers still agree with np.allclose.

Why it matters

Vectorised rewrites change numerical results — a different summation order gives a different last digit — and an optimisation that is not verified against the simple version is a rewrite of unknown correctness.

In the real world: Timing two versions of a pricing calculation on real data before choosing one, rather than trusting a hunch.

import time

t = time.perf_counter()
fast = sliding_window_mean(x, w)
elapsed = time.perf_counter() - t
slow = np.array([x[i : i + w].mean() for i in range(len(x) - w + 1)])
assert np.allclose(fast, slow, atol=1e-8)

Benchmarking on a toy array. Below a few thousand elements the per-call overhead dominates and the vectorised version can look slower — the crossover is exactly what you are trying to measure.

When to leave NumPy

What it is

Arrays too large for memory want np.memmap or chunked processing; heavily branchy element-wise logic wants Numba or Cython; whole-pipeline work on tabular data wants Pandas or Polars, and GPU work wants PyTorch, whose array API deliberately mirrors this one.

Why it matters

NumPy's model is dense, in-memory, single-machine and element-wise-uniform. Knowing the boundary stops you from writing a distributed system out of np.add.at calls.

In the real world: Moving a very branchy fraud rule to Numba, or a huge dataset to chunked processing, when NumPy alone isn't enough.

big = np.memmap('features.f32', dtype=np.float32, mode='r', shape=(10_000_000, 64))
totals = np.zeros(64, dtype=np.float64)
for start in range(0, len(big), 100_000):        # chunked, not element-wise
    totals += big[start : start + 100_000].sum(axis=0)

Treating a chunked loop as a failure of vectorisation. A loop over a hundred chunks of a hundred thousand rows each is vectorised — the work inside it is whole-array. It is a loop over elements that is the problem, not a loop as such.

Practise it: Class Weights From Label Counts, Flatten a Batch of Images, One-Hot Encode a Label Vector, Pairwise Euclidean Distances, Pick the Rows That Pass a Threshold, Reproducible Train and Test Indices, Running Share of a Total, Scale Every Column to 0 and 1