NumPy · Module 5 of 13 · 6 concepts · about 8 min

Broadcasting

The rule that lets arrays of different shapes combine without a loop, and the wrong answers it produces when the shapes line up differently than you thought.

At work: Applying one set of numbers across a whole table in one line: a discount per product, a scale per feature, a distance to every store.

The broadcasting rules

What it is

Shapes are compared from the right. Two axes are compatible if they are equal, or if one of them is 1, in which case that axis is stretched. Missing leading axes count as 1.

Why it matters

It is what makes 'subtract the column mean from every row' one expression rather than a loop, and it is the mechanism under every pairwise similarity matrix you will write.

In the real world: Applying a different discount to each of 10 products across 365 days of prices in one line, without writing a loop.

Visual example

Broadcasting

A (3, 1) column and a (1, 4) row broadcast into a (3, 4) grid: each length-1 axis is stretched to the other's length.

col — shape (3, 1)

	0
0	10
1	20
2	30

col + row

row — shape (1, 4)

	0	1	2	3
0	1	2	3	4

col + row — shape (3, 4)

	0	1	2	3
0	11	12	13	14
1	21	22	23	24
2	31	32	33	34

Four of the twelve cells

[0, 0] = 10 + 1 = 11

[0, 3] = 10 + 4 = 14

[2, 0] = 30 + 1 = 31

[2, 3] = 30 + 4 = 34

Why this matters

Every pairwise matrix — distances, IoU, similarity between two sets of vectors — is this picture. A column against a row gives you every combination with no loop and no index arithmetic.

Common misconception

Wrong idea: shapes must match, or you have to tile the smaller one with np.repeat.

Correct idea: a length-1 axis is stretched for free. No copy is made; the same value is read repeatedly.

1Align from the right(3, 1) against (1, 4): compare 1 with 4, then 3 with 1.
2Stretch the 1sEach axis of length 1 is read repeatedly to match the other.
3Compute element-wiseThe result's shape is the per-axis maximum: (3, 4).

col = np.array([[10], [20], [30]])      # (3, 1)
row = np.array([[1, 2, 3, 4]])          # (1, 4)
(col + row).shape                       # (3, 4)

X = np.zeros((100, 8))
X - X.mean(axis=0)                       # (100, 8) - (8,) -> fine
np.broadcast_shapes((3, 1), (1, 4))      # (3, 4), without allocating

Two shapes that broadcast when you meant them to match. Subtracting a (100,) row of per-sample values from a (100, 8) matrix raises, but a (8,) array of per-feature values subtracted when you meant per-sample succeeds and is wrong.

Practise it: Clip Outliers to a Percentile Range, Cosine Similarity Without Loops, Normalise Rows to Unit Length, One-Hot Encode a Label Vector, Pairwise Euclidean Distances, Scale Every Column to 0 and 1, Intersection Over Union for Every Box Pair, Softmax That Does Not Overflow

Adding an axis with None or np.newaxis

What it is

Indexing with None inserts a length-1 axis at that position: x[:, None] turns (n,) into (n, 1), and x[None, :] turns it into (1, n).

Why it matters

It is how you give a 1-D array an orientation, which is the whole trick behind turning two vectors into a pairwise matrix.

In the real world: Computing the distance from 1,000 customers to 50 warehouses in one step, to pick the nearest warehouse for each.

x = np.array([1, 2, 3])
x[:, None].shape         # (3, 1) — a column
x[None, :].shape         # (1, 3) — a row
x[:, None] - x[None, :]  # (3, 3) all pairwise differences

Forgetting the orientation and writing x - x, which is a (3,) array of zeros rather than the (3, 3) pairwise matrix. Both run; only one is what you wanted.

Practise it: Pairwise Euclidean Distances

keepdims

What it is

keepdims=True leaves the reduced axis in place with length 1, so the result broadcasts back against the original array.

Why it matters

Without it, a reduction on the last axis returns a shape that re-broadcasts along the *wrong* axis. This is the mechanical reason softmax and row-normalisation go quietly wrong.

In the real world: Dividing each store's daily sales by that store's own total to get the share of sales per day.

Visual example

keepdims

keepdims=True keeps the collapsed axis as length 1, so the result can be broadcast back against the input.

x — shape (2, 3)

	0	1	2
0	1	2	3
1	4	5	6

sum(axis=1)

keepdims=True — shape (2, 1)

	0
0	6
1	15

The two spellings

sum(axis=1) → [6, 15], shape (2,)

keepdims=True → [[6], [15]], shape (2, 1)

x / (2, 1) → rows sum to 1

x / (2,) → aligns to columns

Why this matters

Row-wise normalisation — softmax, unit vectors, per-row shares — is a division by a reduction of the same array. Only the (2, 1) shape divides row by row.

Common misconception

Wrong idea: keepdims is cosmetic, it just keeps the printout tidy.

Correct idea: it decides which axis the result broadcasts along. Drop it and the division can silently apply across the wrong axis.

1ReduceSum along axis 1: one total per row.
2Keep the axisThe axis stays, with length 1, giving (2, 1).
3Broadcast back(2, 3) / (2, 1) stretches the column across the row.

x = np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
x.sum(axis=1).shape                   # (2,)
x.sum(axis=1, keepdims=True).shape    # (2, 1)
x / x.sum(axis=1, keepdims=True)      # rows sum to 1 — correct
x / x.sum(axis=1)                      # raises here; on a square x it would not

On a square array the version without keepdims does not raise: (n, n) / (n,) broadcasts across the columns instead of down the rows and returns a plausible, wrong matrix. Always pass keepdims=True when the reduction is going back into an expression.

Practise it: Normalise Rows to Unit Length

Row vector versus column vector

What it is

A 1-D array has no orientation. In any binary operation it is treated as the *last* axis — that is, as a row.

Why it matters

It explains why subtracting a per-sample value from a feature matrix needs [:, None] while subtracting a per-feature value does not.

In the real world: Subtracting each feature's average (a row) versus each customer's average (a column) gives very different results; mixing them up quietly breaks a model.

X = np.zeros((4, 3))
per_feature = np.array([1.0, 2.0, 3.0])     # (3,)
per_sample = np.array([1.0, 2.0, 3.0, 4.0])  # (4,)
X - per_feature          # fine: (4, 3) - (3,)
X - per_sample[:, None]  # needed: make it a column first

Reaching for .T to fix a 1-D array. Transposing a (n,) array is a no-op — it returns the same shape — so the fix has to be reshape(-1, 1) or [:, None].

The memory cost of broadcasting

What it is

Broadcasting itself allocates nothing, but the *result* is fully materialised. a[:, None] - b[None, :] on two arrays of 50,000 elements is a 2.5-billion-element array.

Why it matters

It is the most common way to turn a fast program into one that is killed by the OOM killer, and the fix is usually an algebraic identity or a chunked loop over blocks.

In the real world: Comparing 1 million users with 1 million products in one step would need terabytes of memory, so recommendation systems work in batches.

# 20_000 x 20_000 x 8 bytes = 3.2 GB of intermediate
# D = ((A[:, None, :] - B[None, :, :]) ** 2).sum(-1)
# The identity avoids the 3-D intermediate entirely:
D = (A ** 2).sum(1)[:, None] + (B ** 2).sum(1)[None, :] - 2 * A @ B.T

Judging a broadcast by the size of its inputs. Two small arrays can produce an enormous outer product; the shape of the *result* is the number to check.

broadcast_to and the outer form of a ufunc

What it is

np.broadcast_to(x, shape) gives a read-only view at the broadcast shape without copying. Every ufunc also has an .outer method: np.add.outer(a, b) is a[:, None] + b[None, :].

Why it matters

broadcast_to makes an intended broadcast explicit and free, and .outer states 'every pair' in one call rather than two index tricks.

In the real world: Building a price-by-quantity table of all order totals for a pricing page without copying data.

np.broadcast_to(np.arange(3), (4, 3)).shape   # (4, 3), no copy, read-only
np.add.outer([1, 2], [10, 20])                # [[11, 21], [12, 22]]
np.maximum.outer([1, 5], [3, 4])              # every pairwise maximum

Writing into the result of broadcast_to. It is read-only for good reason — several positions share one memory cell, so a single write would appear in many places.