Bezen

NumPy · Module 3 of 13 · 8 concepts · about 11 min

Indexing & Slicing

Every way to name part of an array — one index per axis, a slice that aliases its source, and an array of positions that gathers a copy.

At work: Pulling out exactly the rows, time ranges or pixels a job needs, without accidentally changing the original data.

Multi-axis indexing

What it is

One index per axis inside a single bracket: arr[i, j]. A slice in a position keeps that axis, an integer drops it.

Why it matters

arr[i][j] also works, but it builds an intermediate view for the row and then indexes that — slower, and it cannot be assigned to reliably. One bracket is the idiom.

In the real world: Reading one store's sales on one day from a stores-by-days grid, or a single pixel from a product image.

x = np.arange(12).reshape(3, 4)
x[1, 2]        # 6 — a scalar
x[1]           # row 1, shape (4,)
x[:, 2]        # column 2, shape (3,) — the axis is dropped
x[:, 2:3]      # column 2, shape (3, 1) — the axis is kept

Expecting x[:, 2] to be a column. It is 1-D, so it broadcasts as a row in the next operation. Use x[:, 2:3] or x[:, 2, None] when you need the (n, 1) shape.

Slices are views

What it is

A basic slice returns a view over the same memory: no copy, and writes through it change the original array.

Why it matters

It makes slicing free, which is what lets a rolling computation slice the same array twice without paying for two copies — and it is why an in-place function that receives a slice edits its caller's data.

In the real world: Normalising “just the last month” of a price series in place can silently change the original history used by every other report.

Visual example

Views

A slice is a window onto the same memory — writing through the view writes into the original.

a — the base array
value
00
11
22
33
view[1] = 99
a, after writing to a[0:3]
value
00
199
22
33

What happened

view = a[0:3]

view[1] = 99

a[1] → 99

view.base is a → True

Why this matters

Pipelines pass slices around constantly. A function that normalises 'its' array in place is editing whatever buffer that slice belongs to.

Common misconception

Wrong idea: slicing gives you a fresh little array to play with.

Correct idea: it gives you a view: same buffer, different shape and strides. Take .copy() when you intend to write.

  1. 1Slicea[0:3] records an offset, a shape and strides — no data moves.
  2. 2Writeview[1] = 99 writes into the shared buffer.
  3. 3Both changea and the view are two descriptions of one block of memory.
a = np.arange(6)
window = a[1:4]        # a view
window[0] = 99
a                      # array([ 0, 99,  2,  3,  4,  5])
window.base is a       # True — it borrows a's buffer
safe = a[1:4].copy()   # now independent

Returning a slice from a function that the caller will mutate, or holding a small slice of a huge array to 'save memory'. The view keeps the whole base array alive; .copy() is what actually frees it.

Practise it: Flatten a Batch of Images

copy, and how to tell

What it is

arr.copy() allocates an independent array. arr.base is None for an owner and points at the source for a view; np.shares_memory(a, b) answers the question directly.

Why it matters

Every aliasing bug in NumPy is answered by one of these three checks, and they are the fastest thing to reach for when a value changed and nothing in the code assigned to it.

In the real world: Taking a real copy of a customer feature table before experimenting, so a test doesn't corrupt the data the production model uses.

b = a[::2]
b.base is a                    # True: a view
np.shares_memory(a, b)         # True
c = a[::2].copy()
c.base, np.shares_memory(a, c) # (None, False)

Believing advanced indexing aliases like a slice. a[[0, 1]] is always a copy, so writing to the *result* does nothing to a — while writing to a[[0, 1]] directly does. Same brackets, opposite behaviour.

Negative indices, steps and reversal

What it is

Negative indices count from the end, and a negative step walks backwards: x[::-1] reverses, x[:, ::-1] reverses each row. All still views.

Why it matters

Reversal is how you get a descending argsort and a backward fill, and it costs nothing because it only flips a stride.

In the real world: Getting the last 7 days of sensor readings, or flipping product photos left-to-right to create extra training images.

x = np.arange(5)
x[-1]         # 4
x[::-1]       # array([4, 3, 2, 1, 0]) — a view with a negative stride
np.argsort(s)[::-1]        # indices, best first
np.maximum.accumulate(x[::-1])[::-1]   # a running max from the right

Passing a negatively strided view into code that requires C-contiguous memory — some compiled libraries silently copy, others raise. np.ascontiguousarray makes the copy explicit when you need one.

Fancy (integer) indexing

What it is

Indexing with an array of positions returns a *copy* in that exact order, with the shape of the index array. Repeats are allowed, so the result can be longer than the input.

Why it matters

It is the gather operation: reordering by a sort, taking a random batch, mapping ids into an embedding table, expanding a lookup over rows.

In the real world: Pulling the rows for a list of 500 selected customers, in exactly that order, to send to a marketing campaign.

Visual example

Fancy indexing

Fancy indexing gathers by position: the result takes the index array's order and length, repeats included.

x
value
010
120
230
340
x[[2, 0, 2]]
x[[2, 0, 2]]
value
030
110
230

Index by index

c: idx 2 → 30

a: idx 0 → 10

c: idx 2 → 30, again

Why this matters

Gathers are everywhere: embedding lookups, minibatch sampling, reordering by argsort, and every forward fill built from an index array.

Common misconception

Wrong idea: indexing with a list is a filter, so the result is a subset.

Correct idea: it is a gather. Position 2 appearing twice appears twice in the result, and the result is a copy, not a view.

  1. 1Read the indices[2, 0, 2] names positions, in the order wanted.
  2. 2GatherEach position is read out of x into the result.
  3. 3Copy, not viewWriting to the result does not touch x.
x = np.array([10, 20, 30, 40])
x[[2, 0, 2]]                 # array([30, 10, 30]) — order and repeats kept
emb = np.zeros((1000, 16))
emb[[7, 7, 3]].shape         # (3, 16) — a row gather
X[rng.permutation(len(X))[:32]]   # a random minibatch

Mixing a slice and two index arrays and expecting the axes to stay put. With more than one advanced index the broadcast result is placed at one position in the shape, which is rarely where you expected; index one axis at a time, or use np.ix_.

Practise it: A Confusion Matrix in One Pass, One-Hot Encode a Label Vector, Fill the Gaps in Every Row

Fancy assignment, and np.add.at

What it is

x[idx] = values scatters values into those positions, but a repeated index keeps only the *last* write. np.add.at(x, idx, values) accumulates instead.

Why it matters

Scatter-add is how per-group totals are computed inside a training loop, where there is no DataFrame and no groupby — and the plain += spelling of it is silently wrong.

In the real world: Adding each order's amount to its store's total when many orders belong to the same store, without losing any of them.

x = np.zeros(3)
idx = np.array([0, 0, 2])
x[idx] += 1        # array([1., 0., 1.]) — both writes to 0 collapse

y = np.zeros(3)
np.add.at(y, idx, 1)   # array([2., 0., 1.]) — accumulated correctly

x[idx] += 1 with duplicate indices. It is read-modify-write on a *copy*, so duplicates count once. Use np.add.at for correctness, or np.bincount when the operation is a sum over integer groups — it is far faster.

One-hot encoding by indexing

What it is

Allocate a zero matrix of shape (n, n_classes) and set one element per row: out[np.arange(n), labels] = 1. Or index an identity matrix: np.eye(n_classes)[labels].

Why it matters

It is the canonical two-array fancy index — one index array per axis, paired element-wise — and a loop-free encoding of labels for any model that needs them dense.

In the real world: Turning product categories into 0/1 columns so a demand-forecasting model can use them.

Visual example

One-hot

Two index arrays, paired element-wise: row i gets its 1 in column labels[i].

labels
label
02
10
21
out[arange(3), labels] = 1
one-hot — shape (3, 3)
c0c1c2
0001
1100
2010

The index pairs

2: (0, 2) → 1

0: (1, 0) → 1

1: (2, 1) → 1

Why this matters

It is the clearest example of paired advanced indexing: two index arrays of the same length select cells, not a rectangle of rows by columns.

Common misconception

Wrong idea: out[rows, cols] selects every combination of those rows and columns.

Correct idea: the two arrays are zipped: (rows[0], cols[0]), (rows[1], cols[1])… For the rectangle, use np.ix_(rows, cols).

  1. 1Allocatenp.zeros((n, n_classes)) — the shape comes from the class count, not the data.
  2. 2Pair the indicesnp.arange(n) down the rows, labels across the columns.
  3. 3Write once per rowExactly n cells become 1; everything else stays 0.
labels = np.array([2, 0, 1])
n_classes = 3
out = np.zeros((labels.size, n_classes))
out[np.arange(labels.size), labels] = 1.0
np.eye(n_classes, dtype=float)[labels]    # the same matrix, one expression

Deriving n_classes from labels.max() + 1. A batch that happens to contain no examples of the last class silently produces a narrower matrix, and the shape mismatch surfaces much later. Pass the class count in.

Practise it: One-Hot Encode a Label Vector

take, take_along_axis and np.ix_

What it is

np.take(x, idx, axis=) gathers along one axis. np.take_along_axis(x, idx, axis=) gathers with an index array shaped like x, which is what an argsort or argpartition per row gives you. np.ix_(rows, cols) builds the open mesh for the rectangle selection.

Why it matters

Per-row top-k, per-row interpolation and per-row fills all produce a full index array, and take_along_axis is the only clean way to apply one.

In the real world: Taking each user's top 5 recommended products, using a separate list of product positions per user.

scores = np.array([[0.1, 0.9, 0.5], [0.7, 0.2, 0.4]])
order = np.argsort(scores, axis=1)[:, ::-1]        # best first, per row
np.take_along_axis(scores, order, axis=1)          # [[0.9, 0.5, 0.1], [0.7, 0.4, 0.2]]
X[np.ix_([0, 2], [1, 3])].shape                    # (2, 2) — the rectangle

Using scores[order] when order came from a per-row argsort. That indexes axis 0 with a 2-D array and returns a 3-D result. Row-wise index arrays need take_along_axis.