PyTorch

A PyTorch tensor carries values together with shape, dtype, device, and optionally autograd history. nn.Module registers parameters and submodules so model.parameters(), device transfers, optimisation, and checkpointing can find them; forward describes how an input tensor becomes an output tensor. Autograd records differentiable operations executed on tensors that require gradients, while Dataset and DataLoader define how individual samples are decoded, shuffled, collated, and batched. The standard training loop is deliberately explicit, which makes state and failure boundaries inspectable.

Most production training mistakes are state or device mistakes: an unregistered layer is not optimised, a tensor on CPU cannot combine with GPU weights, and a saved state_dict is not the same thing as arbitrary Python state. Treat shape, dtype, and device as part of every function's contract; printing only tensor values leaves out the information that usually identifies the error.

A tensor is an ndarray with two extra properties. device says which piece of hardware holds the memory — cpu, cuda:0, mps — and operations require all their operands on the same one. requires_grad says whether autograd should record the operations that produced this tensor, building the graph that backward() will later traverse. Shape, dtype, broadcasting, axis semantics, views and strides all behave exactly as they do in NumPy.

Recognising how much transfers saves relearning a library. x.sum(axis=0) is x.sum(dim=0), (5,) versus (5, 1) bites identically, a slice is still a view that shares memory, and x.numpy() and torch.from_numpy(a) share the same buffer with no copy when both are on the CPU. What is genuinely new is that a tensor now has *history*: it may be a node in a graph that is keeping earlier tensors alive, which is why accumulating loss tensors in a Python list is the classic PyTorch memory leak and float(loss) or loss.item() is the fix.

Subclassing nn.Module requires two things: call super().__init__() before anything else, and assign every layer and parameter as an attribute so the base class can register them. Registration is what makes model.parameters(), model.to(device), model.state_dict(), model.train() and model.eval() work — each walks the registered tree. forward defines the computation, but you never call it directly: calling model(x) invokes __call__, which runs registered forward pre-hooks, then forward, then forward hooks.

Two real consequences follow. First, anything not assigned as an attribute is invisible: a list of layers stored as a plain Python list is not registered, so those layers are never moved to the GPU, never optimised, and never saved — nn.ModuleList and nn.ModuleDict exist for exactly this. Second, model(x) and model.forward(x) are not interchangeable: the latter skips every hook, which silently disables anything built on them, including many profiling tools, feature extractors, quantisation observers and distributed-training wrappers.

Dataset has a two-method contract: __len__ returns the number of examples and __getitem__(i) returns example i, as tensors, one at a time. DataLoader wraps it and owns everything about *batches*: how many samples per batch, whether to shuffle each epoch, how many worker processes decode in parallel, whether to pin host memory for faster transfer, and how individual samples are collated into batched tensors.

The separation is the design, and it is a good one. Your Dataset knows how to read one thing from disk and knows nothing about training; the DataLoader knows about throughput and knows nothing about your data format. That means shuffling, batching and parallel decoding are configuration rather than code you write, and it means the input pipeline can be made faster — more workers, pinned memory, prefetching — without touching a line of your dataset or model. It also isolates a very common performance problem: if GPU utilisation is low, the question is almost always whether the DataLoader is keeping up, and that is a question about one object.

Tensors and parameters live on a specific device, and an operation combining two of them requires both on the same one. model.to(device) moves a module's registered parameters and buffers *in place*; tensor.to(device) returns a *copy* and leaves the original alone. That asymmetry is the source of the single most common runtime error in PyTorch: a model moved to the GPU and a batch that was not, producing Expected all tensors to be on the same device.

It deserves its own concept because the fix is structural rather than attentive. Moving tensors at the point of use scatters .to(device) calls through the codebase and you will miss one — typically on a mask, an index tensor, a positional buffer, or a loss weight created inside forward. The robust habits are: resolve device once at startup, move every tensor a batch contains at the top of the loop, and inside a module derive new tensors from an existing one with torch.zeros_like(x) or torch.arange(n, device=x.device) so they inherit the right device automatically. Registering constants with register_buffer makes them move with model.to() for free.

These do two unrelated things and inference needs both. model.eval() changes *layer behaviour*: dropout stops dropping, and batch normalisation switches from batch statistics to its accumulated running statistics. torch.no_grad() changes *autograd*: operations inside it are not recorded, so no graph is built and no activations are retained for a backward pass that will never happen. Neither implies the other — eval() still builds graphs, and no_grad() still leaves dropout active.

Module 4 established that the forward pass is a pure evaluation and module 11 established that dropout and normalisation behave differently by mode; this is where both become one API habit. Forgetting eval() makes evaluation stochastic and irreproducible, so metrics wobble between identical runs and comparisons become unreliable. Forgetting no_grad() is not a correctness bug but a memory one: every activation is retained, so inference uses several times the memory it needs and often runs out on exactly the large batch you chose for throughput. torch.inference_mode() is a slightly stricter and faster version of no_grad() for pure serving paths.

model.state_dict() returns an ordered dictionary mapping parameter and buffer names to tensors — pure data, with no reference to your classes. The standard checkpoint saves that dictionary, and loading it means constructing the model in code first and then calling load_state_dict. torch.save(model) instead pickles the whole object, which serialises a reference to your module class and therefore depends on your source layout, your package versions and your import paths at load time.

This is why state_dict is the recommended mechanism and not merely one option. A pickled model breaks when you rename the class, move the file, or upgrade PyTorch; a state_dict is just named tensors and keeps loading. A resumable checkpoint also needs more than the weights: the optimiser's state (AdamW's moments are as large as the model), the learning-rate scheduler's step count, the epoch, the AMP grad scaler and the RNG state. Saving weights alone produces a checkpoint that appears to resume and actually restarts the optimiser from scratch, which shows up as a loss spike at exactly the resume point.

Three mistakes account for a large share of real PyTorch debugging, and all three are mode or state errors rather than logic errors. Missing optimizer.zero_grad() lets gradients accumulate across steps, so each update is contaminated by every previous batch. Missing model.train() after a validation pass leaves dropout and normalisation in inference behaviour for the rest of training. Missing model.eval() before validation makes the metric stochastic. None of them raises.

They are named here, in the mechanics module, rather than left with the concepts that introduced them, because knowing *why* gradients accumulate does not stop you forgetting the call. The point of gathering them is the shared shape: each is a piece of state the loop must set at a particular moment, and each fails silently and plausibly. That makes them candidates for structure rather than discipline — put the mode switches inside the loop functions that need them, or use a single train_one_epoch / evaluate pair so there is exactly one place each call can be missing from.