Pandas · Module 1 of 22 · 4 concepts · about 6 min

Pandas Mental Model: Series vs DataFrame

What the objects actually are. Almost every baffling Pandas bug is this model being misunderstood rather than a method being misused.

About Pandas: Pandas lets you work with tables of data in Python, like a spreadsheet you control with code. Most of the work in a machine learning project happens here: loading data, cleaning it up and shaping it before a model ever sees it. You will begin with how a table is built and how to pick out the rows and columns you want, then learn to clean messy data and answer questions with it, and finish with the common mistakes that quietly give wrong answers.

At work: Every analyst's first day with a company's data: a sales table, a customer list, a sensor log. Knowing what a table really is saves hours of confusion later.

Series

What it is

A one-dimensional array of values with an index of labels attached to it, and a single dtype for the whole thing. A column pulled out of a DataFrame is a Series.

Why it matters

It is the unit almost every operation returns. Knowing that df["col"] gives you a Series — and df[["col"]] gives you a one-column DataFrame — explains half the 'why does this print differently' confusion.

In the real world: An online shop's daily revenue for the last 90 days is a Series: one number per date. Plotting the trend or finding the best day starts from it.

Visual example

Pandas Series

A Series is values plus labels: one column of data with an index beside it.

Index labels

(the index)

Values

(the data)

	score
a	10
b	20
c	30

s["b"]

by label

s.iloc[1]

by position

Both return 20

Why this matters

Almost every operation hands you back a Series. Seeing it as labelled values explains most 'why does this print differently' moments.

Common misconception

Wrong idea: a Series is just a list of numbers.

Correct idea: it is values plus an index of labels, with one dtype for all of them.

1ValuesOne column of data, one dtype (int64 here).
2LabelsEach value has an index label: a, b, c.
3Two ways inBy label with s["b"], by position with s.iloc[1].

s = pd.Series([10, 20, 30], index=["a", "b", "c"], name="score")
s["b"]        # 20, by label
s.iloc[1]     # 20, by position
s.dtype       # dtype('int64') — one dtype for the whole Series

DataFrame

What it is

A set of Series that share one index. Columns are ordered and named, each column has its own dtype, and the index is shared across all of them.

Why it matters

Because columns are separate Series, a DataFrame is columnar, not row-oriented. Operating column-wise is the fast path; iterating rows fights the structure.

In the real world: A bank's customer table (one row per customer, columns for age, balance and account type) is a DataFrame. Nearly every report starts from one.

Visual example

DataFrame

A DataFrame is several Series sharing one index — one per column, each with its own dtype.

Shared index

(one label per row)

One column

(a Series, own dtype)

	user	vip	spend
0	a	True	9.99
1	b	False	12.5
2	c	True	0

df["spend"]

a Series

df[["spend"]]

a one-column DataFrame

Same data, different shapes

One dtype per column — df.dtypes

user → object

vip → bool

spend → float64

Why this matters

Because each column is its own array, column-wise work runs in compiled code while row-by-row loops fight the structure.

Common misconception

Wrong idea: a DataFrame is a list of rows, like a CSV.

Correct idea: it is a set of columns that share an index. Adding a column is cheap; adding rows in a loop is not.

df = pd.DataFrame({"user": ["a", "b"], "spend": [9.99, 12.50]})
type(df["spend"])      # Series
type(df[["spend"]])    # DataFrame — note the double brackets
df.dtypes              # per column, not per frame

Thinking of it as a list of rows, like a CSV or a list of dicts. That intuition is what leads to iterrows, and to surprise when adding a column is cheap but adding a row in a loop is quadratic.

The index, and alignment

What it is

The index is a label for each row, not a row number. It can repeat, it can be non-integer, and nearly every binary operation aligns on it before computing.

Why it matters

Alignment is the most surprising behaviour in Pandas. Adding two Series matches labels, not positions, so mismatched labels produce NaN rather than an error.

In the real world: Subtracting last month's sales per store from this month's lines up by store name, not by row order, so a store added mid-month doesn't shift every number.

Visual example

Pandas Series

Series arithmetic aligns by label, not by row position.

	value
x	1
y	2
z	3

match by index label

	value
z	10
y	20
x	30

a + b

	value
x	31
y	22
z	13

The calculations

x: 1 + 30 = 31

y: 2 + 20 = 22

z: 3 + 10 = 13

Why this matters

Pandas aligns Series by index label automatically. Even if the order changes, matching labels are added together.

Common misconception

Wrong idea: first row + first row.

Correct idea: x pairs with x, y with y, z with z. A label with no partner gives NaN.

1Read labelsEach Series has index labels (x, y, z).
2Align matching labelsFind the value with the same label, whatever its row position.
3Compute the resultAdd the values for each matching label.

a = pd.Series([1, 2, 3], index=["x", "y", "z"])
b = pd.Series([10, 20, 30], index=["z", "y", "x"])
a + b
# x    31
# y    22
# z    13   <- aligned by label, not by position

Calling .reset_index(drop=True) on one side to 'fix' a mismatch. That does not align anything; it renumbers the labels so the mismatch stops being visible and the rows are now silently paired wrongly.

Practise it: Add a Line Total Column, Each Product's Share of Its Category

Relationship to NumPy

What it is

Under most columns is a NumPy array, reachable with .to_numpy(). Pandas adds labels, per-column dtypes, missing-value handling and alignment on top of it.

Why it matters

It explains the performance model: work that stays inside NumPy is compiled and fast, and anything that drops back into Python per element is not. It also explains why object columns are slow — those are arrays of Python pointers.

In the real world: When a data scientist hands a cleaned table to scikit-learn to train a churn model, it goes in as the NumPy array underneath the DataFrame.

df = pd.DataFrame({"a": [1, 2], "b": [3.0, 4.0]})
df["a"].to_numpy()     # array([1, 2]) — int64, no copy needed
df.to_numpy().dtype    # float64: one dtype had to cover both columns

Assuming .values or .to_numpy() on a mixed-dtype DataFrame is free. It has to find one dtype that fits everything, which usually means object, and silently costs you both speed and type information.