Pandas · Module 17 of 22 · 4 concepts · about 5 min

Window Calculations

Statistics over a moving frame of rows — and the direction they must never look.

At work: Trends and running totals: 7-day average sales, a customer's spend so far, rolling fraud signals.

rolling

What it is

A sliding window: rolling(7).mean() over seven rows, or rolling("7D").sum() over seven days on a time index. min_periods decides how partial windows are treated.

Why it matters

Recent-behaviour features — spend in the last seven days, error rate in the last hour — are the most predictive features in most applied problems.

In the real world: A 7-day average of daily sales smooths out the weekend spikes so a manager can see whether the business is really growing.

Visual example

Windows

A rolling window keeps its width and slides: each output sees itself and the rows just before it.

rolling(3).sum()

input

output

NaN

The first few sums

3 + 1 + 4 = 8

1 + 4 + 1 = 6

4 + 1 + 5 = 10

Why this matters

Recent behaviour — spend in the last 7 days, errors in the last hour — is among the most predictive features there is, and it never reads the future.

Common misconception

Wrong idea: the first rows get a partial sum.

Correct idea: they get NaN until the window is full. min_periods changes that.

1Cover 3 rowsThe window ends at the current row.
2AggregateSum what is inside it.
3Slide oneDrop the oldest row, take the next.

ts = df.set_index("ts").sort_index()
ts["spend_7d"] = ts["total"].rolling("7D").sum()          # seven days
ts["avg_7"] = ts["total"].rolling(7, min_periods=3).mean()  # seven rows

A row-count window on irregular data is not seven days, it is seven observations. Use the time-based form when the interval is what matters.

Practise it: Rolling 7-Day Spend Per Customer

expanding

What it is

A window that grows from the start of the series: expanding().mean() is the running average of everything so far.

Why it matters

It expresses cumulative-to-date metrics without a manual loop, and unlike a global mean it only ever sees the past.

In the real world: Tracking a new store's average daily sales since it opened, updated every day.

Visual example

Windows

An expanding window never drops a row: its left edge stays put and it grows by one each step.

expanding().mean()

input

output

2.67

2.25

2.8

3.83

Each mean uses all history so far

(3 + 1) / 2 = 2

(3 + 1 + 4) / 3 ≈ 2.67

(… + 1) / 4 = 2.25

Why this matters

It is a running average that can never leak: every row sees all of its past and none of its future.

Common misconception

Wrong idea: expanding is a big rolling window.

Correct idea: rolling forgets old rows; expanding keeps all of them — row n always sees rows 1 to n.

df["avg_to_date"] = (
    df.sort_values("ts").groupby("user_id")["total"].expanding().mean()
      .reset_index(level=0, drop=True)
)

Cumulative functions

What it is

cumsum, cumprod, cummax, cumcount accumulate down a column, and combine with groupby to accumulate within each group.

Why it matters

groupby(...).cumcount() numbers events within a session, and a cumsum over a boolean is the standard way to assign session ids from gap flags.

In the real world: A running total of revenue through the month, to see on the 20th whether the monthly target will be hit.

Visual example

Time series

cumsum keeps a running total — and under a group-by it restarts for each group.

sorted by time

user	total
a	10
a	5
b	7
a	2

groupby("user").cumsum()

running total per user

user	total	running
a	10	10
a	5	15
b	7	7
a	2	17

The running totals

a: 10 → 15 → 17

b: 7

Why this matters

'Orders so far' and 'spend to date' are cumulative features. They are leak-free only if the frame is sorted by time first.

Common misconception

Wrong idea: cumsum gives the same answer whatever the row order.

Correct idea: it adds up the rows above it. Sort by time first, or the running total includes the future.

df["event_no"] = df.groupby("session_id").cumcount() + 1
df["running_spend"] = df.groupby("user_id")["total"].cumsum()
df["peak_so_far"] = df.groupby("user_id")["total"].cummax()

Windows and leakage

What it is

A window must be causal: it may include the current row and earlier ones, never later ones.

Why it matters

This is the leak that most often explains a model that scores brilliantly offline and fails in production — it was trained on the answer.

In the real world: A fraud model must only use a customer's past transactions. A window that peeks at future ones scores brilliantly in testing and fails in production.

ts["ok"] = ts["total"].rolling("7D").mean()                 # past only
ts["leaks"] = ts["total"].rolling(7, center=True).mean()     # sees the future
ts["also_leaks"] = ts["total"].shift(-1)                     # tomorrow's value

center=True on a rolling window. It looks like a smoothing option and it pulls future values into every row.

Practise it: Fill Missing Prices With the Median, Pad Each Store's Own Calendar, Not the Global One, Rolling 7-Day Spend Per Customer, Point-in-Time Feature Join