ML System Design · Module 10 of 47 · 11 concepts · about 14 min

Streaming and Real-Time Features

Computing features from events within seconds of them happening: event streams, stream processors, the kinds of time window, late and duplicate events, the architectures that combine batch and streaming, and when streaming is not worth it.

At work: The part of an ML platform behind fraud detection, live delivery estimates and session-aware recommendations. It is powerful, and the most expensive part to run.

Why real-time features

What it is

Some predictions depend on what happened in the last few seconds or minutes:

fraud: this card was used in three cities in the last ten minutes;
delivery time: forty orders are waiting at this store right now;
recommendations: the user just added pasta, so suggest sauce;
surge pricing: demand in this area has doubled in fifteen minutes.

A nightly batch cannot know any of this.

Why it matters

For these products, freshness is accuracy.

A fraud model that only knows yesterday's activity misses an attack happening now.
A delivery estimate that ignores the current queue is wrong exactly when it matters most, at the rush.
Measure the gain: train with the feature computed fresh and computed a day old, and compare.

In the real world: Kirana Kart's experiments show the live store-queue feature cuts delivery-time error at peak hours by about a fifth, which justifies the streaming job behind it.

Assuming fresher is always better. If a feature's value barely changes within a day, streaming it adds cost and failure points for no gain.

Event streams and stream processors

What it is

Real-time features are built from two parts:

an event stream: an ordered, durable log of events that many consumers can read. Apache Kafka is the common choice; cloud versions include Amazon Kinesis and Google Pub/Sub.
a stream processor: a program that reads events continuously, keeps running state (counts, sums) and writes results. Apache Flink, Spark Structured Streaming and Kafka Streams are common.

Results are written to an online store (Redis, DynamoDB, Cassandra) where the prediction service can read them in milliseconds.

Why it matters

Each part does one job well.

The stream decouples producers from consumers: the app writes events once; any number of feature jobs read them.
The processor handles windows, state and failures.
The online store makes the latest value available at prediction time.

In the real world: Uber, Lyft and DoorDash have each described feature platforms built from Kafka for events, Flink or Spark for processing, and a key-value store for serving.

Visual example

Streaming

The prediction service only does a fast key lookup.

From an event to a feature the model can read

Event'Order placed' to Kafka
Stream processorsecondsFlink updates the store's queue
Online storeKey 'store:118'
Prediction< 5 msETA service reads it

Having the prediction service compute aggregates by querying raw events on each request. It is slow, it hammers the database at peak, and it breaks the latency budget.

Time windows

What it is

A stream processor aggregates events over windows. Three common kinds:

Tumbling: fixed, back-to-back blocks that do not overlap. Orders per 10-minute block.
Sliding (or hopping): a window of fixed length that moves, so each moment looks back the same distance. Payments in the last 10 minutes, at every payment.
Session: a window that stays open while events keep arriving and closes after a gap of inactivity. One shopping session.

Why it matters

The window type changes the feature's meaning.

Tumbling windows are cheap but jumpy: activity just before and after a boundary is split.
Sliding windows match 'in the last N minutes', which is what most features mean.
Session windows match how people actually use an app.

In the real world: Fraud teams usually use sliding windows for velocity features and session windows for 'what has this user done in this visit'.

Visual example

Computed

One card used at minutes 1, 2, 3, 12, 13, 31. The same six events, counted three ways.

Tumbling 10-minute blocks

block	payments
0–10 min	3
10–20 min	2
20–30 min	0
30–40 min	1

Sliding and session

s1 min: at 1 min: 1 payment(s) in the last 10 minutes

s2 min: at 2 min: 2 payment(s) in the last 10 minutes

s3 min: at 3 min: 3 payment(s) in the last 10 minutes

s12 min: at 12 min: 3 payment(s) in the last 10 minutes

s13 min: at 13 min: 3 payment(s) in the last 10 minutes

s31 min: at 31 min: 1 payment(s) in the last 10 minutes

sess: sessions with a 5-minute gap: [1, 2, 3] [12, 13] [31]

Tumbling splits the burst at minutes 1–13 across two blocks; the sliding count shows three payments at minutes 3, 12 and 13.

Using tumbling windows for velocity features. Five payments at 9:58 and five at 10:01 show as two windows of five, hiding ten payments in three minutes.

Practise it: Design a Predictive Maintenance System

Event time and processing time

What it is

Every event has two times:

event time: when it actually happened (the phone recorded the tap);
processing time: when the stream processor received it.

They differ: phones go offline, networks delay, services retry. A watermark is the processor's estimate that 'all events up to time T have probably arrived', so it can close windows up to T.

Why it matters

Windows should be based on event time, or features become wrong whenever delays occur.

An order placed at 7:59 that arrives at 8:02 belongs in the 7:50–8:00 window.
The watermark trades freshness against completeness: wait longer and fewer late events are missed, but features are older.
Decide what happens to events later than the watermark: drop them, or update the window and re-emit.

In the real world: Rider apps buffer GPS pings when signal is lost and send them in a batch later; delivery-tracking pipelines must place each ping at its real time.

Using processing time because it is simpler. During a network hiccup, a burst of delayed events lands in one window, and the feature shows a spike that never happened.

Duplicates and exactly-once

What it is

Streams deliver events at least once: retries mean the same event can arrive twice. To keep counts right:

give every event a unique event ID;
deduplicate on it within a time limit;
make updates idempotent where possible (set 'last order time' rather than increment a counter);
or use a processor's exactly-once mode (Flink and Kafka support it), which costs some speed.

Why it matters

A duplicated payment event doubles a velocity count and can make an honest customer look like a fraudster.

The fix belongs in the design from day one; retrofitting deduplication onto months of state is painful.

In the real world: Payment events at most companies carry an idempotency key from the moment they are created, used both to avoid double charges and to deduplicate the event stream.

Counting events without IDs. A mobile app that retries on a slow network sends the same 'add to basket' three times, and the model thinks the user is unusually eager.

Lambda and Kappa architectures

What it is

Two ways to combine batch and streaming:

Lambda: a batch path computes accurate features from full history; a streaming path computes approximate recent ones; serving merges the two. Two codebases.
Kappa: only a streaming path. History is handled by replaying the stream from the start through the same code. One codebase.

Many modern platforms use a hybrid: one feature definition, compiled to both a batch job and a streaming job.

Why it matters

The main risk is two pieces of code computing 'the same' feature differently.

Lambda's two paths drift apart, which creates training-serving skew.
Kappa avoids that, but replaying months of events can be slow and costly.
Defining each feature once and generating both jobs from it is the common answer today.

In the real world: Feature platforms such as Tecton and Feast let a team define a windowed aggregate once and run it both as a backfill over history and as a live streaming job.

Writing the batch version in SQL and the streaming version in Java by different people. They will disagree on edge cases (time zones, window boundaries) and nobody will notice until accuracy drops.

Backfilling streaming features

What it is

A new streaming feature has no history, but the model needs months of it to train. Options:

Replay stored events through the streaming job (Kappa style).
Recompute the same definition as a batch job over historical events.
Log and wait: start logging the live values and train once enough history exists (module 5).

Why it matters

Training data must hold the value the feature would have had at each past moment.

Replaying or recomputing from event time gives that, if the logic is identical.
Log and wait is the most faithful but the slowest.

In the real world: Kirana Kart backfills its store-queue feature by replaying six months of order and pickup events through the same Flink job in a batch mode, then checks it against a month of logged live values.

Backfilling with today's value for every past row. 'Orders waiting at the store' copied from now into last year's training rows is meaningless, and the model will learn nothing from it, or worse.

Practise it: Design a Real-Time Anomaly Detection System, Design a Real-Time Fraud Detection System, Design an ML Feature Store

When not to stream

What it is

Streaming is worth it only when all of these are true:

the feature's value changes meaningfully within minutes;
the fresher value measurably improves predictions;
the prediction itself is made live, not in batch;
the team can run and support a streaming system around the clock.

Otherwise use batch features, or compute from the request itself.

Why it matters

Streaming carries real, ongoing costs.

Always-on processing clusters and online stores.
Harder debugging: state lives in the processor.
Late, duplicate and out-of-order events to handle.
A failure at 3 a.m. affects live predictions immediately.

In the real world: A team replaces three of its five streaming features with hourly batch versions after finding no measurable loss, and halves its platform bill.

Visual example

Choice

Read down and stop at the first row that fits.

Streaming, batch or on-demand

the feature	use	example
Comes from the request itself	On-demand	Distance to the address
Changes daily or slower	Batch	Average order value over 90 days
Changes hourly; an hour old is fine	Frequent batch	Product popularity today
Changes by the minute and it matters	Streaming	Orders waiting at the store now

Building a streaming pipeline before measuring the gain. Compute the feature in batch with a short lag first; if a one-hour-old value is nearly as good, stop there.

Writing and reading online features

What it is

The stream processor writes, the prediction service reads. Design both sides:

keys: one key per entity (card, store, user), so one lookup returns all its features;
batched reads: fetch every entity's features in one round trip;
expiry: stale values should expire rather than be served for ever;
timestamps: store when each value was computed, so the service can tell a stale value from a fresh one.

Why it matters

Feature fetching is often the largest part of serving latency.

Ten separate lookups at 3 ms each use most of a 50 ms budget.
A timestamp lets the service fall back to a default when the stream is behind, instead of serving a value from an hour ago as if it were current.

In the real world: Kirana Kart's fraud service counts how often it falls back to defaults because a feature is stale; a rise in that count is one of its first alerts.

# One round trip for every entity the request needs.
keys = [f"card:{card_id}", f"user:{user_id}", f"device:{device_id}"]
rows = online_store.mget(keys)                    # ~2 ms

feats = {}
for key, row in zip(keys, rows):
    if row is None or now() - row["computed_at"] > MAX_AGE[key.split(":")[0]]:
        feats.update(DEFAULTS[key.split(":")[0]])  # stale or missing: use known defaults
        metrics.increment("feature_stale", tags={"entity": key.split(":")[0]})
    else:
        feats.update(row["values"])

Serving whatever is in the store with no age check. When the stream processor stalls, the model keeps using frozen values and nobody notices until the predictions are clearly wrong.

Interview questions: streaming and real-time features

What it is

The questions interviewers ask about streaming and real-time features, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Why it matters

Reading an idea and explaining it are different skills.

Each answer below is the two or three sentences an interviewer is listening for.
The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

In the real world: Fraud and marketplace interviews often push here: 'the user's last five minutes matter, how do you get that into the model in time?'.

Q: When do you need streaming features?

When the value changes within minutes, the fresh value measurably improves predictions, and predictions are made live. Fraud velocity and current store queues are classic cases.

Q: What are tumbling, sliding and session windows?

Tumbling: fixed, non-overlapping blocks. Sliding: a fixed-length window looking back from each moment. Session: grouped by activity, closed after a gap of inactivity.

Q: Event time versus processing time?

Event time is when it happened; processing time is when it arrived. Windows should use event time, with watermarks deciding when a window is complete and a rule for late events.

Q: How do you handle duplicate events?

Unique event IDs and deduplication, idempotent updates where possible, or a processor's exactly-once mode.

Q: Lambda or Kappa?

Lambda has separate batch and streaming paths and risks them disagreeing; Kappa uses only streaming and replays history. Today, defining a feature once and generating both jobs is common.

Q: How do you backfill a new streaming feature?

Replay historical events through the same logic using event time, or recompute it in batch with identical logic, or log live values and wait. Never copy today's value into past rows.

Proposing Kafka and Flink for every feature without saying why freshness matters for it.

Practise it: Design a Real-Time Anomaly Detection System, Design a Real-Time Fraud Detection System, Design an ML Feature Store

Exercise: build a velocity feature

What it is

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

Why it matters

An idea you have only read fades in a week.

Building it once, even in fifty lines, is what makes it yours.
Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.

In the real world: This exact feature, built properly, is a common take-home task for fraud and risk ML roles.

Your task. Compute 'payments by this card in the last 10 minutes' from a stream of events, correctly handling duplicates and late arrivals.

Steps

Generate a stream of payment events with event IDs, card IDs and event times; include duplicates and some events arriving up to two minutes late.
Write a small processor in Python that keeps, per card, the event times in the last 10 minutes and updates a dictionary acting as the online store.
Deduplicate on event ID.
Place late events by event time and update the count if they fall inside the window.
Write a batch version of the same feature over the full event history and compare its output with the streaming version for every event.

You are done when

Duplicates do not change any count.
Late events within the window are counted correctly.
The batch and streaming versions agree on every event.

Stretch. Rewrite the processor with Flink, Spark Structured Streaming or Bytewax, using its built-in windows and watermarks.