Real-Time Analytics Platform — DataFlow Project
Real-Time Analytics Platform — DataFlow Project: free step-by-step lesson with examples, common mistakes, and interview tips — part of MySQL Tutorial on Toolliyo Academy.
On this page
MySQL Tutorial · Lesson 96 of 100
Real-Time Analytics Platform
Basics ✓ → Advanced
Advanced · 2 — Production · ~10 min · MySQL — Real-World Projects
What is this?
Real-time analytics ingests events (orders, clicks) into MySQL staging or summary tables, often via queue workers rolling up per minute/hour. MySQL serves dashboards; warehouse handles petabyte history.
Why should you care?
Ops wants “orders last 5 minutes” on TV wall — rolling summary table updated every 30s beats scanning raw orders each refresh.
See it live — copy this example
Run in MySQL Workbench or the mysql CLI.
CREATE TABLE orders_per_minute (
bucket_start DATETIME NOT NULL PRIMARY KEY,
order_count INT NOT NULL DEFAULT 0,
revenue_inr DECIMAL(14,2) NOT NULL DEFAULT 0
);
INSERT INTO orders_per_minute (bucket_start, order_count, revenue_inr)
VALUES (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:00'), 1, 499.00)
ON DUPLICATE KEY UPDATE
order_count = order_count + VALUES(order_count),
revenue_inr = revenue_inr + VALUES(revenue_inr);
SELECT * FROM orders_per_minute
WHERE bucket_start >= NOW() - INTERVAL 15 MINUTE;
What happened?
- Upsert increments current minute bucket.
- Dashboard SELECT reads tiny rollup table — sub-10ms.
- Raw orders table untouched for live chart.
Practice next
- Create rollup table.
- Simulate worker INSERT ... ON DUPLICATE KEY UPDATE on each order.
- Query last 15 minutes for chart.
- Add avg_order_inr generated column.
- Read rollup from replica for dashboard only.
Remember
Rollup tables for live KPIs. Upsert increments buckets atomically. Keep hot summary small; archive raw to warehouse.
DataFlow NOC screen
Grafana polls orders_per_minute every 10s during sale.
Outcome: Ops sees dip in revenue within one minute.
Interview prep for this lesson
Practice these questions aloud after reading—each links to a full structured answer.
Sign in to ask a question or upvote helpful answers.
No questions yet — be the first to ask!