Drag boxes. Watch the system break. Understand why.
You can draw a system on a whiteboard and it will look perfect. Then real traffic shows up and the database falls over. StackMonkey is where you find that out first. Drop in your boxes, crank up the load, and see what breaks.
10 Designs to Start From
Ten classic system design problems, each sized for a stated load. Open one, push it past its limit, and see which box gives out first. Each one also says what it leaves out.
URL Shortener
Redirects check a cache first and only misses reach the database. New links take a separate path, so a flood of them cannot slow redirects. About 20,000 requests a second at peak.
Chat Service
50,000 people keep a WebSocket open and send messages through Kafka into a Cassandra history store. Stop a chat server and watch everyone reconnect at once.
News Feed
A new post goes through Kafka to a worker that fills the timeline cache, so reading a feed is one lookup. Slow the worker down and feeds go stale while reads stay fast.
Video Streaming
Viewers fetch video segments from a CDN that answers 96 percent of them from the edge. Purge the CDN and watch the origin take everything, or surge the uploads and watch the transcode queue.
File Storage & Sync
A Dropbox style service: 4 MB chunks in S3, a metadata database behind a cache, and a queue that tells your other devices. Pause S3 or the database and see which parts keep working.
Payment System
Every payment is written to a primary database with a synchronous replica, so no accepted payment is lost. Stop the primary and watch the failover. A slow merchant backs up webhooks without delaying payments.
API Rate Limiter
One customer sends 6,000 requests a second against a 3,000 limit. The gateway turns the extra away with 429s and the services behind it never see them. Switch the limit off to see the same flood unprotected.
Notification System
Services send 5,000 notifications a second into a push queue and an email queue. Slow the push provider and only the push queue grows. Email and the callers carry on.
Web Crawler
Fetchers spend their time waiting on other people's servers, so they need many workers, not many cores. Cut the fetchers and the backlog grows. Lose the seen URL filter and the queue fills with repeats.
Ride Sharing
40,000 drivers send their location every few seconds, kept in Redis and never in the trip database. Riders look up nearby drivers and ask for rides. Kill the location store and riders cannot be matched, but ride requests still go through.
Watch These Break Under Load
Each of these is a built in design. Stop the box, slow it down or press the spike button, and watch what the rest of the system does about it.
The Retry Storm
A checkout service calls a slow inventory service with a 150 ms timeout and three retries. During a ten-second traffic spike, retries bury the inventory service. Without a circuit breaker, it never recovers.
The Cache Goes Down
Redis stops. Redirects keep working because the service falls back to the database, but the database now takes every read. It gets much busier and a little slower. At double the traffic it would not survive.
A Slow Provider
The push provider starts taking three seconds to answer. Only the push queue backs up, with thousands of waiting messages. Email keeps flowing, and the services sending notifications never notice.
The Cold CDN
Someone purges the CDN. Until it refills, nearly every viewer request goes to the origin instead of the edge. The origin goes from comfortable to flat out, and it is the next box to fail.
How We Check the Numbers
The queue part of the engine is tested against textbook formulas. Each test runs the same situation in the simulator and in the math, and checks that the answers match.
Average wait times match the textbook formula for one worker and for many workers, from a nearly idle system to a nearly full one.
When the waiting room is full, new requests are turned away. The share turned away matches the formula.
The average number of requests inside the system equals how fast they arrive times how long each one stays. It is a good check that worker counts add up.
Waiting time when response times vary: a few slow calls hurt even when the average looks fine. Checked for fixed, exponential and long tailed response times.
The checks cover the queueing part of the engine. StackMonkey does not run your code, and it does not know your real hardware or network. It shows how a design behaves when workers, queues and connections run out, with numbers you set yourself.
Who It Is For
Architects and Senior Engineers
Put a running design next to your proposal instead of a diagram. Find out roughly where it runs out of workers or connections before you pay for the machines.
- ✓ Back up a design with numbers
- ✓ Find the box that fails first
Platform and SRE Teams
Try timeouts, retries and circuit breakers on a model before you change them in production. See how one slow service drags down the ones that call it.
- ✓ Try timeouts, retries and circuit breakers
- ✓ Watch queues build when a box slows down
Students and Interview Prep
Build the classic designs, push them until they break, then try a fix and see if it helped. The formulas from your course show up on real numbers, so they stop being abstract.
- ✓ Same design, same result every time
- ✓ Checked against textbook queueing formulas
Frequently Asked Questions
Excalidraw, Draw.io and Lucidchart draw boxes and arrows. They look fine in a slide, but they cannot tell you what happens when traffic doubles or one database gets slow.
StackMonkey runs the diagram. You connect clients, services, databases, caches and queues, send traffic through them, and watch queues fill, latency climb and requests fail. It shows p50, p95 and p99 latency, queue depth, worker and connection limits, and which box gives out first.
Yes, and it is a good use for it. Interviewers like to ask "what happens when traffic triples?" or "why is p99 bad when CPU is only 40%?", and most people answer with a guess.
Here you can build the design, push it past its limit, see which box breaks first, and then try a fix (more workers, a cache, a rate limit, a circuit breaker) to see if it actually helped. It also shows Little’s Law (L = λW) on real numbers, so the formula stops being abstract.
You can build each of these and watch it happen:
- Retry storms: retries add load to a service that is already struggling, which slows it down further and causes more retries.
- Cold cache floods: a cache that starts empty, or loses its keys, sends every read to the database at once.
- Connection and thread starvation: slow calls hold on to workers and connections, so fast calls queue up behind them.
- Head of line blocking: a few very slow requests make everything behind them wait, so p99 climbs while the average looks fine.
- Dead and slow dependencies: stop a box mid run, add network delay, and see whether timeouts, retries and circuit breakers contain the damage or spread it.
Network partitions and packet loss are not simulated.
It means the queueing part of the engine is tested against textbook formulas. The test runs the same situation in the simulator and in the math, and checks that the answers match. Four results are used:
The average number of requests inside a system (L) equals the arrival rate (λ) times the average time each request spends there (W). It is a good check that worker and connection counts add up.
The average wait in a queue when service times vary (E[S²] is the mean square of the service time, ρ is utilization). It explains why a few slow calls hurt latency even when average CPU looks fine.
The share of calls refused by c servers with no waiting room, at offered load A = λ/μ. It is what a pool with no queue does when every worker is busy, and it is the zero queue case of M/M/c/K.
c servers with room for K waiting requests, so extra requests are refused. The closed version has a fixed number of users who each wait for a reply before sending the next request.
Same design, same result. The simulator moves its own clock from one event to the next instead of using real time, so a run does not depend on how fast your laptop is. Run it twice and you get the same latency percentiles, queue lengths and drop counts.
The test suite compares it with the queueing formulas: mean latency lands within about 5% of the formula, utilization within 2 percentage points, and the refusal rate within 0.6 points.
Yes. Most numbers on every box can be changed.
- Traffic: requests per second with Poisson arrivals, optional busy and quiet swings, a spike button that doubles the load for 10 seconds, or a fixed number of users who pause between requests.
- Service time: fixed, exponential, or a long tailed lognormal shape, plus workers, cores, memory and queue size.
- Policies: connection limits, rate limiting, timeouts, retries with a fixed delay, and circuit breakers with a recovery time.
AI Review evaluates your active simulation data and provides a structured 4-part architectural critique: a verdict with simulated metrics, per-hop request flow traces, what works well, what bottlenecks will break under load, and prioritized recommendations on what to change first (such as connection pool sizing or replica failover).
It is task-aware during challenges, scales to large event budgets on freeform designs, and includes 5 free reviews per rolling 24-hour day with a free Google sign-in.
Challenges place you on an empty canvas with an active timer to build an architecture that satisfies specific traffic volumes and survives dependency failures. Anyone can browse challenge briefs without an account; starting a curated (Easy, Medium, Hard) or AI-generated challenge requires a free Google sign-in.
Submissions are evaluated live by the simulation engine against 6 criteria: steady throughput, p99 latency, failure survival, latency during outage, required components, and machine count budgets (1.5x for Medium, 1.25x for Hard).
Find out what breaks first.
Build a design, stop boxes mid run and push the load up. Free to use, and you only need a Google sign in to save, export, or take design challenges.
A deterministic simulator built to break whiteboard designs.
Every system looks clean on an architecture diagram until real traffic arrives. StackMonkey gives you a discrete-event simulation engine with measured bare-metal runtimes, realistic cascading failures, and mathematical checks against queueing theory.
Break It & Red Outlines
One-click stress scenarios. Overloaded boxes glow red, stressed nodes outline amber, and summary cards tint with weakest-link indicators.
5 Measured Bare-Metal Stacks
Node.js/Express, Python/FastAPI, Go/net-http, Rust/axum, and Java HttpServer measured on a physical test machine. Every single number displays its provenance.
"Show Your Working" Cards
Estimated limit cards identify your bottleneck node. Per-node breakdown cards compare textbook hand maths side-by-side with live discrete simulation.
Interactive & AI Challenges
Curated break-and-fix challenges across Easy, Medium, and Hard, plus on-demand AI challenges. Free Google sign-in unlocks the live timer, machine budgets, and 6-point grading.
Instant Run — Zero Signup to Simulate
No sign-in walls or credit cards to model and simulate. Anyone can drag boxes and test architectures immediately. Free Google sign-in is only needed to take challenges, export, or sync designs.
Simulate, saturate, and break
Draw your topology with 14 draggable building blocks. A discrete-event engine steps its internal clock forward from event to event, running live at 1x, 2x, 5x, or 10x speed. Because requests block workers and fill bounded queues, failures cascade naturally rather than silently succeeding.
Pause or Stop Any Box
Freeze a primary database or pause a worker pool mid-flight. Watch how quickly requests back up, timeouts trigger, and circuit breakers open.
One-Click "Break It" Stress
On the URL shortener template, one click freezes Redis and doubles client traffic. Watch PostgreSQL hit 100% CPU, success drop to ~83%, and the estimated limit pin PostgreSQL as the weakest link.
Spike Picker (2x, 5x, 10x)
Instantly multiply incoming traffic across every client node proportionally—scaling open Poisson rates, concurrent users with think times, and open WebSockets without altering their relative share.
Red & Amber Overload Heat
Overloaded boxes glow red, stressed nodes outline amber, and KPI summary cards tint with weakest-link indicators so you spot bottlenecks at a glance.
Load Balancer Software Presets
Switch balancer software between Generic, nginx (passive health, max_fails=1), HAProxy (active checks every 2s, redispatch), and AWS ALB (active checks every 30s). Evaluates Round Robin, Least Connections, and Power of Two.
Connection Circuit Breakers
Protect struggling dependencies. Configure consecutive failure trip thresholds (breakerFailures) and cooldown windows (breakerOpenMs) with half-open probe testing to halt cascading brownouts.
Realistic behaviour, not idealized diagrams
Real-world distributed systems don't fail neatly. They suffer from cascading retry storms, memory leaks, thread starvation, and cold-cache floods. StackMonkey simulates the nuanced physical constraints of services and databases.
Timeouts & Retry Storms
Configure retry counts, backoff multipliers, and deadlines. Watch naive retries multiply traffic onto a database that is already struggling, creating an inescapable brownout that backoff alone cannot resolve.
Network Faults: Lost Calls & Cut Links
Inject connection loss (0–100% lossPct) or severed links (cut). A lost answer means server work was completed but the caller timed out, triggering duplicate work and redeliveries. Severed links drop WebSockets with reconnect backoff.
Single-Core & Idle Cores Realism
Node.js and Python run CPU work on 1 core per process (processes setting); Redis runs single-threaded. Sizing a Node service up to an 8-core instance does nothing unless processes are scaled up—StackMonkey flags idle cores explicitly.
Memory Budgets & OOM Kills
Every service has a memory budget. When request concurrency and in-flight payload queues exceed available RAM, the process crashes (OOM), in-flight calls fail, and the service suffers restart downtime. Swap slowdown triggers above 85% RAM.
Cache Fallback & 5 Strategies
Model read-through, cache-aside, write-through, write-behind, and write-around patterns. Toggle the fallback switch: when Redis is paused, full, or timed out, traffic routes around the cache to the database instead of failing.
Replication Lag, Shards & Failover
Buffer cache dynamics calculate disk read delays when datasets exceed RAM. Model primary-replica replication lag (>100ms lag flagged as stale reads), partition skew across sharded tables, and automatic leader promotion during failovers.
Numbers you can trust (your real differentiator)
Most simulators generate synthetic numbers from arbitrary random math. StackMonkey's engine runtimes are benchmarked on dedicated hardware, and every number explicitly discloses its source.
Engine Benchmark & Modeling Matrix
We distinguish bare-metal empirical measurements from published architectural specs.
Timed on a dedicated 2 vCPU physical test machine to measure context switching, per-core throughput, and memory allocations. Unmeasured frameworks (Django, Flask, Rails, Spring) are deliberately omitted:
Engines where physical bare-metal isolation was unavailable are modelled rigorously from vendor benchmarks and peer-reviewed whitepapers:
Source Provenance on Every Stat
Hover over any parameter in the inspector. Every single number states its exact origin: measured, published, assumed, or typed by you.
"Show Your Working" Cards
The Estimated Limit card identifies your weakest component with an uncertainty band. Detailed breakdown cards display step-by-step hand calculations side-by-side with simulator results.
Checked Against Queueing Theory
Verified live against 8 analytical queueing proofs: M/M/1, M/M/4, M/M/1/K blocking, M/G/1 Pollaczek-Khinchine, M/D/1, Little's Law, and closed crowd finite sources.
Read validation proofs →Learn with templates and live challenges
Stop memorizing static whiteboard diagrams. Build intuition by interacting with production-grade templates and solving auto-checked break-and-fix challenges.
10 Architecture Templates
Pre-built architectures sized honestly (50–70% busy on busiest tier): URL shortener, chat service, news feed, video streaming, file storage, ride sharing, rate limiter, notification system, web crawler, and payment system.
Tagged: Reference vs. ExperimentalCurated & AI Challenges
Tiered into Easy, Medium, and Hard, plus on-demand AI challenge generation. Free Google sign-in opens the challenge canvas with a live elapsed timer, tracking your best solve times across reloads.
Easy, Medium, Hard · Free Google sign-in required6-Point Automated Grading Engine
Evaluated against 6 criteria: steady throughput, p99 latency, failure survival, p99 during outage (hard), required components, and machine budgets (1.5x for Medium, 1.25x for Hard). Trivial shortcuts fail automatically.
Show a solution unlocked after solveAI Architectural Review (Beta)
Click AI Review for an instant structured critique: Verdict with simulated stats, per-hop request flow traces, what works well, what will break under stress, and prioritized fixes. Includes 5 free reviews per rolling 24 hours.
Task-aware · 5 free reviews / dayDiagnostics & Loss Logs Drawer
An interactive inspector explains exactly why each request failed: down, queue full, rate limited, timed out, or process killed by OOM. Pinpoints missing consumers, saturated queues, and replication lag.
Pinpoints missing consumers and lagBuilt for your actual workflow
StackMonkey respects your flow. You can model, break, and explore complex distributed topologies with zero friction and zero proprietary lock-in. A free Google sign-in unlocks challenges, exports, and cloud sync.
Native draw.io (.drawio) Export
Export directly to uncompressed .drawio files with embedded metadata. Open directly inside draw.io or diagrams.net for architecture reviews and presentations.
Retina PNG, SVG & JSON
Export high-resolution 2x retina PNGs for design docs, scalable SVGs, JPGs, or portable JSON diagram files. System save dialog lets you pick file name and folder.
Zero-Upload URL Sharing
Share links compress the entire topology, configuration, and traffic profile into the URL hash itself using deflate-raw. Nothing is uploaded to a remote server.
File Interface for AI Agents
Coding agents (Claude, Gemini, Cursor) can generate, import, and test system designs via standard JSON architecture files documented in the agents specification.
Download for-agents.md spec →Undo, Redo & Safety Copies
100-step canvas history with undo/redo shortcuts, automated local autosave, named designs, and an automatic "Previous Canvas" backup if you accidentally reset.
Zero Signup to Simulate
Jump right in. You don't need to create an account, enter an email, or provide credit card information to model and simulate. Free Google sign-in is only needed to take challenges, export diagrams, or sync designs.
Engineering Integrity & Scope Guarantees
StackMonkey is an educational and analytical tool designed to teach resilience and capacity planning. Here is what we promise never to claim:
StackMonkey is a deterministic learning model with measured inputs, not a replica of your AWS VPC or live production cluster.
Only Node.js/Express, Python/FastAPI, Go/net-http, Rust/axum, and Java HttpServer are measured on bare metal. Unmeasured frameworks are omitted.
Kafka, SQS, SNS, S3, NoSQL and CDN are modelled from published specs, never misrepresented as physical hardware benchmarks.
Downstream service calls execute sequentially in order; abandoned calls consume downstream capacity unless shedding expired calls is enabled.
AI Review and AI Challenges run on Amazon Bedrock to critique topologies and generate verified scenarios with rolling daily quotas (5 reviews/day, 3 challenges/day). The engine simulation math remains 100% deterministic.
We use standard aggregate analytics (Google Analytics) to monitor simulator usage and never claim "zero telemetry" falsely.
Ready to break your first architecture?
Jump straight onto the canvas with pre-configured templates or build your own distributed system from scratch.
Built because whiteboard boxes never fail.
Every system looks clean on an architecture diagram. The boxes have neat arrows, everything scales forever, and the database answers instantly. Then real traffic arrives. Connection pools run out, retries pile more load onto a service that is already struggling, and one slow query makes everything behind it wait.
StackMonkey replaces guesswork with a simulator that runs in your browser. Its queueing math is checked against textbook formulas.
Same Result Every Run
The simulator moves its own clock from one event to the next instead of using real time, so a run does not depend on how fast your laptop is. Run the same design twice and you get the same numbers.
Checked Against the Math
Queues and worker pools are tested against textbook formulas, including Little's Law and the Pollaczek-Khinchine formula. The test suite keeps results within a few percent of the math.
Live Failure Injection
Stop a box, slow a dependency, empty a cache or start a retry storm, and watch how the failure spreads to the boxes around it.
Follow StackMonkey
Follow our updates, suggest a feature, or tell us what broke.
LinkedIn arrow_outward
Connect on professional updates, distributed systems deep-dives, and platform milestones.
Twitter / 𝕏 arrow_outward
Follow announcements, queueing simulations, failure experiment breakdowns, and tips.
GitHub arrow_outward
Updates and announcements from the team.
Support arrow_outward
Direct email assistance for technical questions, bugs, feature suggestions, or enterprise modeling inquiries.
Try your design before you build it.
Nothing to install. Free to use, and you only need a Google sign in to save, export, or take design challenges.
Privacy Policy
Effective date: October 2026 · Last updated: October 2026
1. Information We Collect
We collect minimal information necessary to deliver and authenticate your experience:
- Google sign in: If you sign in, Google tells us your email address, name and profile picture. We use them to identify your session and show who is signed in.
- Your designs: Designs you choose to save while signed in are stored on our database provider (Supabase). A design you only share by link lives inside the link, and nothing is uploaded.
- Messages you send us: The text of messages you send from the Ask AI window, with your account, so we can see what people want. Please do not put anything private in them.
- Product updates choice: Whether you ticked the box to receive product update emails.
- Anonymous usage counts (our own): Named events such as "ran a simulation" or "opened a template", with a random visitor ID kept in your browser and the date. No cookies and no IP address are stored, and these counts are never linked to your account.
- Google Analytics and Google Tag Manager: Google receives page visits and the same event names, plus technical details your browser sends such as device type, browser, language and an approximate location worked out from your IP address. Google processes this under its own policy.
2. How We Use Your Information
We use your information exclusively to:
- Sign you in and keep your saved designs.
- Count visits and see which features are used, so we know what to improve.
- Keep the service working and prevent abuse.
- Send product update emails, only if you ticked the box for them. You can ask us to stop at any time.
- Communicate vital system security or service status notices.
3. Cookies & Local Storage
We keep as much as possible in your browser instead of on a server:
- Local storage: Your theme, your current and saved drafts, which challenges you solved, notes you dismissed, and the random ID for our anonymous counts. Our sign in provider also keeps your signed in session here (not in a cookie).
- Cookies: We set none ourselves. Google Analytics sets its own cookies on this site to recognise a returning visitor.
- Opting out: Our own counter is switched off if your browser sends Do Not Track or Global Privacy Control, or if you set "sl-no-track" to "1" in local storage. Google Analytics does not currently follow those signals. To stop it, use your browser's tracking protection or Google's browser add-on at tools.google.com/dlpage/gaoptout.
4. Third-Party Infrastructure
We partner with leading cloud infrastructure providers to run StackMonkey:
- Google (sign in, Analytics, Tag Manager): Sign in with Google, and the visit counting described above.
- Supabase: Holds accounts, saved designs, your Ask AI messages and your update choice.
- Cloudflare: Serves the site and protects it from attacks.
5. Data Retention & Deletion
You retain complete ownership of your simulation designs. You may delete saved designs yourself in the app, or ask for your account to be erased by emailing us at [email protected]. Erasing an account also erases its designs and Ask AI messages. Our anonymous usage counts are not tied to you, so they cannot be looked up or deleted per person. Google keeps its Analytics data for the period set in our Google Analytics settings..
6. Contact Us
If you have questions regarding this Privacy Policy or wish to exercise data rights, reach out to our team at [email protected], or connect with us on X / LinkedIn at @_StackMonkey.
Terms of Service
Effective date: October 2026 · Last updated: October 2026
1. Acceptance of Terms
By accessing or using StackMonkey (the "Service"), you agree to be bound by these Terms of Service. If you do not agree to these terms, you may not access or use the application.
2. Description of the Service
StackMonkey provides a browser-based discrete-event simulation platform for distributed systems engineering, queueing theory verification, and capacity planning. The service includes interactive canvas modeling, telemetry graphing, and failure injection tools.
3. User Authentication & Accounts
To save system topologies and maintain customized configurations, users authenticate via Google OAuth. You agree that:
- You will maintain the security and confidentiality of your credentials.
- You will notify us immediately of any unauthorized use of your account.
- You are responsible for all activities that occur under your authenticated session.
4. Intellectual Property & Your Content
Your Architecture Designs: All system architectures, topologies, custom parameters, and simulation results created by you remain your sole intellectual property. StackMonkey claims no ownership over your designs.
Platform Software: The StackMonkey simulator, user interface, brand assets and simulation algorithms belong to StackMonkey. All rights reserved. Your own designs remain yours, as described above.
5. Acceptable Use Policy
You agree not to use the Service to:
- Attempt to compromise, disrupt, or launch denial-of-service attacks against StackMonkey servers or edge infrastructure.
- Circumvent authentication, rate limits, or security restrictions.
- Deploy automated scrapers or crawlers without explicit written authorization.
6. Disclaimer of Warranties & Limitation of Liability
StackMonkey simulations are based on mathematical models and discrete-event approximations. While our models are verified against canonical queueing theory, real-world production networks are subject to unpredictable physical variances (kernel packet drops, hardware faults, hypervisor scheduling).
THE SERVICE IS PROVIDED ON AN "AS IS" AND "AS AVAILABLE" BASIS WITHOUT WARRANTIES OF ANY KIND. IN NO EVENT SHALL STACKMONKEY BE LIABLE FOR ANY INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF OR INABILITY TO USE THE SERVICE.
7. Modifications & Inquiries
We reserve the right to modify these terms at any time. For questions regarding these terms, contact our support team directly at [email protected], or connect via LinkedIn or X (@_StackMonkey).
Numbers validated against queueing theory, not animated request flow.
Most system design visualizers simply animate colored dots across SVG paths with arbitrary timeouts. StackMonkey is different: every single node, worker pool, queue buffer, and load-shedding policy is evaluated against exact closed-form solutions from operations research and classical queueing theory.
8 Mathematical Proofs Tested on Every Build
We test the discrete event dispatcher across continuous parameter spaces. Theory applies strictly under steady-state conditions (timeoutMs = 0 and maxRetries = 0).
M/M/1 Mean Delay & Queue Depth
Evaluates single-worker Poisson arrival queues across utilization levels ρ = λ / μ from 0.10 to 0.92. Validates that simulated queue wait strictly converges to the asymptotic singularity as ρ → 1.
M/M/c Erlang-C Delay Model
Tests multi-threaded worker pools with c ∈ {2, 4, 8, 16} concurrent execution threads. Validates that queueing probability and waiting times conform to the Erlang-C delay function.
M/M/1/K Finite Capacity Drop & Blocking
Tests bounded queue buffers under sustained overload (ρ = 1.6). Validates that request drop rates strictly mirror state-space birth-death balance equations when capacity K is saturated.
Little’s Law Invariance (L = λW)
Little’s Law holds across ANY queuing discipline, arrival distribution, and network topology. We prove that the time-average number of items in flight L strictly equals effective throughput λ multiplied by mean dwell time W.
M/G/1 Pollaczek-Khinchine Variance Scaling
Real production workloads do not follow exponential service times; database queries and file transfers exhibit heavy tails. We verify that queue wait times correctly scale with the coefficient of variation Cs².
M/D/1 Deterministic Service Times
When service duration is constant (Cs = 0, such as fixed cryptographic hashing or static cache evaluation), queue wait times are exactly halved compared to exponential service (M/M/1).
Poisson Thinning Across Balancers & Caches
Splitting a Poisson arrival stream λ with Bernoulli probability p yields independent Poisson substreams with rate pλ. Proves that load balancer round-robin and cache hit bypasses preserve memoryless inter-arrival intervals.
Closed-Crowd Finite Source Dynamics
Simulates fixed concurrent user pools (N users with think-time Z). In contrast to open Poisson traffic, arrival rates self-throttle when requests are held in flight, accurately matching Little's interactive law.
Where Every Number Comes From
A simulator is only as trustworthy as its inputs. StackMonkey explicitly tags the provenance of every component and parameter so you can audit the chain of evidence.
Hardware Loopback Tests
Base framework and storage engine costs timed on dedicated 2 vCPU loopback hardware. Covers Rust, Go, Java 21, Node 22, Python 3.13; Postgres 16, MariaDB, Redis 7, and RabbitMQ.
Cloud Vendor Limits
Derived directly from published SLA and architecture limits: DynamoDB partition token buckets, Kafka partition consumer group concurrency, and S3 503 SlowDown capacity ceilings.
Algorithmic Queues
Approximations for complex multi-node distributed primitives where loopback timing is impractical: multi-region network latency, Cassandra replica rings, and cross-shard scatter-gather.
User-Defined Logic
Your proprietary business logic, custom service runtimes, and external third-party webhook latencies. These inputs are provisional placeholders until calibrated with production telemetry.
Why Math Alone Fails in Production
Closed-form queueing equations only apply in steady state with zero timeouts (timeoutMs = 0) and zero retries (maxRetries = 0). The instant clients begin timing out after 150ms and retrying three times, the steady-state assumption collapses:
Textbook equations establish the baseline ceiling; StackMonkey’s discrete event engine simulates the nonlinear cascade that closed-form math cannot calculate.
8 Honest Technical Bounds (What We Do Not Model)
We believe in total transparency. StackMonkey is designed to evaluate architectural queue contention, backpressure, and capacity limits, not to make inflated claims.
Simulate Your Design Against Textbook Theory
Drag boxes, configure queue depths, inject failure scenarios, and verify where your architecture breaks before shipping to production.
URL Shortener System Design: Capacity Planning, Bottlenecks & Simulation
A complete architectural breakdown of a TinyURL / Bitly tier service handling 50,000 read QPS: Base62 key generation, 100:1 read/write traffic ratios, 80/20 cache sizing, database connection pool exhaustion, and discrete event simulation.
Requirements & Capacity Planning
Production sizing starts from hard capacity constraints and storage budgets, not hand-waving. Here is the operational capacity math for 100M writes/month under skewed read traffic.
checklist System Requirements Matrix
- Given a long URL, return a unique 7-character short URL (e.g.,
https://short.io/abc12xy). - When accessing a short URL, redirect users to original destination via HTTP 301 or 302.
- Users can optionally provide custom aliases and expiration timestamps (TTL).
- Analytics recording (clicks, referrer, geolocations) without blocking redirect latency.
301 (Permanent): Browsers cache the redirect locally; subsequent clicks bypass your servers completely. Decreases server load by 35% but blinds click analytics.
302 (Found / Temporary): Every request hits the backend first. Mandatory if accurate billing, rate limiting, and real-time click telemetry are required.
calculate Back-of-the-Envelope Capacity Math
Key Generation Strategy: Base62 vs Distributed Range
How do you encode 64-bit integer IDs into compact URLs without hash collisions, database locks, or coordination bottlenecks?
MD5 / SHA-256 Truncation
Compute MD5(original_url), convert to Base62, and take the first 7 characters.
Centralized Auto-Increment ID
Store URLs in PostgreSQL with an auto-incrementing BIGSERIAL primary key. Convert integer ID to Base62.
Distributed Range Key Generator
ZooKeeper or Raft coordinator assigns unique blocks of 100,000 IDs to compute instances. Each worker generates IDs locally in memory without coordination.
Length 6: 62^6 = 56,800,235,584 (56.8 Billion unique URLs)
Length 7: 62^7 = 3,521,614,606,208 (3.52 Trillion unique URLs). At 500M URLs/month, length 7 lasts for over 580 years.
Where the Naive Design Crashes Under Load
Why naive architectures fail in production: Little's Law, thread starvation, and database connection pool exhaustion.
The 50,000 QPS Direct Database Crash
Junior engineers often assume a relational database with an indexed hash column can handle 50,000 reads/sec. Queueing theory proves mathematical impossibility on single-node storage:
The Cascade Sequence
- Connection Pool Exhaustion: Postgres default
max_connections = 100saturates in under 2ms. Subsequent queries queue in memory. - Upstream Worker Lockup: Node.js event loops and Go worker routines block waiting on the DB pool, exhausting thread limits.
- Client Retry Storm: Edge clients time out after 150ms and trigger 3 retries, multiplying ingress load from 50k to 150k QPS.
- Total 504 Outage: API Gateway returns HTTP 504 Gateway Timeout across 100% of incoming traffic.
The Resilient Production Architecture
How StackMonkey models the complete, fault-tolerant topology with edge caching, token bucket throttling, read-through Redis clusters, and async Kafka logging.
allkeys-lru to automatically cycle out cold links.
Simulate the URL Shortener Topology Live
Watch Redis absorb 80% read traffic, observe database pool saturation under cache eviction, and adjust queue depth thresholds in real-time.
Chat Service System Design: 500k WebSockets, Epoll Limits & Pub/Sub Routing
A complete architectural breakdown of a Discord / Slack / WhatsApp scale real-time messaging cluster: stateful TCP/WebSocket connection pooling, Linux epoll file descriptor limits, Redis Pub/Sub session affinity routing, and Cassandra time-series commit logs.
Socket Memory & Epoll Capacity Math
Why naive HTTP REST polling melts infrastructure at 500k concurrent users, and how operating systems budget RAM and file descriptors for long-lived WebSockets.
Short Polling HTTP Breakdown
If 500,000 active mobile clients poll for new messages once every 2 seconds, the gateway cluster receives an unsustainable request barrage:
Linux Kernel TCP Overhead
Unlike stateless HTTP, each WebSocket holds a permanent TCP socket descriptor and kernel buffers in gateway memory:
Epoll & File Descriptors
Default Linux distributions cap process file descriptors at 1,024, immediately rejecting connection 1,025 with EMFILE: Too many open files.
Session Affinity & Inter-Gateway Pub/Sub
Alice is connected to Gateway Pod 02. Bob is connected to Gateway Pod 09. How does Gateway Pod 02 route a message to Bob in sub-millisecond time without broadcasting to every server?
Distributed Session Registry (Redis)
When a client connects and completes TLS negotiation, the gateway writes an ephemeral session record into Redis Cluster:
Clients emit heartbeat pings every 45s to extend the 90s TTL. If a gateway node dies without sending a disconnect event, stale sessions expire naturally within 90 seconds.
Inter-Gateway Message Bus (Redis Pub/Sub)
Each gateway pod subscribes to its own unique topic channel: channel:gw-pod-09.
- Alice sends message destined for Bob to Gateway 02.
- Gateway 02 checks Redis: Bob is on
gw-pod-09. - Gateway 02 publishes payload to
channel:gw-pod-09. - Gateway 09 receives event, finds Bob's local socket descriptor, and flushes frame.
If Bob has no active Redis key, Gateway 02 immediately drops an event onto RabbitMQ to trigger Apple APNs / Google FCM mobile push alerts.
The Reconnect Thundering Herd Storm
What happens when a cell tower blip drops 50,000 mobile devices simultaneously, and all 50,000 attempt to reconnect within 1 second?
The TLS Cryptographic CPU Spike
Maintaining an established TCP connection costs near-zero CPU. However, establishing a TLS 1.3 connection requires Diffie-Hellman asymmetric key exchange (ECDHE curve computation) and RSA signature verification.
The Cascade Death Spiral
- Event Loop Lockup: Gateway CPU reaches 100% saturation. Event loop lag surges from 2ms to over 4,000ms.
- Liveness Probe Failure: Kubernetes kubelet checks
GET /healthz. The blocked thread cannot respond within 3 seconds. - Pod Termination: Kubernetes kills 4 of the 10 gateway pods for failing health checks.
- Cascading Pod Collapse: Remaining 6 pods instantly absorb the entire 50k handshake queue, freeze, and are killed in turn. The cluster enters an endless CrashLoopBackOff.
The Resilient Production Architecture
How StackMonkey models the fault-tolerant Chat Service topology with Layer 4 load balancing, session affinity registries, Redis Pub/Sub, and ScyllaDB time-series storage.
Simulate the Chat Service Topology Live
Stress test 500k concurrent WebSocket sessions, inject network disconnect waves to simulate thundering herd reconnects, and verify queue depth thresholds live in your browser.
Designing a Scalable Social Timeline & Hybrid Fan-Out Engine
A production-grade news feed must serve cached timelines in sub-100ms while ingesting tens of thousands of new posts every second. Learn how to solve the celebrity fan-out bottleneck using hybrid push-pull architectures, Redis sorted sets, and asynchronous queue dispatching.
Capacity Math: Read Heavy 100:1 Asymmetry
Timeline consumption outpaces timeline generation by orders of magnitude. The system must optimize ruthlessly for the read path while preventing fan-out write amplification from overwhelming worker message brokers.
calculate Read & Write QPS Calculations
- 1. Read Throughput: 300M active users open their feed an average of 10 times daily = 3B total timeline fetches. Baseline average: 3B / 86,400s = 34,700 QPS. Peak load multiplier (10x during viral breaking news) = 350,000 QPS.
- 2. Write Throughput: 300M posts created daily = 3,470 write QPS average. Peak ingress reaches 35,000 QPS.
- 3. Fan-Out Amplification: Average user has 200 followers. Standard fan-out requires 3,470 * 200 = 694,000 queue writes/s. Celebrity users with 20M followers would require 20M writes for a single tweet, crashing standard worker queues.
memory Timeline Cache Memory Footprint
-
1.
Per-User Cache Entry: Rather than caching full post blobs, timelines store only 800 recent
post_ids (64-bit integer, 8 bytes) scored by epoch millisecond timestamp in a Redis Sorted Set (ZSET). - 2. Storage Per Timeline: 800 posts * (8 bytes ID + 8 bytes score + 24 bytes overhead) ≈ 32 KB per user timeline.
- 3. Active Working Set: 300M DAU * 20% high-frequency active cohort = 60M cached active timelines. 60M * 32 KB = 1.92 TB RAM. A modest 16-node Redis cluster with 128 GB RAM per node comfortably holds the hot timeline state.
Push (Fan-Out on Write) vs. Pull (Fan-In on Read)
Neither pure push nor pure pull scales across asymmetric follower distributions. The production solution is a hybrid architecture that dynamically splits user models based on follower thresholds.
| Strategy | Read Complexity | Write Complexity | Failure Mode | Production Suitability |
|---|---|---|---|---|
| Fan-Out on Write (Push) | O(1) Redis ZREVRANGE | O(N) where N = followers | Celebrity write stalls workers; million-follower users trigger backpressure. | Ordinary users only (< 25k followers) |
| Fan-In on Read (Pull) | O(M log M) multi-merge | O(1) single DB insert | High p99 latency; database read amplification degrades cluster under traffic spikes. | Celebrities only (> 25k followers) |
| Hybrid Push-Pull (Recommended) | O(1) + O(K) celebrity merge | O(N) capped at 25k | Requires aggregator proxy layer to merge timeline cache with celebrity feeds. | Standard for Twitter / Instagram / Threads |
The Celebrity Ingestion Algorithm
When an ordinary user (e.g. 300 followers) posts, async worker queues fan out the post ID directly into each follower's Redis timeline. When a celebrity user (> 25,000 followers) posts, the write fan-out is skipped entirely. Instead, the post is saved only to the celebrity's public post list. When a follower requests their feed, the Aggregator Service pulls their cached timeline and merges the recent posts of any celebrities they follow on the fly using a multi-way k-merge in memory.
1. Fetch base timeline: cached_ids = redis.zrevrange("timeline:" + user_id, 0, 100)
2. Identify followed celebrities: celebs = db.get_user_celebrities(user_id)
3. Fetch recent celebrity posts: celeb_ids = redis.mget(["posts:" + c for c in celebs])
4. In-memory k-way heap merge by timestamp: final_feed = merge_sorted(cached_ids, celeb_ids)[:20]
5. Hydrate post metadata from Redis MGET / PostgreSQL read replica
Queue Explosion & Little's Law Saturation
Why naive message brokers collapse during major world events. See the mathematical breaking point where queue backlog becomes insurmountable.
Suppose a background task queue has c = 200 workers, each taking an average service time of 1/μ = 2ms to execute a timeline ZADD. Maximum sustainable processing rate is c * μ = 100,000 tasks/s.
During an unexpected global live event, 5 popular accounts post simultaneously, generating 2,500,000 fan-out write tasks in 500ms. Inflow rate λ = 5,000,000 tasks/s exceeds capacity by 50x.
Utilization ρ = λ / (c * μ) = 50.0. By queueing fundamentals, when ρ > 1, queue length L_q grows linearly toward buffer capacity K. Once buffer fills, queue drops tasks, causing silent timeline desynchronization and missing posts.
- verified Follower Threshold Capping: Strictly exclude accounts with > 25k followers from the write fan-out pipeline.
- verified Active User Filtering: Do not fan-out to followers who have not opened the application in over 14 days. Populate their timelines lazily on login.
- verified Priority Queue Partitioning: Separate instant fan-out queues for friends and close contacts from broad public distribution queues to ensure personal notifications remain fast.
Six-Tier Distributed Architecture
The complete component architecture required to maintain sub-150ms p99 latency across 300 million daily active users.
Simulate the News Feed Topology Live
Test push vs pull fan-out trade-offs, trigger celebrity follower posting storms, and observe queue length escalation before tuning worker pool concurrency.
Designing an Ultra-Low Latency Adaptive Bitrate Video Pipeline
Video streaming represents over 65% of global internet downstream traffic. Learn how to design a fault-tolerant HLS/DASH video delivery network with dynamic chunk encoding, multi-tier CDN caching, and stampede-proof origin shields.
Bandwidth & Transcoding Math
Video architectures are defined by petabyte-scale egress and heavy compute transcoding loads. Here is the mathematical foundation for bandwidth and storage budgeting.
speed Egress Bandwidth Budget
- 1. Average Bitrate: Modern adaptive streaming encodes in H.264 / AV1 across 1080p (4.5 Mbps), 720p (2.2 Mbps), 480p (1.0 Mbps), and 360p (600 Kbps). Blended average playback stream: 2.5 Mbps.
- 2. Concurrent Watchers: 500M video minutes / 1,440 minutes = ~347,000 average concurrent streams. Peak prime-time viewing multiplier (3.5x) = 1.2 million concurrent active streams.
- 3. Total Egress: 1.2M streams * 2.5 Mbps = 3.0 Tbps peak baseline, scaling to 4.8 Tbps during major live broadcasts. Origin servers can never absorb this directly; CDN edge caching is non-negotiable.
video_settings Transcoding Compute & Storage Footprint
- 1. Ingress Volume: 50,000 video uploads/day with average duration of 6 minutes = 300,000 source minutes daily.
- 2. Transcoding Factor: Each video generates 5 bitrate ladders segmented into 4-second `.ts` or `.m4s` chunks. Transcoding consumes ~2x real-time CPU minutes = 600,000 compute minutes/day. Requires a fleet of 500 parallel GPU/CPU worker pods.
- 3. Annual Storage Accumulation: Crunched output totals ~150 MB per video across all resolutions. 50,000 * 150 MB = 7.5 TB/day = 2.74 PB per year stored in object storage.
Adaptive Bitrate Streaming: HLS vs. DASH Chunking
Continuous video playback over variable cellular networks requires dynamic chunk switching. The client video player continuously monitors buffer fullness and TCP throughput to select the optimal bitrate ladder.
Chunk Size: 2s vs. 6s Trade-off
Smaller 2-second chunks reduce startup latency and switch bitrates faster when user bandwidth drops, but increase HTTP request overhead by 300% and reduce video compression efficiency (more I-frames).
Standard practice uses 2-second chunks for the initial 3 segments of a video to ensure instant start (< 800ms), then transitions to 4-second or 6-second chunks for steady-state playback.
Client ABR State Machine
The player does not query the backend for bitrate advice. It uses a purely client-side heuristic algorithm (e.g. BOLA or MPC):
if buffer_fill > 15s and throughput > 1.5 * bitrate_1080p: step up ladder
segment_download_time > segment_duration: step down immediately
Origin Egress Saturation & Thundering Herd Stampede
When a new episode drops or a live stream starts, millions of users simultaneously request the exact same manifest file and chunk 001. See how cache misses bring down storage origins.
Suppose 500,000 viewers join a live broadcast at 8:00 PM sharp. The edge CDN has zero cached copies of chunk segment_0001.ts (file size 1.2 MB).
Without request collapsing, 500,000 edge nodes forward the miss to the origin shield: 500,000 * 1.2 MB = 600 GB requested within a 2-second window = 2.4 Tbps origin ingress spike.
The S3 / storage origin rate limits connections with 503 SlowDown errors. The CDN marks the backend dead, cascading 504 Gateway Timeout errors to 100% of the player fleet.
- verified Request Collapsing (Wait-on-Lock): Edge CDN locks incoming requests for the same chunk URI. Only one request is forwarded to the origin; all remaining requests block and receive the cached response.
- verified Two-Tier Origin Shield: Place a dedicated intermediate caching layer (e.g. Fastly Shield / Cloudflare Tiered Cache) between regional edge PoPs and the S3 storage bucket.
- verified Pre-warming Viral Manifests: Push newly published video manifests and initial 3 video chunks proactively to Tier-1 regional POPs before releasing the video link to consumers.
Six-Component Media Streaming Grid
The complete infrastructure powering global low-latency playback from edge PoPs down to distributed object storage.
Simulate the Video Streaming Network Live
Model multi-tier CDN hit ratios, trigger origin cache stampedes, and verify buffer playback continuity under variable simulated consumer network latency.
Designing a Distributed Block Deduplication & Sync Engine
Architecting a Dropbox or Google Drive clone requires strict file consistency, cross-device sync synchronization, and byte-level deduplication. Learn how to design chunk-based sync engines, content-defined chunking (CDC), and ACID metadata trees.
Chunk Sizing & Deduplication Economics
Splitting files into discrete immutable chunks allows delta sync: only modified blocks are re-uploaded. Sizing the chunk correctly balances metadata overhead against bandwidth conservation.
inventory_2 Chunk Sizing Trade-offs (4 MB Standard)
- 1. Chunk Size Selection: If chunk size is too small (e.g. 64 KB), a 10 GB file produces 160,000 chunks, overwhelming database metadata indexes. If chunk size is too large (e.g. 64 MB), changing 1 byte requires re-uploading 64 MB. 4 MB chunks represent the optimal sweet spot.
- 2. Global Deduplication: Each 4 MB chunk is identified by its SHA-256 hash (32 bytes). Before uploading, the client sends a list of hashes to the server. If a hash exists in the global chunk table, upload is skipped entirely, saving bandwidth and storage.
- 3. Ingress Bandwidth: 50M users modify an average of 4 files/day (16 MB modified data). Total raw daily upload: 800 TB. With 62% deduplication efficiency, real network transfer drops to 304 TB/day (~28 Gbps average).
account_tree File Metadata & Version Tree Capacity
-
1.
Metadata Schema: A file is represented as an ordered sequence of chunk IDs in a relational table:
file_chunks(file_id, chunk_index, chunk_hash). Directory hierarchy stored as an adjacency list with parent pointers. - 2. Total Chunk Metadata: 10 PB storage / 4 MB chunk = 2.5 billion unique chunks. At 128 bytes of indexing data per chunk, the global chunk metadata fits in 320 GB, easily cached in an in-memory index.
- 3. ACID Isolation: Renames and directory moves require strict transactional consistency. A file rename must update the path atomically across all devices without mutating underlying object storage blocks.
Conflict Resolution & Delta Sync State Flow
What happens when two users edit the same document offline? The synchronization engine must detect parent-version divergence and prevent silent data overwrite.
Client-Side Upload State Machine
1. Chunk File: Client chunks modified file into 4 MB blocks and computes SHA-256 for each.
2. Query Server: Client sends POST /files/check-blocks with chunk hash list.
3. Direct S3 Upload: Server replies with list of missing hashes and presigned S3 PUT URLs. Client uploads only missing blocks directly to S3.
4. Commit Metadata: Once uploads verify, client sends POST /files/commit with expected base_version_id.
Optimistic Concurrency & Conflict Forking
The metadata server executes commit with conditional CAS (Compare-And-Swap):
SET version = version + 1, updated_at = NOW()
WHERE file_id = :id AND version = :expected_version;
If 0 rows are affected (someone else committed first), the server rejects the commit with a 409 Conflict. The client creates a conflicted copy (e.g. Document (Alice's conflicted copy).docx) rather than destroying the remote file.
Sync Notification Storm & Socket Exhaustion
Why shared folders with 1,000 collaborators cause cascading connection drops.
Suppose a team pushes 10,000 files into a shared engineering workspace. The notification service must push 10,000 invalidation alerts to all 5,000 member desktop clients = 50,000,000 outbound push notifications.
All 5,000 clients immediately wake up and issue concurrent metadata diff queries: GET /files/delta?cursor=xxx.
If connection pooling limits metadata DB workers to 100 concurrent threads, average response time spikes from 12ms to 4,800ms. Clients time out after 5s and retry with exponential backoff, quadrupling load and freezing client synchronization.
- verified Notification Coalescing: Buffer invalidation events on the server for 2 seconds. Send a single batch cursor update rather than 10,000 individual notifications.
- verified Cursor Delta Pagination: Clients request diffs strictly by monotone cursor sequence. Results are paginated in chunks of 500 items, preventing giant memory allocations in the metadata layer.
- verified Client Jitter & Random Backoff: When reconnecting or receiving bulk notifications, clients apply random jitter (100ms - 3,000ms) before hitting the delta API.
Six-Component Distributed Sync Grid
The complete architecture decoupling control-plane metadata operations from data-plane block storage.
Simulate the File Storage Topology Live
Test 4 MB vs 64 MB chunking tradeoffs, inject network reconnect notification storms, and verify metadata CAS conflict resolution live in the browser.
Designing a Real-Time Geospatial Dispatch & Matching Engine
Matching riders with moving drivers within milliseconds requires processing continuous GPS ping streams, indexing geospatial coordinates into hexagonal spatial grids, and solving bipartite graph matching without race conditions.
Geospatial Telemetry & In-Memory Footprint
Driver location coordinates change constantly. Traditional relational databases cannot survive 250k writes/sec of spatial indexing. The architecture must live entirely in partitioned in-memory spatial caches.
near_me Ingress Throughput Calculation
- 1. Ping Cadence: 1,000,000 active drivers stream an updated GPS packet every 4 seconds = 250,000 updates/second ingress.
-
2.
Packet Payload:
{driver_id: uint64, lat: float64, lng: float64, bearing: uint16, speed: uint8, status: uint8}≈ 48 bytes uncompressed, compressed to 28 bytes via Protocol Buffers. - 3. Ingress Bandwidth: 250,000 * 28 bytes = 7 MB/s (56 Mbps) network bandwidth. Modest network load, but massive state mutation rate.
grid_4x4 Hexagonal Spatial Index (Uber H3)
- 1. H3 Resolution: Resolution 8 cells have an edge length of ~461 meters and an area of 0.737 km², representing the exact walking distance radius for city pickups.
- 2. Memory Allocation: Each driver's active location is stored in Redis Geo / In-Memory H3 Set: 1M drivers * 64 bytes index entry = 64 MB RAM. The entire world's live moving fleet fits inside a single Redis instance.
- 3. Partitioning Key: Sharded horizontally across 16 Redis nodes using parent H3 Resolution 4 cells (~11,000 km² metro regions: NYC, London, Tokyo, SF) to avoid cross-cluster geospatial searches.
Greedy Local Match vs. Batch Bipartite Matching
Why greedy matching causes globally terrible ETAs. Discover why Uber and Lyft batch rider demand into rolling 3-second matching windows.
| Dispatch Pattern | Algorithm | Driver Idle Time | Average Pickup ETA | Concurrency Vulnerability |
|---|---|---|---|---|
| Greedy First-Come (Naive) | k-Nearest Neighbor (KNN) | High (sub-optimal assignments) | 6.4 minutes | Rider A takes the only driver 1 min away; Rider B who requested 100ms later must wait 12 mins. |
| Batch Optimization (Recommended) | Hungarian / Min-Cost Flow | Minimal (global optimum) | 3.8 minutes (-40% wait) | Requires 3-second demand buffer window, but completely eliminates race conditions. |
The H3 Hexagonal Advantage
Unlike square Geohash grids where diagonal cells are $\sqrt{2} \times$ farther than orthogonal neighbors, every hexagon in an H3 grid has exactly six equidistant neighbors. Finding all drivers within a 2 km radius requires simply querying the central hexagon and its 2 concentric rings of neighboring hexagons (19 hexagons total) via constant-time index lookups.
Double-Booking Races & Surge Surge Storms
When high density demand hits a single cell (e.g. concert ends), multiple dispatchers attempt to assign the same 5 drivers simultaneously.
Suppose Rider 1 and Rider 2 request rides at the exact same millisecond in downtown Manhattan. Driver D is the closest driver to both.
Dispatcher 1 checks D (status: IDLE) and dispatches ride offer. Dispatcher 2 checks D (status: IDLE) and dispatches ride offer. Both drivers tap "Accept" on their phone.
Without distributed state machine locks, both rides are confirmed. When D arrives, one customer is left stranded on the curb, generating an immediate customer support escalation.
- verified Atomic Redis CAS State Transitions: Driver status transition from `IDLE` to `OFFERED` is executed via atomic Redis Lua script with a 15-second expiration lease.
- verified Trip Finite State Machine: PostgreSQL trip status transitions follow strict monotonic states: `REQUESTED` → `MATCHED` → `ARRIVING` → `IN_PROGRESS` → `COMPLETED`.
- verified Cell-Level Rate Limiting: Surge pricing algorithms cap local matching queue rates to match verified available driver capacity $\rho < 0.95$.
Six-Component Dispatch Architecture
The distributed infrastructure linking driver GPS telemetry to real-time bipartite matching workers.
Simulate the Ride Sharing Dispatch Live
Test greedy vs batch matching algorithms, trigger localized surge spikes, and observe driver idle time and rider wait latency live in your browser.
Designing a High-Throughput Distributed API Rate Limiter
Defending production services against malicious scrapers and thundering retry storms requires evaluating millions of API requests per second in under 2ms. Learn how to compare Token Bucket vs Sliding Window algorithms, execute atomic Redis Lua scripts, and engineer fail-open circuit breakers.
Latency Overhead & Memory Geometry
A rate limiter sits inline on every API call. If your rate limiter takes 15ms to evaluate, your entire API slows down by 15ms. The system must operate with extreme memory efficiency and zero disk I/O.
timer Latency Budget & Local Reservoir
- 1. Network Hop Latency: A round-trip query to a centralized Redis cluster consumes 1.0ms - 2.5ms across regional VPCs. At 1,000,000 req/s, hitting Redis on every request requires 1,000,000 Redis ops/s, requiring massive sharding.
- 2. Batch Token Pre-allocation: Edge API gateway nodes acquire a block of tokens (e.g. 50 tokens) from Redis in a single atomic call, then satisfy local requests from in-memory token reservoirs with 0.01ms overhead.
- 3. Accuracy vs Performance: Pre-allocation introduces a minor burst tolerance (~5% variance across edge pods) in exchange for reducing Redis network traffic by 98%.
memory Memory Footprint (Sliding Window Counter)
- 1. Bucket State: Storing exact timestamps (Sliding Window Log) requires 8 bytes per request. Under 1,000 req/min, that is 8 KB per user. For 100M users = 800 GB RAM.
- 2. Sliding Window Counter Optimization: Only stores 2 integers: current minute counter (4 bytes) and previous minute counter (4 bytes). Total memory per rate key with Redis metadata: ~64 bytes.
- 3. Cluster Sizing: 100M active tracking keys * 64 bytes = 6.4 GB RAM. A 3-node Redis cluster with 16 GB RAM easily accommodates the entire global rate limiting state with 100% in-memory headroom.
Comparing the 4 Core Rate Limiting Algorithms
Different API endpoints require different algorithms. Login APIs protect against brute-force attacks via sliding window counters; user upload APIs permit traffic bursts via token buckets.
| Algorithm | Memory Complexity | Burst Handling | Boundary Vulnerability | Best Use Case |
|---|---|---|---|---|
| Fixed Window Counter | O(1) - 8 bytes | Poor (drops legitimate spikes) | 2x burst at window edges (e.g. 59s & 01s) | Simple low-traffic quotas |
| Sliding Window Log | O(N) - 8 KB / key | Accurate (exact boundary) | None (mathematically precise) | Sensitive security / auth endpoints |
| Token Bucket | O(1) - 16 bytes | Excellent (allows burst up to capacity) | None (continuous token refill) | General REST API rate limiting |
| Sliding Window Counter | O(1) - 12 bytes | Smooths spikes over window | Approximation error < 0.05% | High-throughput edge gateways (Envoy) |
Atomic Redis Lua Script Implementation
Preventing race conditions requires executing bucket replenishment and token deduction atomically on Redis without multiple round-trip locks:
local limit = tonumber(ARGV[1])
local current = tonumber(redis.call('get', key) or "0")
if current + 1 > limit then
return 0 -- Rejected (429 Too Many Requests)
else
redis.call("INCRBY", key, 1)
if current == 0 then redis.call("EXPIRE", key, 60) end
return 1 -- Allowed (200 OK)
end
The Retry Storm Amplification Trap
Why a poorly configured rate limiter actually worsens cascading failures during a service outage.
Suppose a database slowdown increases API response time to 2,000ms. Clients time out after 1,000ms and immediately retry.
The rate limiter rejects retried requests with 429 Too Many Requests. However, if clients lack exponential backoff and jitter, they retry the 429 immediately.
Incoming request rate $\lambda$ quadruples from 100k to 400k req/s. The edge rate limiter CPU saturates evaluating Lua scripts, degrading edge proxy throughput and turning a partial database outage into a total edge outage.
-
verified
Retry-After Header Standards: Every 429 response must include
Retry-After: {seconds}to force client libraries to sleep before retrying. - verified Fail-Open Circuit Breakers: If Redis cluster latency exceeds 5ms or errors spike > 2%, the rate limiting filter automatically switches to fail-open pass-through mode.
- verified Client Exponential Backoff with Jitter: Enforce client SDKs to implement Decorrelated Jitter: $T_{\text{wait}} = \min(M, \text{random}(B, T_{\text{prev}} \times 3))$.
Six-Component Rate Limiting Grid
The distributed architecture providing low-latency token evaluation across multi-region edge locations.
Simulate the Rate Limiter Topology Live
Test Token Bucket burst capacity, trigger malicious client retry storms, and verify fail-open circuit breaker behavior in the live simulator.
Designing a Scalable Multi-Channel Notification Engine
Delivering 500 million daily notifications across SMS, Push (APNs / FCM), and Email requires priority queue tiering, external vendor circuit breakers, user preference filtering, and distributed deduplication guarantees.
Priority Breakdown & Vendor Rate Limits
Third-party communication providers (Twilio, SendGrid, Apple APNs, Google FCM) enforce strict external rate limits and variable latencies. The notification engine must buffer, throttle, and isolate traffic channels.
send Volume Breakdown by Channel
- 1. Mobile Push (60%): 300M alerts/day. Dispatched over persistent HTTP/2 sockets to Apple APNs and Google FCM. Average latency: 80ms.
- 2. Email (30%): 150M emails/day. Delivered via SMTP / API pools (SendGrid, AWS SES). Average latency: 350ms.
- 3. SMS (10%): 50M messages/day. High-cost, strictly regulated carrier path (Twilio, Sinch). Average latency: 250ms. Hard vendor limit: 100 SMS/sec per shortcode.
layers Queue Sizing & Priority Tiering
- 1. P0 Critical (OTP / Fraud Alerts): SLA < 2 seconds. Never shares a queue with bulk campaigns. Dedicated worker fleet guarantees zero head-of-line blocking.
- 2. P1 Transactional (Order receipts, password resets): SLA < 30 seconds. Buffered during traffic spikes, auto-scales worker concurrency.
- 3. P2 Promotional (Marketing blasts, weekly digests): SLA < 4 hours. Throttled aggressively through rate limiters to avoid saturating email reputation and vendor limits.
Distributed Deduplication & Preference Filtering
Nothing destroys user trust faster than sending the same notification 5 times or blasting users who opted out. Discover the multi-stage filter pipeline.
Idempotency Key Verification
Every inbound notification request contains a client-provided idempotency key (or computed hash of `user_id + event_type + timestamp_bucket`).
The ingestion worker executes an atomic SET {idempotency_key} 1 NX EX 86400 in Redis. If the key already exists, the event is immediately discarded with 200 OK (Duplicate Ignored), preventing duplicated SMS charges.
User Preference & Quiet Hours Filter
Before enqueuing to channel workers, the engine queries the user preference cache:
if current_time in user.quiet_hours and priority < P0:
schedule_delay(queue, user.quiet_hours.end)
else: enqueue_immediate(channel_queue)
Third-Party Vendor Outage & Poison Queue Saturation
When an external vendor (e.g. Twilio SMS gateway) suffers downtime, worker threads block waiting for HTTP timeouts. See how an external failure collapses internal message queues.
Suppose the SMS worker fleet has 100 worker threads, normally processing requests in 200ms (sustainable throughput: 500 SMS/sec).
The upstream vendor gateway becomes unresponsive, holding TCP connections until the 10-second socket timeout expires.
Within 2 seconds, all 100 threads are blocked in `READ_TIMEOUT`. Worker throughput drops from 500/s to 10/s. Inbound messages accumulate in RabbitMQ at 5,000/s. Memory usage crosses threshold $K$, triggering broker node swapping and halting all channels.
- verified Automated Vendor Circuit Breaker: Trip circuit if failure rate > 5% over 10 seconds. Automatically route SMS requests to secondary carrier (e.g. AWS SNS fallback).
- verified Dead-Letter Queue (DLQ) with Exponential Backoff: Failed deliveries move to a DLQ after 3 failed retries. DLQ messages are re-tested with exponential backoff and jitter.
- verified Aggressive Socket Timeouts: Set connection timeout = 1.0s, read timeout = 2.5s. Fast failures prevent worker thread starvation.
Six-Component Notification Infrastructure
The complete fault-tolerant architecture from event ingestion down to carrier vendor dispatch.
Simulate the Notification System Live
Model priority queue isolation between SMS OTPs and email marketing blasts, simulate vendor outages, and test circuit breaker failover live.
Designing a Distributed, Politeness-Governed Web Crawler
Crawling one billion web pages per month requires distributed URL frontier scheduling, strict per-domain politeness throttling, Bloom filter deduplication, and robust protection against infinite spider traps.
Frontier Geometry & Storage Footprint
The web contains hundreds of billions of URLs. The crawl frontier must store, deduplicate, and prioritize prospective links without exploding RAM or thrashing disks.
dataset Bandwidth & Document Ingestion Math
- 1. Average Page Size: 100 KB raw HTML (excluding client media assets). 400 pages/s * 100 KB = 40 MB/s (320 Mbps) sustained inbound network bandwidth.
- 2. Raw Storage Volume: 1 billion pages * 100 KB = 100 TB raw HTML / month. Compressed via Zstandard (4:1 ratio) = 25 TB monthly storage appended to S3.
- 3. Link Expansion Factor: Each crawled page yields an average of 40 outbound links. 1B pages generate 40 billion discovered URLs that must pass through deduplication filters.
filter_alt URL Deduplication (Bloom Filter Math)
- 1. Naive In-Memory Hash Table: Storing 40 billion URLs raw (average 80 characters) requires 3.2 TB RAM for URL strings alone, excluding hash table pointer overhead.
- 2. Bloom Filter Bit Array Formula: For $n = 40\times 10^9$ keys and false positive probability $p = 0.001$ (0.1%), bits required $m = -\frac{n \ln p}{(\ln 2)^2} \approx 575 \times 10^9\text{ bits} \approx \mathbf{71.8\text{ GB RAM}}$.
- 3. Cluster Footprint: 72 GB RAM easily fits across a tiny 2-node Redis cluster with bitfield commands (`SETBIT` / `GETBIT`), reducing RAM requirements by 98%.
The Mercator Two-Tier URL Frontier: Prioritization & Politeness
A naive FIFO crawler either hammers a single web host to death or crawls low-value spam pages indefinitely. The production crawler separates prioritization from politeness into two distinct queue tiers.
Tier 1: Priority Queues (F1 ... Fk)
URLs are ranked by PageRank score, domain authority, and update frequency. High-priority news sites land in queue F1 (crawled every 15 minutes); static documentation lands in F10 (crawled monthly). A biased random selector picks URLs to avoid starvation.
Tier 2: Politeness Queues (B1 ... Bm)
Each politeness queue contains URLs from exactly one domain host (e.g. nytimes.com).
A heap-based Delay Table tracks the next permissible fetch timestamp for each queue:
worker_thread blocks if now() < next_fetch_time
Spider Traps & Infinite Frontier Exhaustion
How dynamically generated calendar pages, session tokens, and symlink loops paralyze distributed crawler fleets.
A misconfigured site generates endless recursive URLs: /catalog/item/1/similar/2/similar/3... or endless calendar pages /events?year=3024.
Because each generated URL has a unique query string or path parameter, Bloom filter deduplication marks them as new URLs.
The frontier fills with millions of junk links from a single domain. Worker threads spend 95% of their CPU parsing dynamic trap pages, starving the crawl of legitimate web content until queue disk buffers crash.
- verified URL Path Depth Hard Cap: Reject any URL with more than 10 slash segments or > 255 total characters.
- verified Per-Domain Page Budget Quotas: Cap non-whitelisted domains at a maximum of 10,000 crawled pages per month.
- verified Content SimHash Deduplication: Calculate a 64-bit fingerprint of page text. If fingerprint similarity > 95% across 20 pages from the same host, flag site as a template trap.
Six-Component Distributed Crawler Grid
The complete infrastructure decoupling URL scheduling, HTTP fetching, document parsing, and storage.
Simulate the Web Crawler Frontier Live
Test politeness delay timing, model Bloom filter false-positive rates, and observe frontier queue length dynamics under simulated spider trap loads.
Designing a Distributed Exactly-Once Payment Engine
Financial payment processing tolerates zero data loss and zero double charges. Learn how to design a fault-tolerant payment gateway with distributed idempotency keys, the Saga orchestration pattern, double-entry ledgers, and transactional outbox architectures.
Double-Entry Accounting & Integrity Invariants
Floating point numbers are forbidden in financial software due to IEEE-754 precision rounding errors ($0.1 + 0.2 \ne 0.3$). Every monetary quantity is stored as an exact 64-bit integer representing cents or micros.
account_balance Double-Entry Bookkeeping Principles
- 1. Fundamental Equation: $\sum \text{Debits} = \sum \text{Credits}$ across every single transaction. Money is never created or destroyed; it is only transferred between accounts.
- 2. Transaction Amplification: A single customer checkout of $50 involves 4 ledger entries: Debit Customer Card ($50), Credit Merchant Settlement ($48.50), Credit Platform Fee ($1.50). 10,000 tx/s requires 40,000 ledger balance mutations/sec.
- 3. Immutable Append-Only Audit: Ledger lines are never `UPDATE`d or `DELETE`d. A refund creates new compensating ledger records, maintaining an unbroken audit trail for regulatory compliance.
fingerprint Idempotency Key Protocol
- 1. Client Key Injection: Every charge request carries an `Idempotency-Key: {UUIDv4}` HTTP header.
-
2.
Distributed Mutex Lock: On request arrival, the orchestrator acquires an atomic lease in Redis:
SET lock:{idempotency_key} {worker_id} NX EX 60. If lock exists, concurrent duplicate requests wait or receive 409. - 3. Cached Response Cache: Once the payment settles, the full response body is cached in PostgreSQL against the unique idempotency key constraint for 24 hours. Retried calls return the identical 200 OK without re-charging the bank.
Distributed 2-Phase Commit (2PC) vs. Orchestrated Saga Pattern
Why modern payment architectures abandon distributed two-phase commit locks across microservices in favor of asynchronous Saga orchestrators with compensating transactions.
| Pattern | Locking Mechanism | Throughput Limit | Failure Handling | Production Recommendation |
|---|---|---|---|---|
| 2-Phase Commit (2PC) | Synchronous distributed DB locks | < 500 tx/s (lock contention) | Coordinator crash leaves database rows locked indefinitely; cascading thread death. | Avoid in distributed architectures |
| Saga Orchestrator (Recommended) | Local ACID DB transactions | > 25,000 tx/s (lock-free) | Compensating backward transactions (reversal / refund events) restore consistency. | Standard for Stripe / Uber / Amazon |
The Transactional Outbox Pattern
Writing to a database and publishing an event to Kafka in separate operations risks silent data corruption if the network fails between the two steps. The Transactional Outbox pattern writes both the business entity and the outgoing event into the same SQL database transaction. An asynchronous Change Data Capture (CDC) process (e.g. Debezium) streams outbox records into Apache Kafka with guaranteed at-least-once delivery.
Bank Gateway Timeouts & The Zombie Charge Dilemma
When the upstream card network (Visa / Mastercard) times out, did the transaction succeed or fail? Discover how blind retries cause double billing.
The payment service sends a POST /charges request to the bank acquiring gateway. The bank successfully debits the customer's credit card for $100.
Before the HTTP response reaches our servers, a network route flap occurs. The socket times out after 3,000ms.
Our service marks the transaction as `FAILED_TIMEOUT` and the user taps "Try Again". If a new charge request is dispatched, the customer is billed twice ($200 total), generating immediate chargebacks, fines, and payment network penalties.
- verified Never Retry Blindly: When a bank gateway times out, transition the payment state to `PENDING_RECONCILIATION`.
- verified Active Status Querying: A background worker polls the acquiring bank's `GET /charges?idempotency_key=xxx` API before triggering any reversal or new charge.
- verified Nightly Settlement Matching: Parse bank settlement batch files (NACHA / ISO 20022) to reconcile ledger entries against actual bank fund transfers.
Six-Component Payment Architecture Grid
The distributed topology guaranteeing zero loss, zero double-charges, and strict compliance.
Simulate the Payment Engine Live
Test Saga compensating transactions, simulate bank gateway timeouts, and verify idempotency deduplication under aggressive concurrent retry attacks.