Two Ways to Make a Busy Store Handle More Customers
Imagine a small store that starts getting crowded. One option is to make the cashier desk bigger and hire a super-fast cashier who can process each customer faster. That is vertical scaling: making one thing stronger. Another option is to open more checkout lanes so multiple customers can be served at once. That is horizontal scaling: adding more things.
Both help, but in different ways. A single bigger cashier may still become a bottleneck, while many lanes can share the load. In systems, the same idea shows up with services, databases, and traffic spikes.
What Scalability Means and Why It Matters
Scalability is the ability of a system to handle growth in traffic, data, or workload without collapsing or becoming too slow. It matters because real systems rarely stay at one size: usage grows, traffic spikes happen, and failures are unavoidable.
The key tradeoff is that scaling is not only about making something faster. It is also about making it more resilient and easier to operate. Interviewers often test whether you understand that different workloads stress a system differently: compute-heavy services, read-heavy APIs, and write-heavy workloads each scale in different ways.
Vertical vs Horizontal Scaling
Vertical scaling means adding more power to one machine: more CPU, memory, faster disk, or a bigger instance type. It is simple because the application usually stays in one place, but it has a hard ceiling and can create a single point of failure.
Horizontal scaling means adding more machines and spreading the load across them. This usually improves capacity and availability, but it requires the system to tolerate distribution: requests can land anywhere, state must be shared or recreated, and coordination becomes part of the design.
Why Stateless Services Scale Better
A stateless service does not depend on local memory or local disk to remember a user’s past requests. Each request contains everything the service needs, or the needed state lives in a shared store.
This matters because stateless services can be duplicated easily. If any instance can handle any request, a load balancer can send traffic to whichever instance is free. That is the main reason stateless services are a natural fit for horizontal scaling.
Reads and Writes Do Not Scale the Same Way
Reads are often easier to scale than writes because many users can read the same data at the same time. You can cache results, replicate data to read replicas, and serve from multiple copies without changing the answer.
Writes are harder because the system must preserve correctness. Every write may need coordination, ordering, locking, or consensus to avoid lost updates or inconsistent state. That is why write scaling often hits a central bottleneck sooner than read scaling.
Latency vs Throughput
Latency is how long one request takes. Throughput is how many requests the system can complete per unit time.
These are related but not the same. A system can have good latency for a few requests and still have poor throughput under load, or it can process many requests per second while individual requests wait in a queue and feel slow. Scaling often improves throughput first, but if contention or queueing grows, latency can still get worse.
Availability and the Nines
Availability is the fraction of time a service is usable. The “nines” describe how much downtime is allowed: 99% is two nines, 99.9% is three nines, and so on.
More nines mean less downtime, but the cost rises quickly. Horizontal scaling often helps availability because one node failing does not take the whole service down. But availability is not automatic: your dependencies, deployment process, and failover behavior also have to be designed for it.
A Search API Under Growing Traffic
Suppose you run a search API that is mostly read traffic. A single server handles the initial load, but traffic triples after launch.
A practical scaling path might look like this:
Start with vertical scaling to buy time quickly.
Make the API stateless so any instance can serve any request.
Add more app servers behind a load balancer for horizontal scaling.
Put popular search results in a cache and replicate the database for more read scaling.
Keep writes centralized or carefully coordinated, because indexing and updates still need correctness.
This improves throughput by serving more queries per second and helps availability because one server failure is no longer catastrophic.
Load balancer
-> stateless API instance A
-> stateless API instance B
-> stateless API instance C
Reads -> cache / read replicas
Writes -> primary store
A Write-Heavy Counter Service
Now imagine a global like-counter where many clients increment the same number. Reads are easy: you can show the current count from replicas or cache it briefly. Writes are hard because every increment must be applied exactly once and in order.
If you add more app servers, you do not automatically scale writes, because they still all contend on the same underlying counter. The bottleneck is not the API layer anymore; it is the shared state that must stay correct. This is a classic interview trap: more servers only help if the bottleneck can actually be distributed.