Most scaling guides explain the theory. This one is about the order of operations — what to fix first, what can wait, and where teams usually waste months optimising the wrong thing.
The most dangerous point in a SaaS application's life is when it starts working. You have real users, growing traffic, and suddenly the architecture decisions you made when you had 50 users are being tested by 5,000. Most founders and engineering leads already know the theory — horizontal scaling, read replicas, caching, queues. The harder question is what to fix first, and how to avoid spending three months optimising something that wasn't actually the bottleneck. This post is about the order of operations.
Understand What You're Actually Scaling First
Before touching infrastructure, instrument your application properly. You need to know where time is actually being spent on your slowest requests — is it database queries? External API calls? CPU-bound processing? Memory-bound operations? The answer changes the solution completely. We've seen teams spend weeks adding application servers when the real problem was a single unindexed database query running on every page load. APM tools like Datadog, New Relic, or even simple query logging will tell you in minutes what the actual bottleneck is.
The Database Is Almost Always the First Thing to Break
- Add indexes on every column you filter or sort on — this alone can turn a 2-second query into a 5ms one without any infrastructure changes
- Identify N+1 query patterns and fix them with eager loading — these are invisible in development and devastating in production
- Use a connection pooler (PgBouncer for Postgres, ProxySQL for MySQL) before adding read replicas — connection exhaustion kills databases before concurrency does
- Move long-running reports and exports to background jobs so they don't compete with web request queries
- Consider a read replica for reporting and analytics queries once your primary is consistently above 70% CPU
Before any infrastructure investment: run EXPLAIN ANALYZE on your ten slowest queries. In almost every SaaS we've scaled, at least two of those queries have a missing index or a full table scan that is trivially fixable — and fixing them matters more than adding servers.
Application Layer: Stateless Before Horizontal
You can't scale horizontally if your application holds state in memory. Session data needs to be in Redis or a database, not in the server process. File uploads need to go directly to object storage (S3, GCS), not to a local disk. Any shared state between requests or between processes needs to be externalised. Once your application is truly stateless, adding more instances becomes trivially easy — a load balancer and an auto-scaling group, and you're done. Stateful applications require complex session affinity configuration and still fall apart under failure scenarios.
Caching: The Most Underused Lever
- 1Cache database query results for data that doesn't change frequently — user profiles, product catalogues, configuration settings
- 2Use HTTP cache headers (ETag, Cache-Control) for API responses — CDN and browser caching handles a surprising amount of read traffic for free
- 3Cache rendered HTML fragments for complex, frequently-viewed pages using fragment caching
- 4Use Redis or Memcached for application-level caching — set appropriate TTLs and have a cache invalidation strategy before you add the cache
- 5Monitor cache hit rates — a cache with a 30% hit rate is barely helping; tune it until you're consistently above 80% for the things you're caching
Queue-Based Architecture for Everything Non-Real-Time
Any task that doesn't need to be synchronous should be queued. Email sending, report generation, webhook delivery, image processing, PDF creation, third-party API calls — all of these belong in background jobs. Synchronous operations block web worker threads, inflate response times, and make your application fragile when downstream services are slow. A queue decouples request handling from task execution, gives you retry logic for free, and makes your application far more resilient under load spikes.
Monitoring at Scale: What Actually Matters
- P95 and P99 response times, not just averages — averages lie; the P99 tells you what your slowest users are experiencing
- Database connection pool utilisation and wait times — rising wait times are an early warning before connection exhaustion hits
- Background job queue depth and processing time — growing queues with slow workers mean users are waiting for things they expect to happen immediately
- Error rates by endpoint — a 0.5% error rate on a high-traffic endpoint is a high absolute number; don't miss it in the aggregate
- Infrastructure CPU and memory utilisation trends — you want to know you're approaching capacity before you're at capacity
The most common scaling mistake we see: throwing more servers at an application-level problem. More application servers can't fix a slow query, a missing index, or a synchronous API call that blocks for three seconds. Profile before you scale.
A Practical Starting Point
If you're facing scaling pressure today, the quickest wins are almost always database query optimisation and moving long-running tasks to background jobs. These changes are low-risk, deployable in days, and consistently deliver the most relief. After that, add a caching layer to your hottest read paths. Only then should you look at adding application servers or scaling your database infrastructure. The teams that scale successfully are the ones that measure first — they know exactly what's slow and why before they start changing things.
