Guides for how to scale your web app

Something missing from many online courses out there, and often learnt-on-the-job, is how to scale a web app up to more users – 100K, 1M or even more than 11M. Due to the COVID-19 situation and with school closures forcing kids to stay at home, Khan Academy scaled to 2.5x their web traffic in a week by using serverless architecture and CDN caching of all static data. They also extensively cached common queries, user preferences and session data.

You can also adopt a bottleneck-centric approach to scaling, by checking your resource monitoring to identify the bottlenecks. It is usually the database first. But bottlenecks can be memory, CPU, Network I/O or Disk I/O.

As a principle, make the web stack do less work for the most common requests.

Some ideas:

• Cache database queries

• Index the database

• Move session storage to an in-memory caching tool

• HTML fragment caching

• Use queues and more workers

• Use HTTP caching headers

• Add a Content Delivery Network in front of a static file host

Here's a guide for scaling up to 11M users, stage by stage:

• Use vertical scaling early on but it has no failover or redundancy.
• At >1000: Add availability zones, load balancers, and slave database to RDS.
• At 10K-100Ks: Horizontal scaling of instances. Move static content to S3 and even some dynamic content to the Cloudfront CDN. Add more read replicas of the database to RDS. Shift session state off your web tier and store session state in ElasticCache or DynamoDB
• At >500K: Add automation tools and decouple infrastructure. Add monitoring, metrics and logging.
• At >10M: Use federation, sharding and explore other types of DBs

--
I'm not allowed to share the multiple links to the guides here, but you can find all the links to the original guides in this post.