How we handle 25M API calls a month from 10 global endpoints for $150

Hey @jonathankosgei. Good stuff, I always love seeing performance enthusiasts.

There are a few small details that aren't included in your article that are really important if you care about performance and I'm not sure you're aware of these, so I'm sharing so everyone can learn:

First, you're missing this immensely important quote from the japronto Github readme:

To be fair all the contestants (including Go) were running single worker process.

They intentionally compared apples to oranges to skew the statistics. In fact, Go's fasthttp library is only 18% slower than the entirety of japronto while Go only runs a single worker.

If you use multiprocessing properly (built into Golang, super easy to enable), you get way higher performance than japronto (as you'd expect).

All the others in the benchmark can also handle themselves much better than what the japronto devs published.

Second, if you're making a call to redis for every visitor (before replying, and assuming only one call to redis), your actual throughput limit is less than 100k reqs/sec because you're now dependent on redis. This number is from a benchmark on a bigger instance than yours, but you can use redis-benchmark to figure out your number.

Third, the japronto benchmark gets multiplied by a very high factor because it enables HTTP pipelining. Problem: none of the real world browsers support HTTP pipelining. Chrome used to have it but subsequently removed it and it's now unsupported.

And lastly, a personal tip: change your structure to use a message bus or queue. I use microservices written in Golang with NATS (https://nats.io/) for messaging. NATS handles over 10 million messages per second on my one server.

If it's an API where the client isn't waiting for a response, you can reply with 200 OK after validating the request with a web server, and then publish the task on the message queue. Any of the background workers (NATS can be clustered, and microservices can be hosted on any number of servers) then pick up the job and execute it - but your response was already sent, so response time will be blazing fast.

For requests where you need a response from the worker, NATS supports request/response, so that's also not a problem.

Anything in that infrastructure can be scaled both vertically and horizontally: load balancers, nginx servers in front of your Golang HTTP servers (I use nginx to deal with initial authentication like JWT and serving static assets because nginx is amazingly fast), background workers, NATS cluster, MySQL/MariaDB/... manual cluster.

I hope this is helpful to someone. I use this architecture to serve several hundred million requests per day on a € 50 server that's usually at 10% - 15% CPU, where each incoming request also relies on an external API (over HTTPS) and workers handle it in ~1ms.

(Note: I love this stuff. Really love it. But most of this isn't practical advice for IndieHackers or their startups, it's way overkill unless you're doing very niche things.)