How do you manage SSL for a multi-tenant application?
For example, I will instruct Client A to setup a CNAME record pointing to my app. Then my app should securely serve the application. I can possibly use something like auto-sni to handle SSL cert generation, etc.. At the same time, I also want security features that Cloudflare offers. However, > $5000 per month is not gonna work for me at the moment. I'm wondering if anyone here has any recommendations?
Openresty with a Lua plugin to automatically request letsencrypt certs is how I do it on normal servers.
If I'm using kubernetes the nginx helm package has parts of what was kube-lego built in now so you can set up your ingress to automatically request letsencrypt certs.
Just make sure in both cases you visit each domain first yourself as the first request takes a while.
I have never used cloudflare sorry can't contribute anything to that part of the question.
I'm not familiar with Kubernetes. I guess this will be my homework for this week! Thank you!
You probably don't need kubernetes to begin and that'll just be a distraction.
https://github.com/GUI/lua-resty-auto-ssl is what you want to start which is the openresty/lua solution in line 1.
This is a pain point, and is not easily solved -- which is why Cloudflare charges so much for this feature.
Internally at my company, we built our own solution for this after determining that existing solutions were either too expensive for us or inadequate. I'll briefly run over the technical details and you can decide if it's worth it for you.
Essentially, the objective is to allow a multi-tenant SAAS to provision SSL termination on any domain that has added a CNAME to one of the SAAS application's referencing domain.
First, you'll need to handle SSL certificate issuing. Nowadays, Let's Encrypt is the easiest way to do this. (I won't go into details about the autorenewal thing, because that's quite easy compared to the rest.)
The problem with SSL authentication via Let's Encrypt is that you will need to verify that the domain is correctly pointing to your servers. What you'll need to do is have some "central" proxy server, preferably behind a load balancer, that will handle this.
Once handled and the domain that you want a SSL cert for is authenticated, you'll need some way of getting the issued cert, saving that information (preferably encrypted at rest), and then pulling that information out and updating your SSL termination servers with the appropriate certs.
That's the general overview of how this whole thing works. Customers love it that they don't really have to do much (eg. moving their domains to Cloudflare, for instance) and are delighted that the SSL cert is issued without intervention from them. But the whole thing is a PITA (pain in the...) for us. Hope that helps.
Hell yeah! PITA!
Thank you for detailed information! That's very helpful. My only concern is I don't feel safe without having Cloudflare sit in front of my app. I want Cloudflare security features. How does your company deal with security stuffs?
Yeah... hence the PITA.
You gotta encrypt everything. Like everything in the database (eg. the issued certs), restrict access, private network, etc. And buy insurance.
We spun up an entire infrastructure on AWS just for this one feature. (We chose AWS as they have one of the most fine grained access controls out there.)
But compared to Cloudflare which charges $5k/month, this thing costs us much less.
Another caveat is that you don't get a global CDN with this -- as you've pointed out things don't go through Cloudflare. The custom domains SSLs must terminate on your servers for this to work.
Route 53 + API Gateway + AWS Lambda with Rust can reduce operation costs a lot (golang eats more mem due to a runtime).
I've seen some companies (in Ukraine) managed to run their "serveless" stuff on a free plan, the only limitation is the DB throughput, size and latencies so far.
So, in general "serverless" is like trading off some cost efficiency for an astonishing complexity.
As for the CloudFlare's security features - it's not like you can't launch a dedicated CDN via Amazon nowadays.
DDoS protection and WAF can be crafted along the way when there would be such demand.
"Serverless" is not a vendor lock-in nowadays, 'cause spunning up a K8S cluster with some OpenLambda is a nobrainer.
Just to be clear. Are you talking about rendering React (or any frontend) app in AWS lambda? If so, how do you deal with SSL cert issuance, etc. for each tenant?
Usually all the AAA services (Authentication, Authorization, Accounting) are implemented with Lambda. State mutations are better handled with Lenses abstraction.
Route 53 provides cheap enough wildcard certs for that.
Route 53 is like 40 cents per million requests, and it's a good DNS DDoS protection.
Route's 53 certs are actually very good, but not suitable for http/2 yet, as far as I remember.
API gateway manages load balancing.
Rendering React components with lambda is a bit tricky, because you have to properly cache and invalidate things.
I'd suggest you check out grammarly/focal for react state management, I do know the dudes behind it.
Feel free to mail me if you've got further questions.
Thanks! This gives me some ideas to research
I make an API call to cloudflare and create the CNAME subdomain for each tenant. I store the tenant names in my db to make sure I don't have duplicate names that would throw an exception. I haven't run into fees yet?
Hey.
Cool Question. However for a lot of people who don't have in depth "knowledge" of the topic of certs and SSL as well as multi-tenant apps/architecture how about someone providing a few "highlevel" use cases. Or a couple of good tutorial/link/pointers!!
Keep building up!
I'm by no means an expert. Here are some of the stuffs that I researched in the last few weeks.
fly.io provides out-of-the-box support for multi-tenant app. No need to worry about SSL, CDN, etc.. But it's a bit pricey for me. Also I don't like the fact that I can't use my domain name for CNAME records.
Cloudflare also provides out-of-the-box support for multi-tenant app. If you're gonna talk to their sales rep, just say SSL for SaaS or Managed DNS. They will give you more information. Before you contact them, just know that the minimum you have to pay for that service is $5000/mo. Don't get heart attack hearing that. If you don't intend to provide custom domain setup for your clients, you can just use Cloudflare API to add subdomains to your DNS settings. However there's a 3500 subdomains artificial limit.
DIY - use
auto-sni, setup load balancer, etc..Example use case: Say you build a SaaS product where companies can sign up and manage a forum for their clients. You want them to be able to send users to forum.theirdomain.com and access their forum instead of yourdomain.com/theirname.
Not sure if it could help you but I wrote a blog post about how to setup multiple subdomains with free SSL from Cloudflare: https://pawelurbanek.com/2018/01/22/setup-multiple-domains-with-free-ssl-from-cloudflare/
auto-sni looks interesting. I'm planning to use autocert for go which looks very similar.
These solutions seem the easiest, and don't even require CNAME changes which is great for clients.
Depending on how many tenants you have and how automated you want it to be, you could also use AWS. ACM can issue certs for you with CNAME validation and API Gateway offers some security afaik (enver tried API Gateway).
Is autocert a Go package? I'm using Node and Elixir on the backend.(just realized you already mentioned that)AWS ACM seems like a viable solution. Gonna dig into that. Thanks!
Yes it is. You could run go as a reverse proxy in front of your node app but since auto-sni looks like it has the same functionality there's not much of a point to that.
The thing with ACM is that it has a 100 cert limit by default. Technically you could have 10 domains per cert but that might be hard to manage. ACM does however also provide wildcard certs which is pretty nifty.
Yea, that's true. I want to use auto-sni but I don't feel safe without having Cloudflare sit in front of my app. I don't like my app "naked".
If that's the case, I think ACM won't work for me since my clients will want to use their own domain names.
You need to look into SSL wildcards, it should easily solve this problem.
Bummer Let's Encrypt still does not support them :(
Yea. Wildcard SSL is something that I've already setup. The problem is my clients will want to use their own domain names.
Based on a quick google search it seems like your clients would have to set up an SSL for their domain name.
https://serverfault.com/questions/454070/ssl-cname-and-multiple-domains
My clients don't have servers. They're non-technical folks. So SSL setup needs to be done on my side. And the Q&A also suggests that I can use SNI to handle this sort of configuration.
How many custom SSLs here? With CA records and others issues managing SSL via cname presents some challenges. Eg support for end users which move dns but don't re-add your cname etc.
You could set up another nginx reverse proxy for custom ssls on a higher plan to make management overhead worth it.
Zendesk does it this way where basic plans can only have yourcompany.zendesk.com but you need a higher plan for support.yourcompany.com.