One thing I don’t get about website builders is how, technically, to implement a feature like offering a custom domain with HTTPS.
What’s the high-level pieces needed for such a thing?
I run a website monitoring service. One of the features I offer is status pages, which can be served either from my subdomain or customer's domain.
Basically, you need to take care of 2 things:
I've done that with Caddy, Let's Encrypt, and Rails (most backend web frameworks will do).
To generate a Let's Encrypt wildcard certificate with Caddy, you will have to compile Caddy (from the source) with a DNS plugin and create a DNS TXT record. See linked forum threads below for more information.
Caddy can also obtain Let's Encrypt certificates on the fly (On-demand TLS is the term-of-art). For this to work, you'll have to collect the customer's domain, and they need to point a CNAME record to you. Then the first time someone tries to visit that domain, Caddy will issue a certificate on-the-fly (after getting permission from your backend server).
Caddy takes care of certificate renewals as well.
The Rails app is responsible for serving the right page given the subdomain/custom domain, and approving domains for which Caddy should issue certificates.
I don't want to bore you with implementation details too much, but suffice it to say it was quite tricky to get it right. Below are two forum threads where I was trying to figure things out (they are a bit outdated at this point, but should you give a rough idea nonetheless):
You don't have to use Caddy per se, but I found it to be the least painful choice (if you want to have a full control over your software that is).
This is very descriptive and helpful. I am happy to know about Caddy. Thank you.
I've wondered this for at least a year and figured it out for my own stack this morning.
I use kubernetes with traefik (1.7) as an ingress proxy:
Now it works instantly with any subdomain not matched by my other subdomains!
- host: '*.noco.io'
- path: /
I'm not broadly an expert on DNS but I've done exactly this recently for https://profiled.app so I can tell you how I did it:
User publishes their site to a subdomain -> S3 bucket is created containing the website files and configured with a policy to host a website. The name of the bucket must match the domain name -> A Cloudfront distribution is created at the same time which sits in front of the S3 bucket -> An A record is created in Route53 (using the API) to point the domain (e.g. my-website.profiled.app) to the Cloudfront distribution. I have a wildcard SSL certificate, *.profiled.app which provides HTTPS for every subdomain.
When a user adds a custom domain, I use the AWS Certificate Management API to create an SSL certificate for that domain. This also sends a validation email to the user on the same email address they used for their domain name provider. Once they've validated, a new Cloudfront distribution is created using their custom domain as the S3 bucket name and using the new SSL certificate. The URL of this distribution is then returned to the user which they need to add as the value for a CNAME record in their DNS provider. Once that's all done their custom domain will work as expected.
I don't think this is the best way to do it as far as UX goes as it involves multiple actions from the user with lags in between, but it does work. For improvements it would be good to offer a domain purchasing service yourself to save the user time, sounds like @johnny_am has the answers here.
Doesn't S3 have bucket limits?
Up to 1000 yeah. There are ways of getting around this using credentials but this is a no go if your users are non-technical of course. In that case, ignore my answer!
I’ve actually always wanted this explained to me as well! Following along. Thanks for a good question.
I can tell you how I do it. It may not be the best way, but if people suggest a better one then I'll benefit from it too :-).
It's using - mostly - AWS but the same principles should work for any provider. There are two cases, depending on whether or not the user wants to delegate their domain.
In this scenario, the users manage their domains the way they want and do some of the steps on their own.
A subdomain is created on the platform domain, e.g. my-publication.<platform>.tld. This is done using Route 53, an AWS service used to manage domains: you can setup name servers, add A/MX/TXT/etc. records, among other things.
At this point, users can access their website through my-publication.<platform>.tld.
They can go to their external domain management UI and add a CNAME record that makes their domain point to my-publication.<platform>.tld, which basically creates an alias pointing towards the platform, e.g.:
CNAME myowndomain.com my-publication.<platform>.tld
They also need to set the domain they want to use in the platform UI, since it will be necessary to properly manage incoming requests via the host HTTP header: since all user sites will go towards the same API, there needs to be a way to know where they are coming from.
Knowing that domain name is also necessary because a certificate is generated using AWS Certificate Manager. When there is no delegation, a small extra step is necessary to verify that the user owns the domain. Most of that process is handled directly by the AWS service though.
And finally that domain needs to be known because the CDN - AWS Cloudfront in this case - needs to know what content the domain maps to. So each Cloudfront distribution is bound to both the platform subdomain and the "real" user domain.
In this scenario, the users manage their domain entirely through the platform (and can actually buy domains from there too).
Again, Route 53 is used to manage all those aspects.
The only differences with the previous model are:
Note: right now the Gandi API is used to buy domains but AWS also offers an API (that actually uses Gandi).
And I think that's about it. In short: they need a CNAME and to either give you the certificate or to let you generate it for them.
I run a service on Heroku that allows for custom domains as a (tiny) part of it. @johnny_am did a great job explaining how this works technically, I'd simply like to share my strategy as someone who has less devops expertise.
You could do the same proxy yourself, I've often heard Caddy as a viable backend for this due to the integrated Let's Encrypt functionality. I use Fly.io because it's cheap enough for me, and I run my main app on Heroku - I'd rather not get involved in devops at this point.
Hi Marcus -
I don't fully understand your question, but can probably help if you clarify a bit.
Some clarifying questions.
With respect to domains.
with respect to https (i.e. TLS certificate)
Off the cuff (based on how I think I understand it).
There are some domain providers that let you resell or purchase via an api. DNSimple purchase via api, Namecheap to be a reseller.
If you want to self host this, you can automate docker container setups where the site is proxied through something like Traefik or Nginx. In either of these scenarios, you can do a customer TLS certificate, or automate the certificate retrieval and renewals via LetsEncrypt (for free).
LetEncrypt has become a cornerstone for TLS (HTTPS websites) because it's free and can be fully automated.
Again, happy to provide more insight/direction if you clarify your question/need a little further.
First of all thanks for your thoughtful answer.
tl;dr; something along the lines of how the custom TLD feature for Carrd works.
I imagine some user either gets a xxxx.mysaas.com or mysaas.com/xxxx. The user then wants their own top level domain to point to that url. How do I as the owner of “MySaas” technically set up that feature?
The user might bring their own top level domain or buy it through “MySaas”, but the users I imagine are probably not even close to being developers. Probably tech savvy but definitely not developers.
What's your current stack? Particularly on the edge? I.e. do you have a proxy server/load balancer? Where do you host?
The first part is pretty easy. The 2nd part could be tricky, but doable.
To support subdomains with TLS, you have one of two choices.
The 2nd part, the client having their own domain that resolves to your platform is a little trickier. If you just wanted http://their-url.com to redirect to https://yourplatform.com, then you could do this with a DNS entry. If they wanted to keep their url so when users visited http://their-url.com, it retrieved the identical content, then I would recommend having them point their DNS to your IP and then you taking care of the LetsEncrypt on their behalf.
This is an area that doesn't get a lot of coverage for developers (only devOps people). I'll likely start covering some of these topics on my blog soon.
Let me know if I can provide some additional insight/help.
Things are clearing a little. Thanks again for stellar help.
If I understand correctly the gist of it is to have the custom TLD configured with DNS so it points to a service (IP address) that I control.
Then, on my service, I’d have automations in place to provision certs with LetsEncrypt for their TLD, and also a proxy to route the request to the right content.
Baby steps. 🍼 🎊
Sounds like it’s not a completely trivial feature...
Hey Marcus -
Here's the thing. Once you know all the moving pieces, terms, and proper configuration, it's easy. BUT - learning all the moving pieces, terms, configuration is a HUGE learning curve. Especially since everyone uses a different hosting provider, tech stack/framework, ingress, cert provider etc...
If you want to send me your desired stack and host, I can probably whip up a quick working example for you.
Under each of these scenarios, I would take a slightly different approach.
I understand. Let’s go with a stack I’m familiar with then. Nextjs with an Express backend hosted on Heroku. I really appreciate the time you’re putting into helping me (and others!) understand the moving parts.
Creator of a web app collaboration platform here. We have a system in place where people can link a domain to our platform - and we'll take care of all the rest like HTTPS.
We have a Kubernetes cluster with Nginx ingress which automatically takes care of SSL for us. We just create a Nginx deployment with a service and ingress. From the customer's perspective they just need to change one DNS setting so it's very low friction.