15
14 Comments

I need help with my fullstack architecture for link shortener.

I'm building a short link application like bit.ly using the MERN stack but have ran into a few conflicts on how to best set up my applications architecture to keep everything decoupled and scalable.

What I need:

example.com/:slug - API route for fetching/redirecting short links to long links.

example.com/blog, example.com/docs, example.com/404, example.com, etc. - Static pages

Ideally, I would like the REST API to be on it's own server, and the static page site to be it's own server.

The issue I'm running into currently is I need to let the server know that a GET to example.com and a GET to example.com/blog is supposed to return a static page on the static site server. While a GET to example.com/:slugis supposed to fetch a short link and redirect on API server. I also need the API to redirect/proxy to the static site 404 page when :slug is not a valid/created short link without changing the URL.

What is the best way to structure this where I can use separate servers, all under the same domain, and handle different (dynamic) paths serving different servers?

on October 12, 2020
  1. 4
    1. put a middleware that checks if the request is for any of the static pages /blog /about /pricing etc. if yes, dispatch the req to the static source.
    2. if not, dispatch the request to the url resolver server.
    3. if number 2 fails, dispatch the request to the static source for 404
    1. 2

      +1 on this solution, furthermore you can use nginx to put these rules

  2. 2

    The coding garden on youtube has 2 full streams about that I think, one of them is using Heroku. hope it helps

    https://www.youtube.com/watch?v=szWlL4wlo6E
    https://www.youtube.com/watch?v=gq5yubc1u18

    1. 2

      Awesome thanks, I think I found my solution for now but definitely going to watch those to make sure I don’t miss anything!

  3. 1

    Hi Justin,

    Not exactly matching your current stack but:

    1. Store hash => URL in DynamoDB
    2. Host your static pages on Cloudfront
    3. Have Cloudfront trigger a Lambda@Edge on :slug, which would check DynamoDB and return the redirect (and other logic if you have that)

    And that's about it? That can also be mapped to other providers.

  4. 1

    Hi @Harrjm!

    To do what you want to do, here's what I'd advice (similar to what some folks already mentioned):

    1. Handle all the routes you own or care about first (eg example.com, example.com/blog/*)

    2. Handle everything else as though it was a short url (eg example.com/:slug)

    However there are issues with this:

    1. If I were to type example.com/bloog (a typo), that could possibly route me to someone else's short URL

    2. Can you possibly know all paths you will need in the future? Remember these short urls could be permalinked somewhere else (eg Google). What if I take a short url at example.com/teams this means you can never ever take that path for your internal use cases

    I suspect this is a similar reason why Github booted everybodys pages off the github.com domain and moved them to github.io

    You can definitely special case all the routes you need, and also blacklist any paths you feel you may need in the future (eg /teams, /careers, /policy etc).

    This can work, but for my use cases I just save myself all that trouble and use a separate domain. I am not comfortable with all the magical problems that could be waiting for me in the future. So I just have my site/app/api hosted at example.com, but shortlinks are hosted at example.io/:slug or examplelinks.com/:slug
    This way, the only collisions will be between different people's shortlinks 👍🏾

    Hope that helps!

    1. 1

      I decided to split it to two domains as to reccomend. I hadn’t noticed before that bitly and rebrandly both do that so it reaffirms my idea. My urls will be something like getshort.ly and then short.ly for the short link (not the actual domains, I wish)

  5. 1

    bitly uses bitly.com for it's website and bit.ly for shortening.
    I think it's better if you don't mix both!

    1. 1

      Wow I never actually noticed that! I had just decided to do the same and that just reaffirms my idea! Thanks!

  6. 1

    Where are you hosting your project? If you're on AWS (and even if you're not), using Amazon CloudFront as a CDN is super cheap (will be free for most startups), and makes it really easy to create path based behaviours.

    1. 1

      I typically host my projects on Heroku, sometimes Digital Ocean, AWS often seems to complicated. I do typically use Cloudflare though which I just checked and it has a "Page Rules" functionality which may work for this.

  7. 1

    The easiest way is to combine your front-end and backend API to be served by the same Express server. IMO, if you're just getting started, trying to split up your API and front-end functionality is a bit of a premature optimization; starting with a monolith isn't a bad thing.

    Since Express would be handling all of your traffic, all you have to do is define your "wildcard" url route as the very last thing and you're good to go. All of the /blog, /docs, etc would take precedence since you've defined them directly. You could even still host your static site assets through a CDN and have your server just serve up the equivalent of an "index.html" file. The other benefit of doing things this way is the flexibility of having that active server for your front-end; server-side rendering is now a few bits of configuration away if you want it.

    Alternatively, if you were to put your site or the shortened URLs on a different domain/subdomain, you can pretty much avoid this all together.

    1. 1

      Yeah a different domain/subdomain would avoid most of these issues but I definitely want to avoid my 'short' links being something like api.example.com. But perhaps I can keep everything else on a 'long' domain and use the short domain specifically for the fetching/rendering the short urls? I'll look into it.

      The main reason I want to split things up is so every time I push a new blog post or doc, or update my homepage I don't have to push a new build to the API potentially affecting availability. But perhaps I could implement a CDN to solve that? But I'd think that just complicates things in a different direction.

      I'm sure it's premature optimization I'm just hoping not to bury myself in technical debt while I still have some time before I launch and I'm more concerned with trying to scale.

      1. 2

        I did something similar to what @seanmcgary suggested above.

        I have two codebases- one for the frontend, one for the backend.

        The frontend is Sapper/Svelte and makes calls through Express and/or directly to the backend API. So a user can hit any given page and get rendered the SSR/Static pages there.

        The backend is Ruby/Rails and is strictly APIs for querying/updating data.

        It works pretty well, and gives me the ability to add my next API consumer (mobile flutter app), as well as explore whether or not I'd want to swap out my backend at any point (I've been tinkering with Golang).