Architecture Sanity Check

I've mostly worked on smaller hobby projects, but my day job is debugging large distributed setups. So I know planning is important, and a second set of eyes to catch glaring details.

I'm building a note taking app. LocalStorage only example here:
https://cheatsheet-pcv7xzs74.now.sh/

The end goal is pretty simple:

UI deployed to a CDN as a static site
Makes api calls to nginx load balancer
API (application server(s)) for business logic/requests
Postgres DB
Elastic Search instance for search bar (critical component to product)
Redis for some caching/auth at some point

Is this overkill? I feel comfortable with the layout + path each transaction would take in my head.

Max Dobeck

posted to

Developers

on May 30, 2020

Say something nice to maxdobeck…

Post Comment

3
Make your app architecture agnostic and don't worry about scale when starting out. If you're using something like AWS, let it scale as needed.

Eventually you could have something like the following: (there are other cloud equivalents too)
1. Docker container for your application. Can add more containers as needed.
2. Do you need ES? Or can you get away with full text search on your db? If you need ES, use managed ES so you don't need to configure it yourself.
3. Managed DB.
4. Can use S3 to host your static frontend.
5. Managed Redis if you need it, but for a note taking app, probably not. Can always add it in later when there's a definite use for it. Only use if it there's a significant benefit from retrieving something in Redis vs a the regular form. E.g. db query takes too long or API response takes too long. And the data doesn't change often.
All that being said, factor in cost. When I said make your app architecture agnostic, I meant that you could deploy all of this onto a $10 per month server from digital ocean and it would probably work fine. Can always add in fancy architecture later.

I remember once I was building a chat bot for a specific fitness use case and we deployed it using kubernetes 😂 no one ended up using it.

Don't optimize for the future. Spend more time on your product than the architecture. Minimize the amount of work and points of failure at the start.
Timberts

·
4 years ago
·
Reply
1

Awesome thanks @hassanprocesslogic. Would a lone docker container be billed differently than some small compute instance? For app server I was picturing a small ec2 instance (or digital ocean, linode, w/e).

Don't optimize for the future.

wise words

Definitely going to be platform agnostic with managed services as much as possible. Heroku, RDS, etc

Redis would be a nice to have as I hate hitting the DB for basic auth/login stuff. But overkill with only myself as a user rn.

Maybe I don't need ES to start? Its a helluva undertaking and I wasn't looking forward to managing an ES instance. But unfortunately the Search bar is going to be the real winner for me. Its a major focus of the app, information discovery and retrieval.

maxdobeck

·
4 years ago
·
Reply
1. 2
  
  Yea drop elasticsearch. I'll bet you can get it working with postgres if you look deeper into it. I've made this mistake a few times.
  
  ianwalter
  
  ·
  4 years ago
  ·
  Reply
  1. 1
    
    Awesome, relieved to hear that :)
    
    To start, the search just needs to work with a handful of fields. So sticking to that.
    
    maxdobeck
    
    ·
    4 years ago
    ·
    Reply
2. 1
  
  Yeah the docker container billing should be a bit cheaper if I remember correctly.
  
  But if you're going the digital ocean or linode route, and you don't have a lot of dependent services on your app server, then a small machine is fine. Even docker would be overkill in that situation.
  
  Definitely recommend it for dev though.
  
  Yeah you can get away with avoiding Redis for the start. But depending on your framework of choice it should be easy to refactor your most used queries later.
  
  There's actually a very nice (relatively new) search service called meilisearch. It's extremely fast to set up. You could install it on the same machine as your app server. It also comes with a docker image.
  
  It's really good for real time search and extremely fast.
  
  The only downside is that it matches every word of the query. E.g. if you type in red fox, it will look for that combines phrase, rather than sort by relevancy of red and fox.
  
  So if that's an issue, then I'd recommend offloading that logic to something like Algolia which is powered by ES.
  
  Play around with it. Search is always context dependent. We've implemented our own search algorithm at work without ES which will eventually need to be refactored to ES some point later this year.
  
  Good luck man! Happy to chat any time and to learn from you as well.
  
  Timberts
  
  ·
  4 years ago
  ·
  Reply