Does anyone need free advice on the best tech stack / architecture for their startup?
I am an experienced software engineer/architect working with the technology startups for the most of my professional life. I have pretty much seen and done it all, from bootstrapped one-man operations to rapidly expanding venture-backed companies.
I could help you to avoid some basic mistakes when choosing a technology stack, and perhaps give some suggestions on how to scale it as the company grows.
The depth of the advice will mostly depend on how much specific details you are willing to share, but some universal principles apply regardless.
DISCLAIMER: I am not providing consulting services at the moment, nor I am looking to join a startup. I just want to contribute something to this amazing community.
I'd be interested in hearing about your process analyzing this. What are a few common factors you take into account? Any common mistakes people do, and why? What does people usually get right?
I've been a consultant for most of my life, focusing on designing and building smaller scale systems, but haven't worked with the larger VC-backed startups like you - it'd be interesting to hear about your experiences.
This a big question :) Probably a book (or two) could be written on the subject.
Nevertheless, I think the most important factor for the startup of any size is agility/adaptability of their tech stack. You need to be able to iterate - and possibly pivot - very fast, and the technology is often the limiting factor. Our natural tendency seems to be building long-lasting solutions, and we don't ask ourselves often enough "how fast can I change, replace or throw this away if needed". Finding the right balance between flexibility and robustness is something of an art though, and it's very hard to get right.
Another important factor - the alignment between your architecture and the business goals. In practice, this means deciding which processes are incidental to the core model, and thus can be handled by 3d-party software or services, versus the custom code that should target the core of the business. This is often trickier than it sounds, especially when it comes to things like CRM and sales pipeline.
Most common mistakes I observed:
Fantastic answer, despite it being a big question. I especially appreciated the bullet points. Many forget that the tech is there to support the business - not the other way around. When the tech gets in the way of the business or makes business more complicated - then you're on thin ice. Thanks for the valuable nuggets of wisdom!
Hey, Oleksiy!
Thanks for helping the community! I am planning to develop an MVP for an interactive content framework (studio). It basically lets user create interactive widgets (games, tests, calcs, quizzes) and embed them into existing sites (iframe/pure js - what do you recommend?)
So it's going to be UI/UX-centric, with lots of animations, tools panels and so. I am looking toward React+Node, but since i dont have any experience with any web framework, may be there are better options out there.
Update: As you already answered, yep, i think backend is less important, but i still need some DB to store content object models. Is it possible with smth like Firebase?
That sounds like a avery complex project from a technical standpoint. You may consider bringing in someone with more frontend experience.
For embedded content, yes, IFrame should be the way to go.
How will users create the content? Are they expected to know how to code or do you plan some kind of visual environment?
it is a visual environment.
Like, typeform.com is for forms, same for other widgets types.
Do you think firebase is enough to store content object models and static resources (like images) and all the logic can be done in client JS app?
Yes, based on your description so far Firebase should do the trick.
Hi Oleksiy,
Very grateful you would post this. I'm not a developer, although I am a product developer (of physical products). However, there's a passion project I've been wanting to pursue for some time now.
I'd like to be able to take an E-book and add a piggy bank to the top right of the page. I want to sponsor people reading a book.For every page (or chapter) someone reads, they would be incrementally paid out for successfully reading. This next part isn't exactly "MVP", but is there a cheap monitoring system to ensure that the person who is reading, is in fact the person who is supposed to be reading it?
Hi,
Are you talking about physical e-book reader device, e.g. Kindle? (In this case, I am afraid, this is way outside my area of expertise).
What do you mean when you say "the person who is reading, is in fact the person who is supposed to be reading it"? What would be the alternative case? Are you worried about the DRM or someone "faking" reading the book?
I was thinking a website, not an e-book reader device.
To put it simply, I am worried that people could bypass the "honor system" and have someone else complete the reading.
If you need a real proof than someone in particular reads the book, I cannot think of many other options except using a webcam to take periodic snapshots. Needless to say, this requires a reader's consent.
Otherwise, maybe an authentication with social network profile (e.g. Facebook) could work - people are not very likely to share that with others.
If you were to use AWS but needed a large number of different static IP addresses but also wanted to minimize cost what would you do?
I was think spin up a number or ec2 nano instances. I at first considered Lambda but you cannot guarantee different ip addresses
Depending on why you need many different static addresses, your approach might be the best option, or you may take a look at some way to associate a static IP with Lambda, like suggested here, for example: https://medium.com/@matthewleak/aws-lambda-functions-with-a-static-ip-89a3ada0b471
Thanks for the link. I would need many multiple ip addresses so I may have to fall back to the many ec2 instances as the Lambda behind a vpc would have a single ip in the example
Your plan seems a way to go then
Hi Oleksiy, I'm planning on building an email marketing software something similar to drip or Active campaign. I have a background in Ruby on Rails and of course JavaScript . I was thinking Rails backend and react front end
It makes total sense to go with the technologies you know. React is still a good first choice for the frontend framework due to its rich ecosystem and solid support. An alternative option to consider for a "serverless" backend could be AWS AppSync, which is based on GraphQL, and is supposed to be working very well with React (disclaimer: I have not built anything real with AppSync yet, as it is a fairly new service, but I would definitely give it a try )
Thanks Olesksiy I was not aware of Amazon AppSync, finally AWS will have something to be an Firebase alternative
Hey! 👋
Thank you for helping us!
I kinda feel stuck now. I'm a frontend developer rocking in React. I'm about to create a tool for businesses to be able to easily build up tech interview rounds and send it to candidates via an email.
I'll need to implement the backend myself. What stack would you suggest for creating something like this?
Hey!
The answer very much depends on the chosen data model, e.g. what exactly constitutes a "tech interview round".
Can each interview be completely self-contained and distributed as a HTML/Javascript mini app? How rich does the user interface need to be? How do you plan to record and store the answers?
Based on what I could infer/guess, you might want to consider it as two independent problems: a self-contained interview "package" and a builder system for such packages. It may make sense to use completely different technologies/stacks for them.
Thanks for this post! It's easy to feel alone when working on the tech stack for a new product idea and I'd love to really just hear your opinions and possibly get some validation on the ideas I've come up with.
The product I'm working on will compete in the CRM space and will contain records such as contacts, organizations, leads, deals, etc. The rate at which records will be inserted could potentially out-weigh the rate at which these records need to be read. Also, there is a need for flexible segmentation of said records via reporting / analytics. The structure of the data has more of a document type format and does not represent transactional data.
For this product, I'm taking into consideration the number of people out there that know how to write code in the particular tech stack as well as the potential for scalability if and when this product gets traction.
On the server at launch, I'll be running nginx to handle routing, load balancing, caching and SSL. The backend of the platform will be written as an API in Node JS. The content/blogging portion of the site will be written using Wordpress and PHP (heavily cached because the content won't change much). Wordpress will be using MySQL, but for the node portion, MongoDB seems to fit the bill because of it's aggregation framework (for it's segmentation and ease of scaling).
The frontend of the app/dashboard will be decoupled and written in VueJS. This will be served via a distributed CDN once the traffic justifies it. Potentially down the road, I can also use this code embedded into Cordova to provide an actual mobile application.
I'd love to hear your thoughts and to discuss some of the potential pitfalls of this stack if you've used these components.
Thanks!
Happy to help in any way I can!
Honestly, a lot of what you're describing goes against what wold be my first choices, so let's try to unwrap it.
First, what is supposed to be your competitive advantage? Since the product is in the CRM space, I assume it's going to be UI/UX (possibly sales/marketing too, but that's beyond the scope). It is highly unlikely that the main source of innovation is going to be on the backend side (most CRMs have more or less the same object model anyway), so the main focus should be on the frontend code and the UI.
So what I would ask first is "how far I can get with writing nothing but frontend code?". Nowadays you can actually get quite far, using something like AWS AppSync or Firebase. Or you could invest minimal effort and use DynamboDB, which is quite similar to MongoDB. In general, I suggest you rely on the cloud services as much as possible and avoid spending time and effort on anything that is not your main differentiator (presumably UX and sales). Things such as server and database management, scaling, load balancing etc. shouldn't be your concern at the moment. Don't underestimate the complexity of those tasks, and definitely don't overestimate the savings from not paying the cloud providers (those are usually negative). You can always add your own managed webserver later, if it is truly justified.
Similarly, why do you need to think of MySQL database for the content/blogging portion? Are you going to manage the Wordpress installation yourself? Is it truly necessary or maybe you can get away with something like Netlify for now?
Oleksiy, can you help me understand better how I might get rid of a backend and just use, say, Firebase?
It just seems like there's so much central backend logic that matters more and more as a project moves towards a 1.0. I was a huge fan of Parse for its ability to let you write some backend logic somewhat minimally. But there's nothing like that anymore, is there?
Have you considered Firebase Cloud Functions?
They are powerful and flexible enough to support many scenarios. They also have limits (for instance, as of recently it was impossible to execute a scheduled/periodic task, although this may have changed now). Also, the developer experience is far from ideal with testing and debugging still.
Of course, being "serverless" or "backend-less" is not and end in itself. If your business logic is heavy enough, it could actually prove more burdensome to maintain than a more traditional approach -the overhead will just have a different nature - shoehorning your app logic into a more limited framework, as opposed to worrying about managing servers. It all comes down to your business priorities and core competencies at the end, and where does most of the products value reside.
Thanks, setting up my second firebase app now and trying to learn the ropes, hoping Cloud Functions will be useful!
Thanks for the response.
You definitely make some good points. Honestly, I don't have experience with firebase, but based on what I do know, that would eliminate the need for a backend altogether. That could be a good route. How would I provide an open API for devs down the road if I went this way though? Maybe I shouldn't care about this right now. How do you handle business logic?
As to the blogging aspect. I've considered using static site generators, but based on what I've read the management of SEO, social sharing, and form submissions is a pain. SEO and content writing seems to be one of the best traction channels for bootstrapping, so I want to make sure I can do this portion right. Plus, my day job involves managing 100s of wordpress installations, so it's 2nd nature to me.
Makes sense about Wordpress. I'd still stick with the cloud/managed version though. As far as I understand, it offer much of the same flexibility.
Do you mean API as a way to extend your product and/or give 3d party access to it (e.g. public REST API)? I definitely wouldn't worry about this now, unless you believe it's going to be one of the major selling points. If that's the case, probably as much thought and iterations needs to go into the API design as into the UI design. You still can get away with little custom backend code, by leveraging, say, a combination of AWS API Gateway and DynamoDB at the beginning. More often than not, a CRM-like product is mostly about doing CRUD operations with the DB.
Nothing in this model prevents you from adding layers of business logic gradually as needed. AWS Lambda may be a good starting point. When it comes to basic things like access control and permissions, solution like Firebase usually have built-in API for that.
I obviously have to make a lot of assumptions about your business goals. This being IndieHackers, I assume you want to proceed in a "lean startup" manner, and that you don't really know if you have the market yet. Then the same principles apply to the tech stack as to everything else: do the minimal amount of work to be able to test your hypotheses, and retain the ability to iterate fast.
My stack will be Elixir for obvious reasons, but I'd love to ask you about what level of server-side event tracking you would do if any for a newly launching membership site with both paid and free sections. Right now, the audience numbers only in the hundreds, but growing ~ 40%/month and I am doing no event tracking locally.
Is recording a few key events externally on Keen.io enough or will I regret not doing it on my own server as well? I've been at two startups as an employee where they hit a wall in what Mixpanel could reasonably do.
"Server-side event tracking" part is a bit hard to answer without knowing your server architecture. I.e. does you server receives a stream of events from the client? (In which case, why wouldn't you capture them all?) Or do you have to extract/infer business events e.g. from REST calls?
General advice is, of course, capture as much (non-PII) data as you can afford.
There is nothing wrong with capturing events with the external service, but you have to make sure you have a process in place to easily extract and store the data somewhere on your end. As a bonus, services like Keen.io may help you to enforce a consistent event nomenclature - something many people struggle with early on.
Thanks for your feedback. It's a fairly normal Phoenix app with REST calls, perhaps migrating to GraphQL in a few months, and websockets for chat. I can't justify spending to log anything that bumps me out of free-tier plans.
Hiii man.
really appreciate you doing this. I know the basics of Java, C and python. I have built some really small applications but nothing fancy. I want to learn coding and become better at it.
So I want to develop a program this year on my own. so that by the time i finish it I will learn somethign about every aspect of development.
Anyways, here is what I had in mind ( and, mind you, i am not making this to sell. I want to make something fun, and learn through it)
I want to build an app like splitwise( have you heard of it?). It should let people create groups, enter the amount everyone spent on anything followed by how it should be divided amongst the members of the groups. you can also add notes with any transaction. Very useful if you are going out on a road trip or any trip with any friends in general.
There are many apps out there that do this.
One thing i want to add to it is a feature that lets you scan a physical copy of a bill, digitise it using OCR or something, and lets every user on the group select( tap on) whatever they ate. The bill gets divided accordingly.
Questions: How do I start building this? What language/ platform should i use? Can you name all the services/ skills I need to learn to make something like this? How much time do you think I should set a target for? Any tips/ ideas?
How do i tackle the planning part of the application? should i read about various software development processes?
p.s. I am doing a 3 month coding challenge, where i am building something useful for me every week and will start working on the abope said project after this.
Hi,
In your case, there is an obvious core technology part that could make or break your product, namely the OCR and visualization. So I would start with researching it and building a prototype that can do just that: scan a given bill and render the extracted data. Most cloud providers nowadays have some kind of solution for OCR (see AWS, Google Could, Azure). Since this is the core part, I would probably evaluate different providers on your real-world use-cases, and then make a choice depending on the results. Eventually you might want to do the OCR part completely on-device, but I wouldn't go there until you have at least some market validation. (I know you're doing it for fun, but I imagine you'd still want the app to be useful to someone. So "market validation" in this case could simply mean "approval by your friends")
You don't need to learn about software development process yet - those exist mostly to organize team work and to properly manage expectations, which is not relevant in the case of a one-man team. Despite the name, software development methodologies provide little answers as to how to build actual software. I think product/business development approach is much more important in your case (e.g. "lean startup"), so just write enough code to test your next hypothesis.
What would you use to drip feed content?
To an audience that reads
And wants to lose weight
But there preference is a branded app
Thanks for taking the time out to do this!
I'm a bit late to the party, but I'm looking at building a data aggregator that pulls in information from web APIs and local devices.
Mainly: Should I pull the data from the web APIs on-device and then push it to my servers? Or should I routinely access the web APIs for every customer on the cloud? I am concerned that I'm going to quickly hit rate limits if I need to ping someone's backend for a large number of customers, especially if many of them are inactive.
Secondly, should I build with Google Cloud or AWS? I like that AWS is a more established player, but are the differences so significant that they're preferable every time? How does this decision change if I'm trying to use a serverless architecture or machine learning?
Google Cloud vs AWS is pretty much impossible to answer in general. Much depends on your usage scenarios and even personal preferences. Based on my own experience with two systems, my first choice would be AWS, unless I need to use a lot of Google APIs. (But I also like to use Firebase, which is technically a part of Google Cloud, though still a bit odd one)
Regarding the aggregator part, how do you plan to deal with authentication/authorization? Do all API providers you have in mind support the same mechanism (e.g. OAuth)? This can be an important factor to consider. Rate limits also tend to differ wildly between providers, including in how they are applied, e.g. by IP vs by Client ID (in the latter case, you'd hit them regardless).
It's pretty much all going to be OAuth (AFAIK), but I also want to upload user-inputted data and access some stuff from Apple Health's native API.
True about rate limits — which means I'm probably better off keeping the stack in the cloud.
The tech stack is the easy part, the hard part is figuring out how to monetize it
I want to start looking into online database connected apps, what's your go to stack for small, mid and large scale apps?
That's very generic. Like I tried to explain elsewhere in the comments, I consider the stack/architecture to be a solution to a specific business problem, so I don't really have "go to stacks".
A scale of the app is also something you almost never truly know in advance, so my go to approach would be to increase the complexity progressively, starting with a minimum required to test your hypotheses, and avoid dealing with incidental complexity (e.g. managing web servers when some cloud-based alternative works just fine)
I am really happy someone asking this question.
If I get busy for a month and comeback into tech world I see a new framework everyone talking about which I never even hard before. Some of these new frameworks have a lifetime of a year or two. I feel some of the contributes to a framework gets upset over a minor issue and start a new framework with a new jack-and-jill name to it. That's why front-end frameworks are growing like India's population.
JavaScript