Developers February 12, 2020

On development stacks and project structures

Josh Sherman @joshtronic

Feel like I'm living in the movie Ground Hog's Day as I keep running into the same topics over and over this week in regard to picking the "right" or "best" dev stack and the "right" or "best" way to structure an application.

I've chimed in on a few of these, usually stating the same thing. Thought maybe a quick brain dump would be a good thing.

Fact is, it doesn't freaking matter. Your customers don't care if you're running on X, they care if your service is fast and has good up time. Over time, they will care about how fast you're fixing bugs and shipping new features.

There's merit in trying to build a perfect project with a clean structure that you /think/ will be maintainable long term. But it's often all for naught because circumstances change that will probably render your current decisions as bad ones.

Technologies change. Project structure ideologies change. Your product will change.

So why spend so much time consuming yourself with these moot things?

The only thing that truly matters is your ability to iterate quickly.

The language you can quickly iterate in is often the language you're the most fluent in. Same deal with project structure. Put files where you will remember where they are.

Not to be a defeatist, but good chance many of us won't ever get to the point we're hiring developers, so solid chance all of the bad code we write today will NEVER impact anybody but ourselves.

Same holds true for hosting. If you're capable with managing a Linux server, go with a VPS. Got a background in containers, do it that way. Which company has the most up time? None of them, 100% up time is a myth and your ability to build a distributed infrastructure is your best bet, but usually completely unnecessary for a long time in a project's early days.

What I haven't seen people asking about, and what I would argue is significantly more important than your server stack is in regard to testing and CI/CD, the DevOps stuff that gets undervalued a ton, especially by folks that are new to development.

Things like an automatic testing pipeline that deploys to your server automatically and takes care of any database updates is a ton more important than if you should use Laraval or Express or Brainfuck.

A solid CI/CD pipeline will help catch bugs, if you're automatically linting and testing (you have to write the tests, this isn't magic). Catching bugs automatically reduces your cognitive load and helps ensure bugs NEVER MAKE IT TO PRODUCTION, which is something your potential customers are big fans of.

Automatic deployments fall in the same vein. If you're doing it automatically, you're reducing the cognitive load of remembering how you did that one thing that one time. Because things are consistent and automatic, you can ship features faster. Again, this is something your customers care about.

That's not to say I don't have opinions on languages and hosting companies and how code should be structured. After two decades as a software engineer, you bet your ass I do.

With that, in the same realm as automated testing, using a strongly typed language in your project will help cut down on errors in your code, and also reduce cognitive load. Bugs happen, so any time you can reduce them automatically is a huge win, REGARDLESS OF LANGUAGE.

Guess what I'm really trying to say in this rant is that there's no silver bullets, just shipping.

That all said, none of this even matters if you haven't properly validated your idea ;)

  1. 3

    Haha - Most other developers will probably fall down in shock if they looked at our development stack. Yes, we use Ruby, but we don't use Rails(!). We actually still use jQuery in our code. No SPA's, and not a shred of React or Angular anywhere (oh, OK, we do use Vue.js on one section, but that is an internal dashboard for our team).

    But hey - my SaaS is making us money now, and in 4 years not a single customer has come to me and said "Oh, I wished the front end was in React", or "I wish you used Node.js in the back end"... 😆

    1. 2

      Nothing wrong with any of that really. I still run stuff that's jQuery and PHP, no shame in generating revenue and not getting weighed down by technical decisions!

  2. 3

    I mostly agree. People worry way too much at the beginning and I totally agree that the speed of iteration is the most important thing.

    In general, doing what you already know is going to be faster. If you don't know any stack, then learning the one your friend can help you with is the next best bet.

    That said, I think there is a massive benefit in using the highest level language and highest level framework you can. In 2012, I knew Objective C far better than I knew JavaScript, but it would have been a very slow process to write a web server in Obj C compared to writing one in Node. A few years later I had the same experience with Rails (which I was just learning) vs Node/Express (which I knew well).

    On the strictness / looseness of a setup, it really depends a lot on what you're doing. If it's prototyping, the last thing I want to do is rigidly TDD everything or fight with a pedantic type checker. On the other hand, as projects get more complex or if I'm working with a large team, then I appreciate the stricter environment.

    CI/CD depends, too. I set it up immediately on all my projects these days, but I've also had several acquaintances and one very good friend who achieved more far more financial success than I have with a "drag updated PHP files into filezilla and overwrite what's on the production server" method of deployment. It wasn't cool. Sometimes it lead to errors and stress. But it got the job done and didn't require much background knowledge to do.

    The final thing to reiterate is people are worrying WAY too much about scaling and project architecture. 99% of the time you won't get than many users and if you make something good enough to get traction, you'll likely rewrite your code base multiple times. My previous startup and my last three jobs before that involved rewrites into a new language, too! No matter how perfectly you plan, your first MVP won't be what you settle on. Your needs, your customers and your entire business may be radically different in 5 years.

    1. 2

      Agree or GTFO... loljk!

      That's an interesting point regarding language level. Most of my career has been on the web, with some of the baby steps including PERL which at the time doing webstuff with SUCKED, hence moving over to PHP and Classic ASP. So yeah, there's still some "right tool" for the job there in terms of language.

      Funny you mention the filezilla success story there. This rant was mostly just that, just wanted to get some shit out of my head, and also spark some healthy dialog outside of just answering the same 3 questions in the developers group all day.

      That said, I've had my share of success stories that don't correlate to unit test coverage or automatic deployment so I know they aren't mutually exclusive. One of my more successful projects (which has netted over $320k) still gets deployed by way of logging out to a server, and pulling down the latest code from the repository (hypocrite much, josh? lol)

      Very true, regarding users / scale. I mentioned folks code never being seen by another engineer but this is probably the most important take away, solid chance nobody's ever going to see your project, so why worry about the day when you have a bajillion users? By the time you get to that point, in theory your revenue would be so much and sustained that you'll have a team of experts handling the day to day while you're off living that yacht life!

      Side note, in true IH fashion, this rant has helped me validate that CI/CD is considered hard for folks and you can expect my educational DevOps web course to be available on DevOpsAfterDark.com and other major e-learning platforms in the near future (only mostly joking ;)

      1. 2

        Gitlab CI/CD FTW!

  3. 3

    Personally, I used to believe you should use something like Node.js, Python, or Ruby to get something out the door quickly, but I found it was hard to maintain and there would be little bugs that would slow down development in the long run.

    A language like Go, Crystal, or OCaml which are barely more verbose than dynamic language but prevent lots of bugs at compile time is my choice now. I'm using Crystal on my current side project.

    using a strongly typed language in your project will help cut down on errors in your code, and also reduce cognitive load

    Exactly. I would only use dynamic for a PoC but switch to a compiled language for 1.0

    1. 2

      Oh hey, I found the other Crystal user on IH. :)

      1. 1

        But there can be only one...

    2. 1

      As an experienced developer, Python works just fine for me, because I know what to look out for. But if you're specifically talking about inexperienced developers (or soon-to-be self-taught developers) who are looking for which language(s) to learn for developing their MVP, then you're spot on.

    3. 1

      I use Crystal for a few small side projects, but I find it is really not as mature for 'serious' products. (example - https://devan.codes/blog/2017/10/28/racing-along-building-a-telemetry-system-using-crystal-rethinkdb)

      As the ecosystem grows and Shards are as plentiful and flexible as Ruby Gems, then I think it will be a great contender!

    4. 1

      Big fan of Node.js/JavaScript all of the things these days, with some active projects still living in PHP / SSR land, but this little rantdump wasn't about that :P

      I think what really kills me is that these questions are posted by folks with little to no development experience. Strikes me as folks looking for some magical way to go from 0 to 100 MPH without putting in the work.

      Didn't mention it in the rant, but I really do think people should decouple learning how to develop from learning how to build and run a business. A short time "sharpening the ax" to learn how to put together a todo app or two will save a ton of time when it's time actually build the product.

      If nothing else, especially for the stubborn "do it all types" taking the extra time to validate, explore existing options, and attempting to solve the problem with little to no code will result in being able to focus on building the right thing when the time comes to start throwing code at the problem.

      1. 2

        I think what really kills me is that these questions are posted by folks with little to no development experience. Strikes me as folks looking for some magical way to go from 0 to 100 MPH without putting in the work.

        It reminds me of language learners who spend months reading blog posts about spaced repetition systems, downloading apps and mapping out strategies to learn Japanese but never actually spend any time listening to Japanese speech, having conversations in Japanese or reading Japanese.

        1. 2

          That's a great comparison / analogy!!

  4. 2

    Thanks for the rant! 😅

    1. 1

      Never a problem... I like to air on the side of candidness when possible ;)

  5. 2

    I'm using Flutter/Firebase/Google Cloud and bitrise for CI/CD. I'm really happy with that choice:

    • I have one code running on both IOS and Android
    • Flutter has plugins for everithing you might think of
    • Firebase has of-the-shelf for everything in backend including analytics, authentication, notification, A/B testing, etc.
    • bitrise allows me to run auto-tests and deploy automatically to Google Play and App Store.
  6. 2

    You could apply the same reasoning to CI/CD pipelines. In theory it is great but in practice it is extremely hard to do well and maintain, which explains why DevOps are paid so well. You can push a product to customers without a single test. I agree it is better to think about these things early but it shouldn’t be one of your priority at the beginning.

    As per tech, something I have learnt over the years is to always bet on JavaScript/Nodejs. It is by far the biggest development community which means the best place to find tools, documentation, help and people to hire.

    1. 2

      Yeah, they aren't the easiest thing in the world, but you can get extremely far with a small amount of shell scripting and using services like GitLab that are owning the end-to-end solution (VCS -> Runners that can build, test, lint, and even deploy).

      Honestly, not sure why folks are even asking about hosting when they could use something like Heroku (yes, fully aware of the price) that makes this kind of stuff so stupid easy.

      I tend to agree about JS, especially with TypeScript in the mix (I'm biased towards strong typing at this point in my life). There's something truly magical about being able to be fluent on both the front and back end with a single language, at least at the start of a project when speed is of the upmost importance.

      Re: pushing code without tests, that one's a line in the sand for me. You certainly can push code to your customers without any tests, but that implies that your customers are your QA department, and that's not fair to them.

      That said, I'm not one of those "100% code coverage zealots". Test the important stuff, test the stuff that keeps breaking. Any time a customer notices something is borked, and you don't write a test to ensure things don't do that again, you're just asking for the same damn bug to crop up again.

      1. 1

        There is no argument about testing vs not. If you have got the bandwidth, you should always implement tests. I personally find it can be rewarding as well.

        Gitlab CI/CD is great indeed. I have been through a couple of outages with their service which forced me to implement my own runners. It's not so hard and even free to spin your own inside a t3.micro on AWS. But then you suddenly face a completely new dimension of issues, far away from developing features.

        And then from Ground Hog's Day, we moved to Alice in Wonderland. 😉

        1. 1

          Ugh, outages in that regard suck, have had my share of "has it really been over an hour and nothing's started yet?" moments. I think the fortunate side of being smaller, even though there is urgency to MOVE FAST, things like a deploy outage isn't the end of the world (of course, if you're building multiple times a night, or the runner not working is blocking a critical fix, that may be a deal breaker).

          I haven't been so dissatisfied with the stock runners that I've implemented my own, but I also run everything locally (as it's still feasible, even with my larger test harnesses). My usual workflow is to code at night, and deploy the next day, after a "fresh eyes" PR/MR review of what I did the night prior, so any evening outages are manageable enough :)

          And through the looking glass we go ;)

  7. 1

    Loved the ending :) Still, it's your side project, allow the geek to live a little and enjoy the development, but don't get buried in tech stuff. ;)

    1. 1

      Totally fair point :) I'm actually dabbling around with a handful of things, so I totally know how that goes, lol

      Should have prefaced this all with a quote from Ron Swanson: "put ketchup on it if you want, I don't care".

      Writing's pretty cathartic for me, so after 2-3 nearly identical responses I was pretty fired up and wanted to brain dump some stuff. Usually this stuff makes it to my blog, but figured good to get some long form contributions over here on IH :)

  8. 1

    Amen 🙏