3
16 Comments

Rewrite your entire codebase or update?

Perhaps this should be called the COST of not staying ontop of updates / upgrades

OR

Throw away your old stinky fish and replace it with sushi...

This is a warning from experience.

Your codebase AGES fast, with the new launch of rails 6 our codebase is starting to smell, we are already on a version that is EOL and suddenly new gems and version management has become an issue as we try to integrate new functionality.

We have a fairly large tool that we had built for a few long-term clients. We now want to productise this and increase the usability and client base.

WE got SCARED of changing things and upgrading because the pain of re-coding for the upgrade FAR outweighed the cost of status-quo and focused on developing existing tickets.

Now in the blink of an eye, our existing codebase is a couple of MAJOR releases away from the latest rails and ruby functionality, and we have to consider do we just scrap our existing codebase and rebuild it in the latest release OR do we step through the upgrade process and do incremental re-writes...

BOTH ARE A NIGHTMARE , if only we had kept the base up to date with releases over time, now it's a structural nightmare and a herculean challenge.

Our codebase is 26 models and huge sets of functionality would appreciate any advice or input from anyone who has gone through the same issues.

https://www.linkedin.com/in/steven-moffat/
https://work-books.uk/

on February 17, 2021
  1. 3

    26 models isn't a lot tbh. We have a 6-7 year old rails app that went from 3.2 to 4 to 5 and now is going to version 6(.1). One good thing that we have is pretty good test coverage, and upgrading would be impossible without it.
    Also because we have a lot of gems (it's a huge project), we are usually waiting for the .1 version to be released before we start thinking about upgrading.

    I did 4>5 as a semi undercover project on the side, we never had 'time' to upgrade since there were new features coming out daily. So I just rebased my branch, upgraded the code and fixed the specs until everything was rails 5 ready. Now another single person is handling the 5 > 6 upgrade in the same fashion (with a bit more transparency this time since the upgrade was officially approved)

    This won't be a short project, but it also doesn't have to be all hands on deck and everyone involved. You just need a single dedicated person that will do the job, and fix all the deprecations and things that come out creeping when you do the upgrade.

    So if you have good test coverage (that you trust), go ahead and do the upgrade, if you don't then you either have to add test coverage beforehand or rewrite the app completely (if you don't have to be backwards compatible, this could be the solution as well)

    1. 1

      Morning Beris,

      You hit the nail on the head, we don't have good trust in our current test suit,

      So we are going with a phased approach,

      1. 1 person is tasked solely with reworking and getting our test suit to date
      2. A slow phased rails upgrade in 2 incremental steps is agreed by all
      3. 1 person is tasked with ensuring working slowly through depreciation issues on a mid term branch
      4. new development is coded for BOTH versions until old is depreciated
        if rails.env.v-x?
        run this
        elsif rails.env.v-y?
        run this
        else
        run the old sushi
        end

      We dont think it will be that difficult as there will only be a few commands to change , just the big architectural stuff will have to be tackled in big blocks

      then depricate the functionality in normal update tickets

      The key to this and to all things is getting the test suit cleaned up which has been on the list of things to do for a year now.....

      PS we don't actually have to be backwards compatible at all and the test suit issue which is what lead us to say, ok let's start again in the first place, but its probably better to chip away and evolve and fix that issue directly.

      1. 1

        Yeah, as someone else mentioned, the full rewrite includes supporting the old app as well, so I'd avoid that one if you have active users on it. Sometimes you absolutely have to change stuff even if you didn't want to (we had to go off paperclip to ActiveStorage to make it possible to migrate from AWS to Azure. Now we can leverage that work in upgrading to rails 6 so no work is lost.

  2. 3

    Hi Steven,
    we are/were at the same position as you.

    At one point, we have found ourselves limited. New features that were well prepared ended up being delivered only as half baked solution, because the technical debt and limitations of the current platform didn't allow it. After trying to build several features like that, we have decided not to try to rewrite bits and pieces and slowly bring it up to speed with most recent codebase/libraries/etc...

    • the codebase started 9 years ago
    • the company itself shifted from various approaches/features/business cases during that period, rendering some of the code irrelevant
    • not all the features are being used, some can be removed

    So we have this large pile of code, which was hard to work with and was outdated (or becoming outdated every day).

    Therefore we have decided to:

    • hire a company to maintain this version, we call it version 1
    • start working on a "new product", version 2, where we take only the features we like and think will still work in this age, re-think business cases, add new features/concepts, etc. Slowly and gradually. We are not making 1:1 version.

    We discussed this a lot and carefully, considered our customers, considered our developers, support team, and everyone in the process. Because you don't want to have everyone frustrated by digging through old things, or supporting bugs that we can't fix (because of the limitation).

    And also the company and people within grew throughout the past years.

    After all, this old codebase is what puts money in our bank and keeps us going. Customers will be still able to use this version 1 for as long as they are willing to pay for it, could be years. And if they will feel that the new version is good for them, they can switch.

    The team is happy, customers are looking forward to seeing where we take it, knowing that there won't be further development on version 1, just bug fixes and maintenance, making sure it is all secure and 24/7 available. :)

    1. 1

      Hi Marek,

      Thanks so much for sharing this, it really helped me to know it's not just me and that there's a middle route and alternative path to a resolution.

      Splitting the client base must have taken a lot of consideration.

      Cheers

      1. 1

        Hi Steven,
        you are welcome, glad it helped.

        And no, it is not just you, we can think of it, that we are just lucky that we can do this and end this frustration. Some are not that lucky and have to work with what they have. We do want to make our life easier, whatever we do. :)

        And as for splitting the client base, for our customers not much will change, quite the opposite I believe. I think they will be happy with the version 2 and what we can offer there.

        This article might also help - it is something you should really think about: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/

        It’s important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don’t even have the same programming team that worked on version one, so you don’t actually have “more experience”. You’re just going to make most of the old mistakes again, and introduce some new problems that weren’t in the original version.

        1. 2

          I remember watching this happen and reading this article years ago when we threw out our ASP servers and moved to Ruby...

          :)

  3. 2

    Sometimes people build and sell tools to covert code bases to newer versions
    Usually the instructions on migrating major versions are kinda clear if you go one by one
    Take your most experienced ppl and let them do it or get some external resources that are experienced in exactly that. It's commonly mass replaces but it's kinda hard to test edge cases unless your already heavily invested in automated testing

    1. 2

      Great input,

      Yeah mabey incremental move codebase forward or outsource is the way.

      Thinking we will task one coder this and nothing but this and give him free rein over the entire codebase.

  4. 1

    The big rewrites again, again 😂

    I have been in this position for a couple times. These are lesson learned.

    • Rewriting the codebase of existing product will take same almost same energy as your current product if, not more. Moral of lesson, do the rewrite of a new product, provide a long term support for existing customers who do not want to move to the new product.
    • In case you have no intention to write an upgraded versions, you have two routes: top-to-bottom approach(big-bang aka big rewrite), or bottom-up(extract and rewrite small chunks ~ as independent modules, or abstract them via proxied means such as microservices or compatible modules)
    • The top-to-bottom: You will basically to stop the current product iterations, and rewrite the whole application from ground up again. If you do not have a big team/nor money to oursource that work, you will be in big trouble.
    • The bottom-up strategy: You will basically extract some features/sections and migrate them to a new platform while fixing bugs/and adding most critical new feature requests. For instance, your database models(26 models) can be extracted in one big chunk and provisioned via a REST API. An additional layer (Service layer) can abstract the way you bring your data to the controllers. Once that part becomes stable, you can move controllers in another bulk.

    Next time, make sure updating to latest stable versions is automated(patches), plan for minor releases upgrades (like even stable releases ~ or once a quarter). At least You will not have to rewrite the whole application when Rails 7 or 8 hits the market.

    All the best.

    PS: I have written some articles about rewrites before here: https://medium.com/hoopeez/the-deep-dive-migration-from-angular-to-react-ea5a807e95eb

    1. 2

      Thanks dude, good points and all taken on board,

  5. 1

    Hey Steven, Great homonym ;)

    I'd look at hybrid solutions that would allow you to replace your whole codebase EVENTUALLY but doing it step by step

    1. 1

      I knew new GNU codebase was the way forward :)

      I think this is the most likely step, we will have to postpone features that need the updated frameworks and tippy-toe through the existing code.

      A nod of the cap to you sir.

  6. 1

    Can I vote neither? Just don't touch your old pieces of code unless you absolutely need to. And then make additive changes or microservices. Create "proxies" or layers that lay on top of your application.
    If you're using an mvc pattern instead of apis, you can create new "applications" and open them up on different ports and different routes, then use nginx to route appropriately.

    I guess what I'm saying is that unless it's broke, don't fix it. You're not in a 'code debt' hole, but an 'infrastructure debt' hole.

    I've seen companies make great products with legacy systems (11 year old php code) that they just don't touch unless absolutely necessary.

    1. 1

      Cheers Logan,

      Yeah Ill add this option to the poll for you :)

      I think its going to be a mixed solution, was really interested in @hatkyinc solution and outsource to a specialist to step code forward, whilst we gingerly keep pushing around him.

      Might set one lone wolf into the codebase with just this mission....

      OR if anyone knows the guy for the task ?

      1. 1

        Note : You appears you cant alter a vote after someones voted, which makes sense