Does anyone get frustrated keeping data models in sync?
Over my years as a software developer I've been frustrated with the work required to update database (sql migrations), API (changes to graphql or REST API contract) and front end (Typescript) models whenever I modify or add a field on my data models. I'm thinking about building some tooling to help me with this problem, but I'm curious if anyone else has run into this issue and found any good solutions.
Trending on Indie Hackers
Unpopular opinion: TypeScript and GraphQL makes things overly complicated for indie hackers.
I don't think adding a db column is much work in any of my Rails, JS or Clojure projects :).
If you "hand code" all that stuff I fully agree with you! I know of 2 different tools that lets me change a db column/ field in a model then run a command and..... boom it's all in the GraphQL API, yes everywhere in CRUD including in nested objects. I only touch the backend to add fields and some custom code e.g. a calculated field/ some custom logic on save. Lots of what for REST is custom code in GraphQL is just generic, so I get for free those parts. Then on the frontend side using GraphQL makes it sooo easy and fast. For me GraphQL is one of the best things happened for indie hackers.
Had same problem and start with a CLI tool and end up with a full blow no code visual app builder, have a look at https://www.bappaz.com, a full stack code base generator. Define or change model and get ready made code for front end and backend!
Definitely.
Having mongoose schema, graphql schemas, some rest api, and a SQL schema can make things super complicated.
From a greenfield perspective I love using graphql + codegen. This works super. But we're not often in that happy place :D
I think TypeScript is the future. So having tooling that can generate types from different sources would be amazing. E.g. ORM schemas, Mongoose, graphql to TypeScript. And keeping them in sync. That sounds like a good solution :)
What about android data model with firebase
I was facing the exact same problem and many more in my back end development journey. So we have started building my own no code tool VadeStudio that could enable us to integrate multiple APIs and databases and generate REST and GraphQL APIs.
Here's a product demo of integrating Twitter and GitHub that generates a unified API.
This seems like a problem that’s been solved in Rails for a very long time and one of the many reasons I stick with Rails. What framework are you using?
Wasn’t sure if you’re suggesting having a tool that updates all layers of the stack in one shot, or referring to each layer in turn?
For database migrations you might like to checkout dbmigrate https://github.com/golang-migrate/migrate
This tool is written in Go but doesn’t require you to be using Go for development.
It allows you up and down between schemas by storing your initial database schema and then applying deltas to take you up and down versions of your schema.
For the API, depending on your language you might be able to generate some database layer code from an intermediate representation. For example, in Go there is a package https://sqlc.dev/ that will auto-generate code from SQL statements. I’ve used this in the past and it works well for simple CRUD applications. Personally I prefer to roll my own and have the finer grained control even through, as you pointed out, it requires more work. Although the example I gave uses Go, I’m sure there is an equivalent tool for your language of choice. I just used that as example to demonstrate the methodology.
For the front end, you could consider autogenerating your client SDK if you document your web APIs using Open API Specification (formally called Swagger), although I’m not a fan of this either, but I know a few devs who have used this approach.
Just a few ideas.
I think this is a problem that lies at the heart of software design and has to do with the very basic concept of software: data, logic, and interfaces. It’s a direct measure of good software design. No? I don’t think it can be fully “solved” can it?
In classic developer fashion I created my own stripped down tool for this called mgrt. It works using the concept of revisions, whereby a revision is an arbitrary SQL script that is executed against the database, that can only be applied once. Revisions cannot be undone, only applied one on top of the other. I found detaching migrations from my data layer gives up more flexibility.
That's super cool! How were your migrations coupled to your data layer before you built mgrt? I really like how it's framework / language agnostic.
For the main platform I was developing, I started out using an ORM, that came with migrations out the box. I wasn't a fan of this personally. For one, I'm not really a fan of ORMs, (I find ActiveRecord style ORMs to be a poor fit for Go), and I find most migration tools do a bad job of implementing version control on top of version control (git in this case). I've written in the past my thoughts around ORMs and query building and the advantages of the latter here.
But, from surveying the landscape of migration tools I found most of them provided a poor abstraction over SQL it self. So, I thought why not have a tool that simply executes SQL scripts against a database and keeps track of what has been done.