Create a DevOps library
I do not have many start-up ideas and especially I have never tried (hard) to implement one. After many years of web app development, in the last 5 years, I have been doing AWS infrastructure, Chef, Ansible, all the DevOps stuff.
One of the ideas was to create a Terraform library to create AWS infrastructure, using infrastructure as code, deployment with CI/CD, all the good practices.
GruntWork guys are very famous and good, but I would like to provide something more affordable (they ask for $745 per month) and I would like to start with a serverless data/big data pipeline (Lambda, Kinesis, Glue to convert JSON to Parquet, a data lake, a data warehouse), which what I am doing at the moment at Sainsbury's (FTSE100). Then I would add Lambda and API gateway with DB for a serverless web app.
I would like to know what do you think if you would or someone would pay ~ $800 so have such a library (and optionally ~ $500 to install it). I could put ads in Serverless Status newsletter, blog about Serverless trying to be featured in DevOps Weekly, Serverless Status, HackerNews etc. Any comments, opinions or advice are welcome.
I think the idea have merit. The space is a bit odd though, (not yet defined?) so maybe it’s helpful to really nail down the value add and differentiating factor and be prepared to try a lot of approaches. Would-be consumers for things like this are likely use to open source catalogs (think hashicorp terraform registry, AWS lambda blueprints, chef supermarket etc). Personally my team at my day job forks off hashicorp registry and supermarket to get started on new things as we go along. Having a standard of quality would be great though for those things. (100% test code coverage, pre-linted, usability docs, issues reviewed with 1day, etc etc)
I’ve toyed with an idea like this after seeing gruntworks success. It seems like they bootstrapped mostly on implementation rather than purely access to a catalog though, which is probably harder to scale (lots of grunt work hah). I suppose a catalog of said “certified” or supported modules would be more attractive to some enterprises when they see their engineers grabbing crap from github. That safety net thing. So you would end up selling more support for you modules and cookbooks. I know that my leaders in the past have had us buy redhat licensing purely for that safety net of being able to get support for things, even though we never used it.
It’s interesting though. I’ve been wanting to do something in this space as well and haven’t quite figured things out.
It is very odd and that is the main thing it is putting me off. The increasing amount of modules on the Terraform Registry is another main point. The main issue is that you can pull out as many modules you want, but a lot of pain points are happening with the integration of TF modules, code for containers and lambdas. I have never seen as well designed architecture were adding a new Kafka worker is as simple as adding a directory with the Python/Scala/Java code and it gets deployed. With current Agile methodologies, companies are building 'staff' glued together in a decent way, but it is usually composed by different parts that are painful to use.
My idea is more about a project with most of the moving parts together. Kafka deployed and scalable, not just MSK, configurable to connect directly to EMR/Spark, a place where you put all your consumers and producers, and CI/CD integrated.
The second step would be API Gateway/App Syn with Lambda and databases, CDN and S3. Yes, you made me think about support, I would sell one year support, renewable. As you said very few or nobody used RedHat support, but they made/are making millions!
I am probably not going to have the traction that Gruntwork has, but I have the certainty that I could provide a template of much much better infrastructure than the projects I have worked so far.
Hi Riccardo ! 👋
I'm working on something similar with Revolv, so maybe I'm a little biased, but yes, I think it's a very good idea! 😀
As a freelancer, I was faced with the difficulties for my clients to create a working cloud architecture on platforms like AWS, GCP or Azure.
Can't wait to see how you approached this problem!
👋
Hi Jeremy, thanks for your feedback. Do you have some information about your project, what you were doing, why you stopped?
Yes, for sure!
I'm still working on it. I hope to release it soon.
Revolv aims to automate cloud architecture creation and to provide additional features on top of it.
It's like Heroku or Netlify but on your own cloud provider account.
I will try to charge for the additional features.
👋