How to price a service that saves hundreds of engineering hours?
I've built a Cypress test runner that completes all tests (no matter how many you've got) in less than a minute. In contrast, based on my experience, a medium-size Cypress test suite takes about 15 minutes to complete.
The only catch is that individual tests have to be under 30 seconds.
The idea came out of frustration running Cypress in my previous company, where it was taking 10 minutes+ to run all Cypress tests while parallelizing across 30 VMs and costing us in excess of USD 2k/month.
In order to achieve this, I have effectively built a new Cypress test runner from the ground up. It understands Cypress syntax, but otherwise have nothing in common with how Cypress works. The way it achieves this performance is by splitting each spec into individual tests and starting all of the tests at once, i.e. if you have 10 specs with 5 tests each, this program will start 50 VMs to run your tests.
I have two companies trialing this at the moment and the feedback has been incredibly positive, saying that it is saving hundreds of engineering hours.
I am trying to establish how to price this. The challenge is that this model is profitable at scale, but losing money if there is not high density of clients. This is because it costs me USD ~0.15 to run a VM for 1 hour and I need to spin up enough VMs to complete all tests, and I am charged in increments of an hour.
My thinking is to charge 2 cents per a test-minute, i.e. Using previous example of 10 specs with 5 tests each, it would cost USD 1 to run all tests once. If you run integration tests 100 times per day, that's USD 100/day.
This may sound much. However, if prior to using this you were waiting 15 minutes to run all tests, that is 14 minutes saved. If avg. engineer in your company is earning USD 60/hour, you are saving USD 14 by having engineers get immediate results rather than waiting for them. If positioned that way, it doesn't sound expensive.
I am currently targeting companies with 30-50 engineers (existing customers are series A and series B companies). At this size, they don't have crazy amount of tests, so I can deliver on the 1 minute promise, and they care a lot about moving fast.
What sounds reasonable?
I'd start with a relatively high price point, as it's unlikely you would get it right on first go. Then the question becomes whether you want to start with a lower price than it should be, or a higher price than it should be. I'd vote for a higher price if you have strong conviction about this and gradually come down depending on the data you start collecting.
I would definitely start w/ a freemium tier if possible. I think users will need to see the value before actually committing. Other than that, your pricing model sounds reasonable.
Ya this is tricky, because if its automated as a part of your CI / CD stack then it won't be directly saving you're engineers money, but 15 minutes is also not long enough to start a new task and switch context, so the engineer probably isn't being that productive during this time while they're waiting. Side note, could engineers utilize this task runner locally as well or no?
So I agree with @pdyc that you should probably price based on the infrastructure savings, but still market it as saving engineers time and helping them finish tasks quicker etc. etc.
you can price by cost or price by value. Conventional wisdom is to price by value.
You mentioned your product has limitation of 30 seconds per test how do you plan to overcome objections on this?
You are not saving engineers time because engineer is not working on test(its already automated). You are saving time spent on infrastructure and providing testing feedback early so while it has value your 60$ calculation doesn't translates to it.
Ideally you should price maximum amount your customer is willing to pay. So you should talk to multiple companies and ask them what they are willing to pay that would determine how much they value the product and you can decide on your price accordingly. If you have any competeitors you can look at their price as well and in that case you can price your product slightly cheaper or at the same price since your product is having unique proposition of saving time compared to your competeitors.
Heads up that there is a great discussion on the subject in HN https://news.ycombinator.com/item?id=32452905 esp. around whether this actually saves engineering time or not.