Reflect

We have a free tier at Reflect. It's not overly generous but we feel it's fair: it lets you run 25 tests per month which is enough to run 3 or 4 tests every single week. For many websites, those 4 tests can cover login, registration and two app-specific user flows. The free tier helps us improve our product as well as turn into paid subscriptions.

Well, we just hit a special milestone of converting someone from our free tier within 3 days of them signing up. Our time-to-close is typically weeks or longer, so this represents a real improvement. The best part? The user has since come into the app and created 18 tests in 30 minutes. The equivalent tests in Selenium or Cypress would take days or weeks even. We're fired up :)

This felt like a big number so I figured I'd share. Over the weekend we surpassed 80,000 tests executed via our platform. We capture full-video recordings and console/network logs for each test run, so our S3 bill has gone up accordingly as well. :)

It's been pretty cool to watch this number accelerate over the past few months. We've also seen a spike in created tests since we launched our free tier back in April. In May there was almost an order of magnitude more tests created compared to February.

Onward to 100,000!

It's been more than a year in the making, and many nights and weekends before leaving to go full-time, but we finally hit our goal of launching Reflect on Product Hunt:

https://www.producthunt.com/posts/reflect-4

This is kind of crazy for us because we've talked for so long about all these different features we saw as blockers for a PH launch. And we've now built and fixed (or ignored ;) ) all of those issues. What a journey, and especially with the pandemic hitting now. It's been an experience.

Thanks to all of you for your continued support. We're very proud of how far we've come and excited for the journey forward.

It's been a minute since we've posted a milestone or update here on IH. The truth is we've been working hard (and learning) on all aspects of the business, from marketing/outreach to fixing esoteric bugs in random corners of the app. We got some press from Abstracta (https://abstracta.us/blog/tools/reflect-review-the-new-automated-web-testing-tool/) and featured on Starter Story (https://www.starterstory.com/automated-software-for-testing-websites) as well.

The good news for all of you is that one of our improvements was tossing out our Free Trial in exchange for a Free Tier. So, any ol' visitor can sign up on https://reflect.run with just an email address and NO credit card, and start creating and running tests. The Free Tier gives you 50 free test runs a month, which you could use with our built-in scheduler to kick off one test a day, or 10 tests a week!

Why this switch? Well, our confidence is growing in our test recorder. The bugs are getting weirder and weirder and so we think any user who legitimately uses Reflect to test their site is going to have a good experience. We believe our product delivers value and in that situation, it makes sense to get as many users as possible.

So, do you have a website or webapp that you would like to test? Sign up for a free Reflect account at https://reflect.run and create your first test. AND, as a special incentive for Indie Hackers, sign up and then drop a comment on this post with the name of your Reflect account and a short plain-text description of a test that you want created, and WE'LL create the test for you in your Reflect account.

We have built a lot of the core features now for our automated website/webapp testing product and lately we've, at least internally, felt like we're right in the ballpark of feature parity with existing automated testing solutions (and maybe even better in some areas :) ).

Over the course of the 4 months we've been on Reflect full-time, we've oscillated between marketing/outreach/sales and product development. We never go exclusively on one thing, obviously, but we'll have weeks where we shift focus between the two. After finishing Test Editing over the holidays, we're shifting into a period focused on outreach and sales.

The milestone for today is that in our latest batch of demos, our tech is shining and potential customers are noticing. This is an amazing feeling. This isn't a project we built to spec like we did years ago in school where if it passes the auto-checker you get an A and feel good about yourself. This is tech we just built to our needs and now it has apparent value.

First-time viewers/users are impressed with the tech and the conversations seem to be changing in nature from skepticism, or polite detachment, in the past to real critical thinking about how they can use this tech at their company. It's kind of become a non-question that this tech can be of value for automating their manual regression tests. So, while we're still figuring out the sales process for this industry and product, the tech has value.

It's not so much a huge rah-rah moment, as much as it is quiet validation of this vision and our consistent effort. Slow and steady wins the race. Just keep swinging that axe and eventually the tree will fall.

We've had customers for 4 months and haven't lost any of them! It'll happen one day, for sure, but we're proud that we've been able to retain all of our customers so far.

We continue to pursue a dual approach to sales of targeting both enterprise and self-serve customers, and we currently have customers in both tiers. Given our early (minor) success with both, we're happy to keep selling to both.

We spent the last few weeks working on Test Editing and it's almost ready for prime-time. So, onward and upward in 2020.

We've been working on our automated testing framework for over a year now and the product has grown tremendously in so many facets. One area that we intentionally ignored until now is the concept of editing a recorded test. It's always technically correct to re-record the test, even if it's a pain. So, we deferred first-classing the Test Editing feature until we had a concrete grasp on the requirements of the problem and the trade-offs in the solutions.

We knew we needed to support editing in some capacity. Even in situations where re-recording the test is actually faster, we (as users ourselves) STILL felt instinctually that we would rather edit than record. Why is that? Maybe it's due to fear of introducing new behavior in the re-recorded version of the test.

In any event, we feel we have the information we need to proceed. So, we jammed on a couple of different designs over the last week and we have put our shoulders to the wheel and begun the work to build it out. We're ecstatic about the design and the build-out is exciting. We're sharing sneak previews with a few of our clients next week. Hopefully we'll get some great feedback and notch a "win" for us as we head into Christmas and the rest of the holidays.

We've hit 5K+ test runs at Reflect, and the rate of daily test runs is increasing. Many of these test runs use our scheduling feature which kicks off tests automatically, but we've seen an uptick in user-initiated test runs. API test runs are just beginning to show up as the first customers integrate it into their CI/CD pipelines.

It's a fact. If you want to sell to engineers or developers, you have to give them flexibility and control in how they use your tool because we (developers) love freedom. We just rolled out support for executing your tests via API. This was probably our most asked-for feature and we're happy to deliver it.

As with all new features, we need to work out a few of the details to make it the most useful that it can be, but it's cool that even just the most basic functionality of kicking off tests via an HTTP request feels like a huge hammer that a developer can wield.

We continue to improve our prioritization, as it's a crucial skill for us. We have a constant stream of product feedback from initial users, but we need to balance product improvements with our growth efforts (marketing, outreach, relationship-building). We don't want to optimize for local maxima and let the pipeline dry up, but we also don't want to capture a massive audience without a clear, self-evident value proposition within the product.

What we are trying to home in on is the key subset of features that users really rely on and, ultimately, use as their implicit litmus test for whether the product is valuable or not. As that subset emerges, it will make it easier to classify product feedback into important, or not. That, in turn, allows us to focus on growth initiatives with confidence that we won't lose users due to poor quality as long as that key subset continues to perform as expected.