Developers October 23, 2020

How do you deal with the fear of bugs when real money is on the line?

Justin @Harrjm

I'm currently integrating Stripe subscriptions into my SaaS product. I'm not far in but it is already making me nervous because it seems like there are so many points of failure. For example:

  1. If the same user makes it to Stripe Checkout twice and completes the payment form both times, they'll now have TWO active monthly subscriptions (or more if they keep going) for the same product.

  2. If I miss a webhook cancelling a subscription I could be charging a customer indefinitely without them knowing.

  3. If I accidentally set the wrong Price ID in my .env variables I could be signing up customers under the wrong pricing tier.

  4. If I sign up a customer to Stripe but miss the webhook that they paid successfully I could be charging them without providing the service.

The list goes on and on, and this is while using their hosted Checkout and Portal pages that are supposed to make things super simple. Obviously I plan on building ways to try to prevent those issues, and I plan on doing a lot of testing but there just seems to be so many moving parts. This is the kind of solution that would normally make me nervous but since it involves real money it really just leaves me with a bad feeling.

  1. 4

    Log everything. Receiving a web hook? Log it. Sending a request to stripe? Log the request and response.

    In any system with a lot of moving parts, you need to log things because inevitably things will go wrong, and usually in areas you didn't think about, like at the load balancer, or misconfigured currency conversion, or a user with a name or address that doesn't work with your system.

    Log errors, log successes, log everything.

    1. 1

      That’s a good call! What’s your favorite logging service for something like this? Paper trail?

      1. 1

        If you're on AWS, you can log some things into cloudwatch. All of the financial transactions should always go into a proper database though, not just log files.

  2. 3

    In addition to testing a lot, there are a few other "monitoring" things you could put in place for additional confidence.

    1. You could run a periodic task to fetch all Subscriptions on your Stripe account and ensure that all active Subscriptions are associated with active users in your database.
    2. Missing a webhook notification once is actually okay because webhooks are retried for upto 3 days with exponential backoff. One thing you could do here, is have a periodic task that runs daily and emails you if a day goes by where you didn't receive any webhook notifications.
    3. The incorrect price ID should be apparent the first time someone pays the incorrect amount, generally if a customer over pays, they'll let you know. When first configuring the flow, you're likely going to excitedly watch those first few payments come in.
    4. This is another one you could solve with a periodic task to fetch all paid invoices and confirm they are related to active subscriptions in your database, sorta adding an additional "backup" layer for confirmation and confidence that things are working as expected.

    Generally, there are ways to setup monitoring for these types of things to help gain confidence that things are working as expected and alert you when something breaks. It's like an additional layer beyond automated tests -- for observability. I'd say that typically folks yolo stuff into prod when just starting out, then they layer this on after missing something or billing incorrectly after X months or years.

    1. 3

      This is awesome advice, I'll definitely be watching intently at first and I've already been toying with fetching all subscriptions so it should be simple to build that into a recurring task to help me out.

      I'm definitely the opposite of YOLO'ing everything into prod, but that's why most of my stuff never makes it out to the world. So I'm trying to find a nice middleground here.

      1. 1

        I should re-emphasize the importance of monitoring. There's a notion that you are always testing in prod because even a staging environment that's identical in infrastructure is not identical in traffic. There's always going to be something even the best tested system is going to miss. Ensuring good monitoring and having recovery procedures in place is as important as good testing.

        One thing you can try to alleviate some anxiety is to launch payments, but hide the option and only enable it for certain users such as yourself or some friends. Then you can try out some of these scenarios, force some kind of error (such as a bad env variable), and make sure your alerts trigger. Practicing a recovery process for these errors will help you feel more confident in handling other errors.

  3. 2

    You test everything, and then hope for the best.

    All of these things happen. Your customer contacts you for support, you fix the issue, and move on. None are disqualifying.

  4. 2

    I consider myself a bit of an expert on stripe. Let me address your concerns

    Q. If the same user makes it to Stripe Checkout twice and completes the payment form both times, they'll now have TWO active monthly subscriptions (or more if they keep going) for the same product.
    A. Who would do that? and if they do - it's their problem and not yours. If I walk into a sports shop and buy a pair of shoes, and then go back later and buy the same pair of shoes again, I have two pairs of shoes and I've been charged twice. I did that, not the shop.

    Q.If I miss a webhook cancelling a subscription I could be charging a customer indefinitely without them knowing.
    A. No this is a misunderstanding. The web hooks Stripe sends you are merely to update your app. The payments logic is handled all inside stripe and wether your web hook is received or not, their payment will be cancelled.

    Q. If I accidentally set the wrong Price ID in my .env variables I could be signing up customers under the wrong pricing tier.
    A. Yes this is correct, so just make sure you don't. It's not difficult to set the correct env variables. Do it once, make sure it's right, and forget about it.

    Q. If I sign up a customer to Stripe but miss the webhook that they paid successfully I could be charging them without providing the service.
    A. Again yes, this is true, so just make sure you don't miss the webhook. Webhooks are very reliable and they don't change. Take the time to do it correctly, test it and you're good.

  5. 2

    Test, test and test again.

    The other comments are pretty much spot-on, too.

  6. 1

    I found honeybadger.io to be a truly amazing tool to notify me if there is a bug. It will notify you if anyone runs into an exception error and gives you the stack trace. I found a number of bugs this way.

  7. 1

    I'd focus on building your product and not worry too much about edge cases. Unhappy customers can easily be refunded and given extra credit since SaaS products have such a crazy high margin.

    Unless you're selling a product linked to a physical good or service, I'd focus more on keeping customers happy and building features they way. Worst case, you give them a free 3 month deal or a refund.

  8. 1

    Folks like @Primer have already provided good answers on your specific concerns so there's not much to add there. I would like to emphasis how valuable leveraging the Stripe test mode is and utilizing the extensive list of test credit card numbers they provide which let you test every explicit behavior that could occur from a payment processing perspective.

    I'd highly recommend that you set up a test deployment of your app and configure the webhook endpoint in the Stripe test mode and run through all the permutations of things that could happen in your subscription processing flow.

  9. 1

    On a bit different note... Don't worry too much. Bugs are inevitable, and they can even have a positive outcome.

    1. Have communication channels in place, so your customers can contact you easily. Something like tawk.to is amazing and free option. Be responsive, reactive and have a customer support as top priority. Be prepared to refund money, to offer bonuses, etc.

    2. Don't forget about point 1 :-), because if your customers are not able to get in touch with you, they can rather turn to social media with negative feedback, etc.

    3. Be prepared to fix bugs fast!

    And the positive outcome? It's not only about bugs, but there are also users that may misunderstand something about your service, etc. But if you are here for them, open to help them solve the issue, you can turn them into loyal customers that love you!

  10. 1

    Exciting times. Bugs like these are great because it means people are signing up.

    I think go in with your eyes open that there will be bugs. Apologise when they happen, deal with the teething issues. It is also important to try limit the number of signups so you can test and fix it as quickly as it's happening. Don't want a 1000 bugged signups, when you could've fixed it on the first two.

    Why don't you get a few customers to be beta testers/trial users and you'll give them a discount for a month or something to try it out and break it?

    Just know that teething issues are normal, and no issue is insurmountable. Good communication is required and a willingness to quickly fix it. Also you can refund double charges etc.

  11. 1
    1. If the customer already has a subscription, don't send them to the checkout page again.
    2. webhooks are retried for up to 72 hours if your webserver doesn't send back a 200 OK. Make sure your webserver only sends back 200 when it's properly processed.
    3. dont set the wrong price id. I hard code those id's into the server code, I don't have them as environment settings.
    4. see #2.

    There are also things to consider around changing plan (if you have multiple plans) or changing payment options (one card is removed and another added mid subscription).

    Yes. Payment, especially recurring subscriptions, have a lot of moving parts and the cost of getting it wrong is high. An ounce of prevention is worth a pound of cure. Co carefully and steadily and get it of high quality. It pays not to rush and to get this stuff as right as you can.

  12. 1
    1. If the same user makes it to Stripe Checkout twice and completes the payment form both times, they'll now have TWO active monthly subscriptions (or more if they keep going) for the same product.

    Why would they make the Stripe Checkout twice?

    You should have fail-safes in your code to prevent that (i.e. check for existing active subscription first before subscribing the customer).

    1. If I miss a webhook cancelling a subscription I could be charging a customer indefinitely without them knowing.

    If they cancelled the subscription, your customer wouldn't be charged indefinitely, because the subscription would have been cancelled.

    What might happen when your webhook fails (assuming your webhook listens for customer.subscription.deletion events) is that you continue to give your customer access to your product even after they've cancelled, which is not such a big deal.

    1. If I accidentally set the wrong Price ID in my .env variables I could be signing up customers under the wrong pricing tier.

    Test, test, and test.

    Are you using the developer Stripe account to test your API integrations?

    And are you using ngrok to test your webhooks as well?

    1. If I sign up a customer to Stripe but miss the webhook that they paid successfully I could be charging them without providing the service.

    Can you integrate the code into your site instead of depending on the webhook?

    Webhooks do fail, which is why I don't use them unless absolutely necessary.

    So if a customer checks out on your site, create the subscription using Stripe API and then immediately mark that customer as a paid customer once the Stripe subscription has been successfully created.

  13. 1

    These are the so called good problems to have!

    You can mitigate a lot of this by having good tests in place and good error handling.

    Just know mistakes will happen. If a mistake happens against your customer's favor, you need to make it right as soon as possible.

  14. 1

    Hey Justin, these are all valid concerns, but they seem unlikely unless you're making a major mistake in your code. Will a customer checkout twice before your webhook is hit and restricts access to the checkout page? Probably not. Worse case, you apologize to the customer, refund them, and resolve the issue. From my experience, customers are pretty forgiving if they like your product and understand that you're an indie hacker.

    1. 2

      Yeah that's very true, these are definitely not super likely (especially because I've noticed them already), it just makes me wonder about bugs I haven't found. But I guess as you said worst case scenario I just issue a refund and it's not the end of the world! Thanks.

Recommended Posts