What payment provider do you use?

I've used Stripe, PayPal and Gumroad. PayPal is good for things like invoicing or consulting type businesses, but I don't think it has many advantages over Stripe for IH-type businesses. With that said, one big advantage of PayPal is it's available in more countries. If your business has any in-person component, PayPal might also be a good option, since they send you a debit card and their card readers are often free. Stripe Corporate Card will make them pretty much equal here too, though.

I found the checkout and onboarding aspects of Stripe (Stripe Connect API) to be better than PayPal. Users are often redirected away from your site during checkout on PayPal.

Gumroad is good for selling digital products, if that's important for your model. They have a built-in referral program which is nice and they also support subscriptions and pre-orders.

In summary, here are my thoughts:

• Stripe - Good for website integration, SaaS and subscription business models for a small number of countries
• PayPal - Good for invoicing for things like consulting businesses and for reaching a larger number of countries
• Gumroad - Good for selling digital products and subscriptions

There's a good IndieHackers thread, albeit a bit dated, about Stripe alternatives here