My background is technical, I knew next to nothing of bookkeeping - but, even a small business like mine has a lot of expenses (mostly small). I do not love "admin", but I've found a few tricks to keep the effort low-ish:
Create a special email alias called "accountspayable@mycompany.com" and use it when you sign up for that must-have SaaS tool. That way, when the invoices come up, you can easily filter for them in your inbox.
Payments from your bank account will eventually need to get matched up against invoices (so your business can do useful things like recover sales tax, and avoid more unpleasant tax issues). This is a LOT easier if the bank transcript is meaningful. For foreign currency transactions though, the transcript may be rather cryptic, e.g. P2108US 0.26@0.84615 (the date, the amount, and the currency conversion rate). Use a service like Wise to get meaningful transcripts (and better exchange rates).
Get an accountant/bookkeeper involved early on. You'll be surprised how quickly they start to pay for themselves with tax refunds and the like (I spent about EUR 550 this month on accounting services, but ALSO collected a EUR 1,700 VAT rebate).
When receipts and invoices come in, file them that day, not a month or two later (shared folder with your bookkeeper works great).
Done right, admin won't take you much time, and the neat records that result will make it easy to get on overview of where your money is going (for me, it is wages, cloud compute, professional services, SaaS fees, in that order).