What parts of a startup are the most difficult to outsource/contract out?

Product strategy. I think that's a founders job for quite a while.

Also financial bits and pieces. It's hard to get to a point where you are comfortable handing your bank or stripe account over to others, so I waste a lot of time doing minor financial stuff like transfers.

Initial Sales.

Sales. Coding, marketing and design can be pretty well covered by freelancers.

The part you understand the least in my opinion.

If you know nothing about coding, then how will you be able to evaluate whether a contractor can code well?

If you know nothing about marketing, then how will you know you're talking to a good marketer?

If you know nothing about design, you can see a designer's visual design examples on their website, but you won't necessarily know if they have a good thought process around designing your application.

It makes a lot of sense at first to contract out things outside of your area of expertise, but that just makes finding a contractor to do the work that much harder.