How do you build a moat around GPT3, 4 or any other open source LLM?
A startup or a product usually is more fundable (or adoptable) if there's a solid moat around the product. With the release of more and more open-source LLMs in the future it seems that AI based small startups will produce brilliant products but the defensibility that they build will always be questionable since big players may easily pip their product.
If your product uses an openly available public model checkpoint, you might compete on pricing through a more efficient infrastructure.
If you have access to valuable domain-specific data, some models will be easy to specialize to your use case through transfer learning.
Generating text might be a smaller part of the overall application and you can build a better app using interfaces powered by these models.