How do you manage your LLM AI models?

Hey everyone!

Many of you use large language models in your products for writing copy, creating customised travel advice, chat bots, etc.

I struggle with managing prompt versions, training data, and evaluating performance. I'm curious how you train and evaluate your LLM models!

• How do you iterate on your prompts?
• How do you validate whether one prompt is better than another?
• Do you know how your prompt performs on various inputs?
• Do you use fine-tuning? How do you manage your training data?
• How do you evaluate your model's performance with your customers?

Your insights would be invaluable to me and others in the community 🙏