How to do DevOps for ML models?

I'd like to monitor my ML models through their entire life cycle, from training all the way up until it is making inferences in production.

I'd like to track or better understand:

• data used during train/test
• input and output of the model in production, for example: capturing outliers
• history of model performance along with indications of changes to the model
• compare and contrast models in production to help make business decisions

Essentially the way I like to think about it is implementing a detailed DevOps system around my ML models to understand their behavior in the real world better.