I am making a deep learning inference on-demand service
Deploying deep learning models can be a tricky affair and one needs to pay attention to not only the deployed code but also to the server hardware and framework optimization. I am building a service which will allow anyone to one-click upload and deploy a model, and access it through a secured REST API endpoint.
On-demand inference will allow developers to focus on application architecture rather than on orchestrating model deployment.
Currently, this platform supports Keras and Tensorflow, and we wish to add more frameworks eventually. We will be releasing a beta version of this service by mid-September. If you have any queries or would like to suggest new features, do put them in the comments below!
How are you making this cost effective both for you and your clients? Do you have users/potential users?
Just curious, are you making something based on tensorflow serving?
I worked on something similar but we also wanted to do training. Didn’t really get anywhere though but I’m happy to chat about it if you’re interested and I’m extremely curious on how it goes for you. Good luck!
Hey! So our product takes out the hassle of model deployment; so unlike TensorFlow serving where you have to scaffold your own code around the model, we provide a one click deployment. We haven’t decided the pricing yet. We perform inference on a setup consisting of intel hardware and a custom TensorFlow library optimised for intel cpu based inference. We are also exploring gpu based options.
How do you export models to work with the one click deployments?
We encrypt and store their models.