OctoML Deployment Platform Overview

The OctoML Deployment Platform is OctoML's Software-as-a-Service (SaaS) product that leverages Apache TVM, ONNX-RT, TensorRT, TFlite, and other acceleration techniques, to automatically generate a deployment-ready artifact for CPUs, GPUs, and accelerators. The result is model performance comparable to state-of-the-art, hand-tuned libraries with no loss in accuracy. In addition, the platform benchmarks your model on diverse hardware targets to help you decide whether investing in faster—but more expensive—hardware makes sense for your use case. Finally, the application packages your model into a light-weight artifact easily deployable in your production environment. The OctoML Platform provides faster machine learning everywhere by:

• Maximizing performance

• Benchmarking comprehensively

• Allowing flexible deployment