We'd like to share new improvements and additional changes that may require action!
New Azure Hardware Support
Intel Ice Lake dv5 instances are available on Azure with comparable coverage to other X86 cloud CPUs in the OctoML Platform. TVM, ONNX, TensorFlow, and TFLite engines are supported.
Dependency Library Updates
As part of OctoML's regular dependency library upgrades, to get you the best-in-class performance and coverage on both TVM and ONNX, we have updated our versioning on all hardware except the Jetson family as follows:
TensorRT to 8.0.3.4
CUDA to 11.5.0
cuDNN to 8.3.0
ONNX Runtime to 1.9.0
ONNX to 1.10
TensorFlow to 2.7
We will upgrade versioning on the Jetson family once Jetpack enables support for CUDA 11 by the end of Q1 '22.
New End User Tutorials
We have released two new tutorials which instruct you on:
An end-to-end example on how to accelerate a PyTorch model in the OctoML Platform for an object detection task, and
How to query for the list of supported hardware along with their details in the OctoML Platform.
SDK Updates
The
octomizer.models.tensorflow_keras_modelmodule in our SDK and API have now officially been deprecated. To upload Keras SavedModels, please modify your code to theoctomizer.models.tensorflow_saved_modelmodule. In addition, the enum for all package types shall now be automatically set on behalf of the end user. Users will no longer be able to set the enum in the SDK or API viaoctomizer.package_type.PackageType.