We'd like to share new improvements and additional changes that may require action!

New Azure Hardware Support

  • Intel Ice Lake dv5 instances are available on Azure with comparable coverage to other X86 cloud CPUs in the OctoML Platform. TVM, ONNX, TensorFlow, and TFLite engines are supported.

Dependency Library Updates

  • As part of OctoML's regular dependency library upgrades, to get you the best-in-class performance and coverage on both TVM and ONNX, we have updated our versioning on all hardware except the Jetson family as follows:

    • TensorRT to

    • CUDA to 11.5.0

    • cuDNN to 8.3.0

    • ONNX Runtime to 1.9.0

    • ONNX to 1.10

    • TensorFlow to 2.7

  • We will upgrade versioning on the Jetson family once Jetpack enables support for CUDA 11 by the end of Q1 '22.

New End User Tutorials

SDK Updates

  • The octomizer.models.tensorflow_keras_model module in our SDK and API have now officially been deprecated. To upload Keras SavedModels, please modify your code to the octomizer.models.tensorflow_saved_model module. In addition, the enum for all package types shall now be automatically set on behalf of the end user. Users will no longer be able to set the enum in the SDK or API via octomizer.package_type.PackageType.

Did this answer your question?