OctoML

Hardware sweeps + FP16 downcasting for even more cost savings!

Release Notes - 02.10.2023

Explore latency/throughput/cost tradeoffs, version upgrades, and new Arm Ampere CPUs!

Classical ML support is here--sklearn and more!

Additional acceleration engines for improved model performance!

New Docker tutorials to deploy your models even faster!

Accelerate models new Arm-based Graviton3 hardware!

Substantial increase in model coverage across TensorFlow, Pytorch, and ONNX!

SDK Upgrades for Packaging & Acceleration

AWS Arm-based Graviton2, improved UX, more tutorials, and SDK updates!

INT8 and FP32 TVM performance improvements!

Jetson Nanos, more Ice Lake instances, and better ONNX-RT coverage on GPUs!

New Azure instances, dependency upgrades, and end user tutorials!

More XL model support, CUDA and TensorRT version upgrades, and benchmarking on the edge!

Intel Ice Lake, EPYC Rome, INT8 improvements, and TF SavedModel support!

Projects organization, performance comparisons, TFLite and ONNX-RT benchmarking!

XL model support and TensorFlow version upgrades!

TensorFlow model upload, TVM performance boost, and more benchmarking!

Tuning time reductions, ONNX tarball support, and package naming!

SDK upgrade, input shape management, and 32-bit Arm A-72 devices!

Find answers and get help from Intercom Support and Community Experts