TFLite Model Ingest
Users can now upload models in TFLite format into the OctoML Platform.
To upload via the web UI, select the TFLite format and upload a model with a
.tflitefile extension:
To upload via the SDK, first ensure you are using the latest version of the SDK— installed after 11.04.2021 or upgraded via:
$ python3 -m pip install octomizer-sdk --extra-index-url https://octo.jfrog.io/artifactory/api/pypi/pypi-local/simple --upgrade
Then upload a TFLite model using:
import octomizer.client
import octomizer.models.tflite_model as tflite_model
client = octomizer.client.OctomizerClient()
model = tflite_model.TFLiteModel(
client,
name=model_package_name,
model=tflite_model_file,
description="Created by octomizer_example_tflite.py",
)
TFLite Model Coverage, Benchmarking, and Packaging
At this time the OctoML Platform supports models trained using TensorFlow versions 2.0, 2.1, and 2.2 that contain any of operators listed here.
For now, users can benchmark a TFLite model in the TVM runtime. Support for ONNX-RT benchmarking and TFLite runtime benchmarking will be added in coming weeks.
TFLite models accelerated by TVM will continue to be packaged in the usual manner and available in .so or python wheel format.