TFLite Model Ingest

Users can now upload models in TFLite format into the OctoML Platform.

  • To upload via the web UI, select the TFLite format and upload a model with a .tflite file extension:

  • To upload via the SDK, first ensure you are using the latest version of the SDK— installed after 11.04.2021 or upgraded via:

    $ python3 -m pip install octomizer-sdk --extra-index-url --upgrade

  • Then upload a TFLite model using:

    import octomizer.client
    import octomizer.models.tflite_model as tflite_model

    client = octomizer.client.OctomizerClient()

    model = tflite_model.TFLiteModel(
    description="Created by",

TFLite Model Coverage, Benchmarking, and Packaging

  • At this time the OctoML Platform supports models trained using TensorFlow versions 2.0, 2.1, and 2.2 that contain any of operators listed here.

  • For now, users can benchmark a TFLite model in the TVM runtime. Support for ONNX-RT benchmarking and TFLite runtime benchmarking will be added in coming weeks.

  • TFLite models accelerated by TVM will continue to be packaged in the usual manner and available in .so or python wheel format.

Did this answer your question?