The OctoML Platform currently offers packaging for deployment and acceleration for the following hardware architectures**:
Intel Broadwell
Intel Cascade Lake
Intel Skylake
Intel Ice Lake
AMD EPYC Milan
AMD EPYC Rome
NVIDIA Tesla V100
NVIDIA Tesla K80
NVIDIA Tesla T4
Arm Graviton2
Arm Cortex A-72 (64 and 32 bit)
Arm Cortex A-53 (64 bit)
Jetson Xavier AGX
Jetson Xavier Nano
Jetson Xavier NX
** Some users also have access to private hardware targets not listed here.
For server-based targets, users can select from the instance types offered by Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Provider (GCP), including various core counts for CPUs.
Some GCP instances display in the product multiple times because GCP allows users to specify a specific architecture within that instance (for example, n1-standard-4 may have a Skylake CPU or T4 GPU attached to it). These are listed separately in the hardware menu along with their associated architecture.
Not all instances available in the market are shown here, in part because public cloud providers do not make all instances available with a specific CPU architecture. In these cases, it is possible that a model accelerated on one architecture (e.g. Cascade Lake) could deployed or benchmarked on a different architecture (e.g. Skylake) even though the instance name has not changed. For this reason, we do not make instances with multiple possible architectures available to users.
AWS Instances (Detailed specs here) | GCP Instances (Detailed specs here) | Azure Instances (Detailed specs here) |
c5n.xlarge c5n.2xlarge c5n.9xlarge c5n.18xlarge c5.12xlarge c5.24xlarge c6i.xlarge c6i.2xlarge c6i.4xlarge c6i.8xlarge c6i.12xlarge c6i.16xlarge c6i.24xlarge m6i.xlarge m6i.2xlarge m6i.4xlarge m6i.8xlarge m6i.12xlarge m6i.16xlarge m6i.24xlarge m6i.32xlarge m6g.xlarge m6g.4xlarge m6g.8xlarge m6g.12xlarge m6g.16xlarge t4g.large t4g.xlarge t4g.2xlarge p2.xlarge g4dn.xlarge p3.2xlarge g5.xlarge | n1-standard-2 n1-standard-4 n1-standard-8 n1-standard-16 n1-standard-32 n1-standard-2 n1-standard-4 n1-standard-8 n1-standard-16 n1-standard-32 n2-standard-4 n2-standard-4 n2-standard-16 n2-standard-48 n2-standard-80 c2-standard-8 c2-standard-16 c2-standard-30 c2-standard-60 n2-standard-4 n2-standard-16 n2-standard-48 n2-standard-96 n2-highmem-4 n2-highmem-16 n2-highmem-48 n2-highmem-96 n2d-standard-4 n2d-standard-16 n2d-standard-48 n2d-standard-80 n2d-standard-96 c2d-standard-4 c2d-highcpu-8 c2d-highcpu-16 c2d-highcpu-32 c2d-highcpu-56 c2d-highcpu-112 c2d-highmem-2 n2d-standard-4 n2d-standard-16 n2d-standard-48 n2d-standard-96 n2d-highmem-4 n2d-highmem-16 n2d-highmem-48 n2d-highmem-96 t2d-standard-4 t2d-standard-8 t2d-standard-16 t2d-standard-32 t2d-standard-48 t2d-standard-60 n1-standard-4 n1-standard-4 custom-12-77824 n1-standard-4 custom-12-77824 | Standard_D4_v4 Standard_D8_v4 Standard_D16_v4 Standard_D32_v4 Standard_D48_v4 Standard_D64_v4 Standard_D4_v5 Standard_D8_v5 Standard_D16_v5 Standard_D32_v5 Standard_D48_v5 Standard_D64_v5 Standard_HB120-16rs_v3 Standard_HB120-32rs_v3 Standard_HB120-64rs_v3 Standard_HB120-96rs_v3 Standard_HB120rs_v3 |