Deploying 🤗 ViT on Kubernetes with TF Serving

Hugging Face · 4 years ago · 50

A tutorial demonstrates how to containerize a Vision Transformer model served with TensorFlow and deploy it on Kubernetes using Google Kubernetes Engine. The deployment uses a 2-node cluster with n1-standard-8 machine types, sets CPU requests at 800m per container, and configures autoscaling to maintain between 1 and 3 replicas based on CPU utilization. The approach enables serving the model at scale with automatic resource management while maintaining granular control over deployment configuration through YAML manifests for the deployment, service, and horizontal pod autoscaler.