mesheryctl design import -f https://raw.githubusercontent.com/meshery/meshery.io/master/catalog/11c27e2d-7354-4eff-a7f1-4e6a65fe1e3b/0.0.2/design.yml
What This Design Does:
A Kubernetes-based reference architecture for deploying a portable AI inference gateway. The design provides a centralized entry point for inference requests and routes traffic to an independently managed model-serving backend. It includes an Ingress resource for external access, gateway and model-server Deployments, Services for internal communication, and supporting resources for configuration management, identity, security, autoscaling, and availability. The architecture demonstrates how AI inference workloads can be organized using Kubernetes-native components while maintaining separation between request routing and model serving. It is intended as a reusable starting point for teams building self-hosted AI inference infrastructure.
Design Snapshot
Caveats and Consideration:
Caveats This design is a reference architecture and is not a fully production-ready AI inference platform. The model-server workload currently uses a placeholder HTTP container rather than an actual machine learning model runtime. It must be replaced with a compatible inference server such as vLLM, KServe, or another model-serving solution before handling real AI inference requests. The Ingress resource requires a compatible Ingress Controller in the target cluster. Horizontal Pod Autoscaling requires a functioning metrics API and appropriate resource requests. NetworkPolicy enforcement depends on CNI support. The Secret contains placeholder configuration and must be populated with appropriate credentials before production use. No real API keys or sensitive credentials should be stored in the design. A successful Meshery dry-run validates resource compatibility with the tested Kubernetes environment but does not guarantee successful runtime deployment, application functionality, security, or performance. Considerations Before deploying this design, consider the following: 1. Replace the placeholder model-server container with an appropriate inference runtime and configure the required model, storage, and serving parameters. 2. Configure resource requests and limits for gateway and model-server workloads to support predictable scheduling and effective autoscaling. 3. Configure the HorizontalPodAutoscaler according to expected inference traffic, available cluster capacity, and supported resource metrics. 4. Configure TLS termination, authentication, authorization, and rate limiting at the gateway or Ingress layer before exposing inference endpoints externally. 5. Review NetworkPolicy ingress and egress rules to ensure that only authorized workloads can communicate with the model-serving backend. 6. Use a secure secrets-management solution to provide API credentials and other sensitive configuration. 7. Configure health checks, logging, monitoring, and tracing to observe gateway availability and inference request performance. 8. Verify Kubernetes API compatibility, Ingress Controller availability, CNI support, and resource permissions in the target environment before deployment. 9. Test the architecture in a staging environment before adopting it in production.
Compatibility:
Design Details
CATEGORY
deploymentCREATED BY
CREATED AT
November 09, 2026VERSION
0.0.2-
MODELS
Related Designs
vault-0.30.1.tgz
