← Engineering Dispatches / Cloud & DevOps

Kubernetes Multi-Tenant Cluster Isolation: Hard Multi-Tenancy with vCluster, Cilium Network Policies, and RBAC

By Aman Aslam · 14 min read read

Deploying a dedicated Kubernetes cluster for every customer or internal business unit results in severe cloud bill sprawl, idle compute overhead, and massive operational maintenance. However, naive namespace-only multi-tenancy leaves clusters vulnerable to noisy-neighbor CPU starvation, DNS spoofing, and lateral container breakout attacks. Achieving enterprise-grade "hard multi-tenancy" requires virtual control planes (vCluster), eBPF-powered Cilium network isolation, and automated Open Policy Agent (OPA) admission controls.

Architectural Takeaways

Namespace-only isolation provides soft multi-tenancy suitable for trusted teams, while vCluster (Virtual Clusters) provides hard multi-tenancy with dedicated API servers and CRD support.
Enforce zero-trust cross-namespace traffic blocking using Cilium eBPF NetworkPolicies that filter packets at the Linux kernel layer without iptables performance degradation.
Automate resource enforcement using hierarchical ResourceQuotas, LimitRanges, and OPA Gatekeeper policies to reject unconstrained container deployments.

1. Soft vs Hard Multi-Tenancy: Threat Models & Tradeoffs

In a standard Kubernetes cluster, cluster-scoped resources like Custom Resource Definitions (CRDs), ClusterRoles, and Ingress Controllers are shared globally. If Tenant A installs an incompatible CRD version or floods the central Kube-API server with requests, all other tenants experience control plane degradation.

Hard multi-tenancy isolates both the control plane (API server, etcd, controller manager) and the data plane (networking, storage, compute execution), allowing untrusted tenants to operate securely on shared underlying physical nodes.

2. Virtual Kubernetes Clusters with vCluster

vCluster creates a lightweight, fully functional virtual Kubernetes cluster inside a single namespace of the host cluster. The tenant interacts with their own private Kube-API server, creating CRDs and namespaces freely, while underlying pods are scheduled onto the physical host cluster nodes.

3. Kernel-Level L3/L4/L7 Isolation with Cilium eBPF

Traditional Kubernetes NetworkPolicies rely on iptables, which degrades in performance when thousands of rules are generated across multi-tenant environments. Cilium uses Linux eBPF bytecode programs attached directly to network sockets, filtering traffic with zero CPU overhead and providing deep L7 HTTP visibility.

4. Least-Privilege RBAC, OPA Gatekeeper & Quotas

By coupling strict ResourceQuotas with OPA Gatekeeper constraint templates, platform engineers prevent privilege escalation attacks, block root-privileged container execution, and enforce automated tenant decommissioning.

Read more technical guides on our Dispatches Index →