Composable GPU Scheduling Policies: How mutex, binpack, spread, and numa Work Together
HAMi has always offered per-Pod GPU scheduling policies through the hami.io/gpu-scheduler-policy annotation: binpack to pack workloads onto as few cards as possible, spread to distribute them, mutex (new in v2.10.0) to demand an exclusive card. Until now, the annotation accepted exactly one value.
Real clusters rarely want just one behavior at a time. A typical production wish list looks like this: pack inference replicas tightly to leave whole cards free, but keep each Pod's GPUs on the same NUMA node for bandwidth, and give the latency-critical tier cards of its own. That is three policies in a single sentence. Before v2.10.0 you had to pick one and give up the rest.
v2.10.0 closes this gap: hami.io/gpu-scheduler-policy now accepts an ordered, comma-separated list of policies, so filter-style and sort-style policies compose (#2621, @mesutoezdil, closes #2010). This post explains how the combination actually works, and how to adopt and verify it. If you prefer to learn by doing, the companion Lab 14: Composable GPU Scheduling Policies on GKE walks through every scenario below on a real cluster.
