Key Takeaways
- Kubernetes Dynamic Resource Allocation (DRA) allows workloads to request specialized hardware by attributes, not just by integer device counts.
- DRA shifts device scheduling from “give me one GPU” to “give me a device that matches these requirements,” e.g., GPU memory, product family, topology, or driver capability.
- DRA core reached general availability (GA) in Kubernetes 1.34 and is locked on in Kubernetes 1.35, but driver and managed-service maturity still vary.
- DRA improves how Kubernetes represents devices, but it does not create fractional GPUs, guarantee safe sharing, or replace device plugins everywhere on its own.
- ScaleOps GPU Platform adds workload-aware optimization on top of DRA by observing real GPU demand, improving placement, and aligning allocation with performance, utilization, and cost.
What Is Kubernetes Dynamic Resource Allocation (DRA)?
Kubernetes Dynamic Resource Allocation (DRA) is a framework that allows workloads to describe the exact hardware they require (e.g., memory, device family, or performance class). Instead of having pods request an integer count of an opaque resource, the scheduler can then match those requests to available hardware, including GPUs, network interface cards (NICs), field-programmable gate arrays (FPGAs), and other accelerators.
The Kubernetes version matters because DRA stability depends on both the upstream API and the device ecosystem around it. While DRA core graduated to GA in Kubernetes 1.34, in Kubernetes 1.35, the DRA feature gate is locked on, i.e., enabled by default, and cannot be turned off. This means that stable APIs are also enabled by default, but it does not mean that every GPU driver, cloud-provider integration, or managed-service workflow is production-ready.
To understand why DRA is such a necessary evolution, we must first look at the legacy approach it aims to replace and why it falls short.
Why Kubernetes Device Plugins Struggle with Modern GPU Scheduling
The legacy device plugin model limits modern AI workloads by treating complex GPUs as opaque, integer-based assets. A traditional plugin merely reports total device counts on a node, and pods must request whole units (e.g., nvidia.com/gpu: 1).
Because the internal Kubernetes scheduler cannot evaluate detailed device attributes or performance metrics, this asset-count model introduces three critical bottlenecks:
- Rigid allocations: Treating hardware as opaque forces whole-GPU assignments, restricting multi-tenant efficiency and causing static Multi-Instance GPU (MIG) fragmentation.
- Weak topology awareness: The scheduler cannot naturally place tightly coupled workloads near high-speed interconnects or specific network paths, forcing teams to rely on vendor-specific workarounds.
- Pre-provisioning bottlenecks: Accelerators must be manually configured and active on cluster nodes before incoming pods can be scheduled.
All of this impacts autoscaling because Kubernetes can only schedule the pod onto hardware that already exists in the cluster. If a workload requires a specific accelerator not currently on any node, the pod stays pending until Cluster Autoscaler or Karpenter adds a suitable node. DRA helps reduce this gap.
DRA Core Objects: DeviceClass, ResourceClaim, and More
DRA introduces a small set of scheduler-aware objects that work together to expose device inventory and workload requirements:
| DRA Object | What it does |
| DeviceClass | Defines a device category and selection rules |
| ResourceClaim | Describes a workload request and records the allocation |
| ResourceClaimTemplate | Creates claim templates for replicated workloads |
| ResourceSlice | Describes device inventory and attributes from the driver |
The workflow is as follows:
- Drivers publish ResourceSlices.
- Admins define DeviceClasses.
- Workloads create ResourceClaims.
- The scheduler allocates matching devices.
Once a claim is allocated, the selected device is prepared for the pod on the chosen node.
How DRA Allocation Runs: API Group to Container
DeviceClass, ResourceClaim, ResourceClaimTemplate, and ResourceSlice all live in the resource.k8s.io/v1 API group, the same group that reached GA in Kubernetes 1.34. Knowing the object model is one thing; knowing which component touches a claim at each step is what actually explains where a pending pod is stuck.
- Driver registration and inventory. A DRA driver runs a kubelet plugin on each node, registers with kubelet, and publishes the node’s available hardware as ResourceSlices.
- Scheduling. When a pod references a ResourceClaim or ResourceClaimTemplate, kube-scheduler’s built-in
DynamicResourcesplugin holds the pod pending while it matches the claim’s requirements against published ResourceSlices across the cluster. Once it finds a fit, it writes the allocation result into the ResourceClaim’s status and binds the pod to that node. This is the step Cluster Autoscaler or Karpenter has to be able to reason about before adding a node, which is why driver and autoscaler compatibility matters as much as the API itself. - Node-level preparation. Kubelet on the chosen node calls the driver’s kubelet plugin over gRPC (
NodePrepareResources). The plugin generates a Container Device Interface (CDI) spec file describing the allocated device. - Container runtime handoff. Kubelet passes the CDI device IDs to the container runtime (containerd or CRI-O) through the Container Runtime Interface (CRI). The runtime uses that CDI spec to actually inject the device into the container before it starts, the same mechanism that replaced vendor-specific runtime patches like
nvidia-container-runtimeshims. - Cleanup. When the pod is deleted or the claim is released, kubelet calls
NodeUnprepareResourcesso the driver can tear down the CDI spec and free the device for the next claim.
The device plugin model collapses steps 2 through 4 into a single kubelet-local Allocate call with no claim object and no CDI standardization, which is why it can’t express cross-node topology or attribute matching. DRA splits allocation (scheduler, cluster-wide) from preparation (kubelet, node-local), which is also why a driver bug at either layer produces different symptoms: a stuck ResourceClaim points at the scheduler side, while a pod stuck in ContainerCreating after a successful claim points at the kubelet plugin or CDI generation.
Kubernetes DRA vs. Device Plugins: How Scheduling Changes
The old device plugin flow is count-based. A plugin advertises a resource count, a pod requests a count, the scheduler finds a node with enough units, and the device is prepared after placement.
The DRA flow is more attribute-aware:
- A driver publishes device attributes through ResourceSlices.
- A workload references a ResourceClaim or ResourceClaimTemplate.
- The scheduler then assesses compatible devices, logs the hardware assignment, and deploys the pod to a node equipped to utilize the chosen equipment.
| Area | Device Plugin Model | DRA Model |
| Request format | Integer resource count | Attribute-based claim |
| Device attributes | Mostly opaque to scheduler | Scheduler can evaluate published attributes |
| Topology awareness | Limited | Better when drivers publish topology details |
| Sharing | Vendor-specific behavior | Represented through driver and claim semantics |
| Autoscaler visibility | Often tied to existing node resources | Better when claims expose scheduler-readable requirements |
| Maturity | Common in production | Stable core API, but driver maturity varies |
ScaleOps pro tip: Use Smart Pod Placement to improve bin packing while respecting Kubernetes scheduling constraints.
How Structured Parameters and CEL Filtering Make DRA Scheduling More Precise
Structured parameters enable the scheduler and Cluster Autoscaler or Karpenter to evaluate device attributes directly. This allows the scheduler to make better decisions since device requirements are visible as structured data rather than hidden inside driver-specific logic.
Because these features are still relatively new, organizations must carefully check their cluster’s Kubernetes version before adopting DRA. Structured Parameters made DRA attributes visible to the scheduler before the current stable DRA workflow. DRA core reached GA in Kubernetes 1.34, and Kubernetes 1.35 locks the DRA feature gate on.
Common Expression Language (CEL) is the filtering mechanism for device selection. CEL provides precise control over device matching, but it also adds to the syntax complexity. Platform teams should provide reusable DeviceClasses so application teams do not have to write complex expressions for every workload.
A simplified selector can express requirements such as minimum GPU memory, GPU family, performance class, or topology:
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: h100-80gb-claim
spec:
spec:
devices:
requests:
- name: gpu
exactly:
deviceClassName: gpu.example.com
selectors:
- cel:
expression: |
device.attributes["gpu.example.com"].family == "h100" &&
device.attributes["gpu.example.com"].memory_gb >= 80Kubernetes DRA Across GKE, EKS, and AKS: Managed-Service Readiness Check
DRA support looks different across managed Kubernetes services. The Kubernetes API may be stable, but each provider still decides which cluster modes, GPU drivers, node provisioning paths, monitoring features, and support levels are available. Before standardizing on DRA, validate the provider-specific workflow you plan to use.
Google Kubernetes Engine documents DRA for GPU allocation in Standard clusters. It also notes important limitations: DRA is available only in Standard mode clusters; GPU DRA is in Preview; and time-sharing GPUs, MIG, and Multi-Process Service (MPS) are not supported with the current GKE DRA GPU path.
Amazon Elastic Kubernetes Service documents DRA for specialized hardware management, including GPU-related workflows and other device paths. AWS notes that DRA is available on EKS with Kubernetes 1.33 and later, but recommends Kubernetes 1.34 and later for DRA. DRA is not currently compatible with Karpenter or EKS Auto Mode provisioned compute, so managed node groups or self-managed nodes remain the supported path for DRA driver workflows.
Azure Kubernetes Service engineering has published guidance for advanced AI infrastructure patterns for DRA, NVIDIA GPU driver, and Dynamic Resource Allocation for networking (DRANET). Treat AKS DRA as an emerging pattern that requires validation before production standardization.
| Cloud Provider | Current DRA Angle | Caveat | Production Validation Step |
| GKE | DRA for GPUs in Standard clusters | Preview/provider-specific workflows; no time-sharing, MIG, or MPS in the current GKE DRA GPU path | Validate launch stage, driver setup, monitoring limits, and unsupported sharing modes |
| EKS | DRA for specialized hardware and GPU workflows | Not currently compatible with Karpenter or EKS Auto Mode provisioned compute | Validate managed node group or self-managed node path and driver versions |
| AKS | DRA, NVIDIA GPU driver examples, and DRANET patterns | Some workflows remain user-managed, feature-gated, or experimental | Validate support status, driver lifecycle, and upgrade path |
DRA Kubernetes GPU Scheduling with MIG, Virtual GPU (vGPU), and Fractional Allocation
DRA represents device options in claims. However, vendor drivers, hardware partitioning, virtual GPU (vGPU) layers, and optimization platforms perform the actual partitioning and sharing.
ScaleOps pro tip: ScaleOps GPU Platform’s Automated Fractional GPUs capability observes actual GPU compute and memory usage in real time, then manages fractional GPU allocation to match real workload demand. This avoids the static rigidity of MIG profiles and the weaker isolation and contention tradeoffs of time-slicing.
How DRA Changes Autoscaling and Node Provisioning
DRA makes node provisioning more specific because pending pods may require exact device attributes. Adding generic nodes does not help if they cannot satisfy the ResourceClaim.
Structured parameters make autoscaling decisions more informed by describing the device a pending pod needs in a scheduler-readable way. Before relying on DRA for GPU scale-up, confirm that Cluster Autoscaler or Karpenter can add nodes with devices that actually satisfy the ResourceClaim.
For EKS teams, the main autoscaling caveat is Karpenter itself. DRA lacks compatibility with both Karpenter and EKS Auto Mode provisioned compute, meaning teams should first validate DRA driver workflows with managed node groups or self-managed nodes.
The table below shows how DRA changes the autoscaling approach:
| Scenario | Device plugin behavior | DRA-aware behavior | Validation needed |
| Pod needs one GPU | Schedules to any node with allocatable GPU count | Can match a GPU claim to required attributes | Confirm driver publishes usable ResourceSlices |
| Pod needs 80 GB GPU memory | Often encoded through node pools or labels | Expressed as a device attribute filter | Confirm attribute names and Cluster Autoscaler/Karpenter behavior |
| Pod needs topology near a NIC | Difficult to model with only resource counts or node labels | Can be represented when driver exposes topology | Validate driver and node inventory details |
| Pod stays pending | Adds nodes with enough named resources | Adds nodes that satisfy the ResourceClaim | Confirm Cluster Autoscaler or Karpenter can model claim requirements |
Troubleshooting a Stuck DRA Pod
DRA splits allocation from preparation, so a stuck pod usually points to one of two layers. Check them in this order:
- Pod events first.
kubectl describe pod <name>surfaces the scheduler’s own status messages: “waiting for resource driver to provide information” means no node has been selected yet, and “waiting for resource driver to allocate resource” means a node was selected but allocation hasn’t finished. - ResourceClaim state.
kubectl get resourceclaimshows a STATE column.pendingmeans the claim isn’t allocated yet;allocated,reservedmeans the scheduler picked a device and bound it to the pod. A claim stuck inpendingis a scheduler-and-driver problem, not a kubelet problem. - ResourceSlice inventory.
kubectl get resourcesliceandkubectl get deviceclassconfirm the driver is actually publishing devices the claim can match. An empty or stale ResourceSlice list means the driver’s controller isn’t running or hasn’t registered with kubelet. - Scheduler verbosity. Running kube-scheduler at
-v=4(or-v=5for per-plugin detail) logs progress from thedynamicresourcesplugin specifically, and theunschedulable_podsmetric buckets these pods under the same plugin label. - Kubelet and CDI artifacts. If the ResourceClaim already shows
allocated,reservedbut the pod is stuck inContainerCreating, the problem has moved from the scheduler to the node. Check the DRA driver’s kubelet plugin logs forNodePrepareResourcesfailures, and confirm the CDI spec file exists under the driver’s configured CDI root (commonly/var/run/cdior/etc/cdi). A missing or malformed CDI file is what leaves a pod with a satisfied claim but no injected device.
This mirrors the split described above: a claim that won’t leave pending is a scheduler-side issue, and a claim that’s allocated but a pod that never starts is a node-side issue.
Current Kubernetes DRA Limitations and Production Maturity Status
DRA has two maturity layers that teams must evaluate separately. The Kubernetes API may be stable, but GPU production readiness still depends on the driver, cloud provider, autoscaler, observability stack, and governance model around that API.
The main caveats in production are not about the DRA API itself, but about everything around it: vendor differences, preemption limits, autoscaler compatibility, RBAC and governance needs, and device health observability. These are the areas where production readiness can still vary even when the Kubernetes API is stable.
Note: Kubernetes 1.34 added alpha support for surfacing DRA device health in pod status when the driver supports it.
| Capability | Kubernetes Status | Vendor Dependency | Production Recommendation |
| DRA core API | Stable in current Kubernetes | Low for API, high for devices | Safe to learn and design around |
| GPU DRA drivers | Driver-specific | High | Validate version, upgrade path, and support status |
| Device health reporting | Available when supported by the Kubernetes version, feature gates, and driver | High | Confirm what device health information your driver exposes |
| Autoscaler integration | Improving with structured parameters | Medium to high | Test scale-up and scale-down with real claims on Cluster Autoscaler or Karpenter |
| Role-based access control (RBAC) and governance | Required operational layer | Medium | Restrict who can create claims and DeviceClasses |
How ScaleOps GPU Platform Adds Workload-Aware Optimization to Kubernetes DRA
DRA allocates hardware, but it does not determine the optimal number of GPUs a workload should receive. ScaleOps GPU Platform adds that layer on top of DRA: observing real GPU compute and memory usage, managing GPU allocation to match demand, improving bin packing for AI and inference workloads, and keeping inference fast as allocation tightens.
GPU Platform runs as an active, autonomous layer above the core scheduling framework:
GPU Observability: Tracks actual GPU compute demand and memory usage through live, workload-level metrics, eliminating manual tuning and guesswork.
Automated Fractional GPUs: Continuously manages fractional GPU allocation and GPU sharing to match real-time application behavior, preventing over-provisioning without risking OOMKill events.
GPU Replica Optimization and Batch Inference Optimization: Optimize bin-packing and inference replica counts for AI and batch workloads, reducing overall node counts while protecting latency.
GPU Memory Optimization: Right-sizes GPU memory for serving frameworks like vLLM, recovering capacity without hurting performance, with cost efficiency following as a result of that optimization.
Conclusion: Bridging the Gap Between Hardware Allocation and Optimization
Kubernetes DRA represents a significant evolution in hardware scheduling, shifting the platform from rigid, count-based requests to flexible, attribute-based claims. This change gives teams the granular control needed to manage complex accelerators and modern AI workloads.
However, upstream API stability is only part of the equation, as visibility is not optimization. To achieve true operational efficiency, teams must combine their scheduling framework with intelligent, autonomous orchestration that ensures hardware is fully utilized.
To see how autonomous resource management can improve your cluster utilization and eliminate manual tuning, book a live ScaleOps demo.
Kubernetes DRA: Frequently Asked Questions
What is Kubernetes Dynamic Resource Allocation?
Kubernetes Dynamic Resource Allocation is a framework for requesting and allocating specialized hardware such as GPUs, NICs, FPGAs, and accelerators. It lets workloads request devices based on attributes rather than relying solely on integer resource counts.
Is DRA stable in Kubernetes 1.34 and 1.35?
DRA core reached general availability in Kubernetes 1.34, and the feature gate is locked on in Kubernetes 1.35. Driver support, provider support, and GPU-specific production readiness still vary.
Does Kubernetes DRA replace device plugins?
Not everywhere. DRA is the direction for richer device allocation, but device plugins remain common in production GPU environments where DRA driver support or managed-service workflows are not yet proven.
Does DRA provide fractional GPU allocation?
No. DRA can represent device options and claims, but MIG, virtual GPU (vGPU), vendor drivers, or optimization platforms perform the actual partitioning and sharing.
How does DRA help Kubernetes GPU scheduling?
DRA helps the scheduler match workloads to GPU attributes such as memory size, product family, performance class, topology, or driver capability. This is more expressive than requesting one opaque GPU unit.
What is the Container Device Interface (CDI) and how does it relate to DRA?
The Container Device Interface (CDI) is the spec format a DRA driver’s kubelet plugin generates after a claim is allocated. Kubelet passes the CDI device IDs to the container runtime through CRI, and the runtime uses that spec to inject the device into the container. It’s the standardized replacement for vendor-specific runtime patches.
How does ScaleOps GPU Platform work with Kubernetes DRA?
ScaleOps GPU Platform works on top of DRA by observing workload behavior, automating fractional GPU allocation and GPU memory right-sizing, improving bin packing, and tying allocation decisions to performance, reliability, and cost.
