Migrate Existing Workloads to Smart Karpenter

This topic describes how to migrate existing workloads from legacy node groups to Smart Karpenter-managed capacity in clusters that are not greenfield. Use this migration when replacing a traditional autoscaler with Smart Karpenter while preserving cluster stability and maintaining critical system services.

Prerequisites

• A non-Smart Karpenter node pool with at least three nodes for critical services
• Smart Karpenter installed and configured in the cluster
• Access to update workload manifests, node pool configurations, and autoscaling settings

Migrate Existing Workloads

Follow the following steps to migrate workloads from legacy node groups to Smart Karpenter-managed nodes in non-greenfield clusters.

Step 1: Prepare a Non-Smart Karpenter Node Pool

1. Verify that the cluster has a dedicated non-Smart Karpenter node pool.
2. Ensure this node pool contains a minimum of three nodes.
3. Confirm that critical cluster services—such as DNS, Smart Karpenter, and foundational platform components—are running on these nodes.
4. Note that Smart Karpenter does not run on nodes it provisions; no explicit configuration is required to enforce this.

Step 2: Gradually Reduce Legacy Node Groups

1. Identify node groups currently managed by the legacy autoscaler.
2. Reduce the size of those node groups incrementally.
3. Monitor workload behavior to verify that Smart Karpenter provisions new nodes as capacity decreases.
4. Continue reducing the legacy node pool as workloads begin shifting to Smart Karpenter-managed nodes.

Step 3: Update Workloads to Schedule on Smart Karpenter Nodes

1. Review existing workload manifests for node selectors, affinities, or labels that bind them to legacy node pools.
2. Remove outdated scheduling constraints tied to old nodes.
3. Add or align node selectors and affinities with labels used by Smart Karpenter-provisioned nodes.
4. Apply taints and tolerations if Smart Karpenter node groups are configured for isolated or specialized workloads.
5. Verify that workload resource requests match the instance types that Smart Karpenter is allowed to provision.

Step 4: Keep Critical Services on the Non-Smart Karpenter Pool

1. Ensure that essential system services remain on the non-Smart Karpenter node pool.
2. Validate that DNS, Smart Karpenter, and other infrastructure components continue operating on this three-node minimum pool.
3. Confirm that only application workloads migrate to Smart Karpenter-managed nodes, including tainted or specialized jobs.

Step 5: Validate Against Official Karpenter Migration Guidance

1. Review Karpenter's official documentation for recommended migration patterns and best practices.
2. Integrate upstream guidance on node consolidation, cluster autoscaler deprecation, and workload placement.
3. Validate the final cluster state before fully deprecating the legacy autoscaler.

Example for Transitioning Deployments to Karpenter Nodes

This example shows how to update an existing deployment so it schedules on Karpenter-managed nodes instead of a legacy (non-Karpenter) NodePool.

Earlier Deployment Pinned to a Legacy NodePool

The deployment below is pinned to a legacy OKE node pool using a nodeSelector. Such workloads will never land on Karpenter-provisioned nodes until the selector is changed.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      # Legacy: explicitly targets the old autoscaled node pool
      nodeSelector:
        oke.oraclecloud.com/nodepool: legacy-worker-pool
      containers:
      - name: app
        image: myorg/my-app:latest
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"

Schedule on Karpenter Nodes

To allow a deployment schedule on Karpenter-managed nodes, you can:

• Keep the pod's nodeSelector and add a matching NodePool requirement (Option A)
• Or remove the legacy selector and use node affinity (Option B and Option C).

Option A: Keep the nodeSelector and Add a NodePool Requirement

If pods have a nodeSelector (for example, targeting an OKE pool name), Karpenter will not provision nodes for those pods until the NodePool satisfies that selector.

Add a matching requirement to the NodePool so that Karpenter provisions nodes with the same label.

Unchanged Pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      nodeSelector:
        oke.oraclecloud.com/pool.name: compute-head-4-16
      containers:
      - name: app
        image: myorg/my-app:latest
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"

Add a NodePool Requirement

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: oke.oraclecloud.com/pool.name
          operator: In
          values: ["compute-head-4-16"]
        # ... other requirements (arch, os, capacity-type, etc.)
      nodeClassRef:
        name: default
        kind: OciNodeClass
        group: karpenter.multicloud.sh
  # limits, disruption, etc.

Option B: Schedule on Any Available Karpenter Node

Remove the legacy nodeSelector. No selector or affinity is required. After scaling down the legacy pool, pods will schedule onto Karpenter nodes.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      # No nodeSelector: can schedule on any node, including Karpenter nodes
      containers:
      - name: app
        image: myorg/my-app:latest
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"

Option C: Prefer a Specific Karpenter NodePool

Use preferred node affinity to guide scheduling toward a specific NodePool or capacity type without making it mandatory. Consider an On-Demand NodePool, as shown in the example below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            preference:
              matchExpressions:
              - key: karpenter.sh/nodepool
                operator: In
                values:
                  - default
              - key: karpenter.sh/capacity-type
                operator: In
                values:
                  - on-demand
      containers:
      - name: app
        image: myorg/my-app:latest
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"

Summary

Step	Action
1	If you keep a pod nodeSelector (for example, oke.oraclecloud.com/pool.name), add the same key as a requirement on the NodePool so that Karpenter can provision compatible nodes.
2	Optionally, remove the nodeSelector (and legacy-only affinity) so that pods can schedule on any Karpenter node.
3	Optionally, add node affinity using karpenter.sh/nodepool or karpenter.sh/capacity-type to prefer specific NodePools.
4	Add tolerations for tainted NodePools (for example, nvidia.com/gpu for GPU workloads).
5	Ensure that your CPU, memory, and GPU requests fit the instance sizes offered by your NodePools.

After applying the updated manifest, gradually scale down the legacy node group so that scheduling demand shifts to Karpenter-provisioned nodes.