Log in

Sangfor Support Center

Table of Contents
< All Topics
Print

vcjob or pod scheduling fails when configured with numa topology policy

Posted
Updated
Byadmin
Views1

Problem Description

The CPU management policy for the k8s cluster is set to "default policy," while the topology management policy is set to any policy other than "default policy."

When creating a vcjob or pod with a numa topology policy through the "Create YAML" interface or by downloading the kubeconfig and using kubectl, the pod will fail to be scheduled.

Alert Information

None

Effective Troubleshooting Steps

Examine the volcano-scheduler logs using the command kubectl logs -f -l app=volcano-scheduler -n ske-system. There will be the following content highlighted in red:

Root Cause

The reason for the issue is that the node's CPU management strategy is set to none, whereas the topology management strategy is not set to none. If a pod has a topology strategy that is not set to none, volcano-scheduler will attempt to schedule the pod to a node with a CPU management strategy of static. If the node's CPU management strategy is not static, it will be considered unsuitable for scheduling.

Summarized conditions for unscheduling:

  • The pod is of type PodQOSGuaranteed
  • The pod has a configured topology strategy
  • Either the node's CPU management strategy is not static or the node's topology strategy does not match the pod's

Solution

Solution 1: Remove the numa topology strategy from the vcjob/pod configuration.

Solution 2: Modify the CPU management strategy to static, or change the topology management strategy to "default policy."

Scope of Operation Impact

Solution 1: Affects only the vcjob/pod being edited.

Solution 2: Affects the entire cluster being edited.

Is This a Temporary Solution?

No.

Recommendations and Summary

  • Typically, resource management policies are not set up this way (CPU management policy set to default, topology management policy set to something other than default). If someone has configured this way, inquire about the reasoning behind it.

Investigation Content

NA

Original Link

https://support.sangfor.com.cn/cases/list?product_id=37&type=1&category_id=28614&isOpen=true