How to Measure CPU Ready in OpenShift Virtualization?

Everyone managing VMware understands what we mean when we talk about CPU Ready (%RDY) as a Performance Metric.

In this explanation, I will define what it is and how it is measured in OpenShift Virtualization.

What is CPU Ready?

The term "CPU Ready" can be misleading. One might think that it represents the amount of CPU ready to be used and that a high value is a good thing. However, the higher the CPU Ready, the worse the performance of the vSphere infrastructure, and the more applications will suffer.

Official VMware Definition:

It is the percentage of time a "world" is ready to run and waiting for approval from the CPU Scheduler.

In vSphere, a "world" is a process.

• The higher the CPU Ready, the more time VMs spend waiting instead of executing.
• In other words, a "world" (vCPU) is waiting to be scheduled on a physical CPU.
• CPU Ready measures the time a vCPU waits to be executed on a physical core.

What Causes High CPU Ready?

Identifying high CPU usage is easy, but determining the cause of CPU Ready can be more challenging.
The two main causes of high CPU Ready are:

1. High CPU Oversubscription
2. Use of CPU Limits

CPU Oversubscription

The most common reason for high CPU Ready is assigning more vCPUs than the physical CPUs can handle.

General rules for vCPU to pCPU ratios:

• 1:1 to 1:3 → No problem.
• 1:3 to 1:5 → Performance degradation may begin.
• 1:5 or higher → Likely CPU Ready issues.

How to View CPU Ready in VMware?

• The best way to analyze CPU Ready is at the VM level and per vCPU, not at the host level.
• In vSphere Client, you can add CPU Ready as a performance metric in the performance charts.
• In esxtop, you can view %RDY per VM.

What is a Normal CPU Ready Value?

• VMware recommends keeping CPU Ready below 5% per vCPU.
• CPU Ready is measured in milliseconds (ms) in the vSphere UI.
• To convert it to a percentage:
  • Example: If a VM has 2173 ms of CPU Ready in a 20-second period (20000 ms): (2173/20000)=10.87%(2173 / 20000) = 10.87\%
  • This is considered high and indicates a problem.

How to Measure CPU Ready in OpenShift Virtualization?

In VMware vSphere, CPU Ready (%RDY) measures the time a vCPU spends waiting to be scheduled on a physical CPU.

In OpenShift Virtualization, there is no direct "CPU Ready" metric, but the equivalent is measured through CPU Throttling.

📌 Equivalent Metric in OpenShift Virtualization

container_cpu_cfs_throttled_seconds_total

• Measures the total time (in seconds) that a container (or VM in OpenShift Virtualization) has been throttled, meaning it has been restricted in its CPU usage because it exceeded assigned resources.

📌 Other Related Metrics:

• container_cpu_cfs_periods_total → Total CPU periods allocated to the container.
• container_cpu_cfs_throttled_periods_total → Number of CPU periods in which the VM was throttled.

How to Monitor CPU Throttling in OpenShift?

📌 Using Grafana in OpenShift Monitoring

If you have Prometheus and Grafana configured, you can create a dashboard with:

• Metric: container_cpu_cfs_throttled_seconds_total
• Filter: {pod=~"my-vm-.*"} (to focus on specific VMs)

📌 Using oc adm top to Get Real-Time CPU Usage

Command:

oc adm top pods -n my-namespace

• Displays real-time CPU and memory usage for each VM.

🚀 Recommendations to Avoid CPU Throttling in OpenShift Virtualization

1️⃣ Avoid Unnecessary CPU Limits

• If a VM has a low CPU Limit, it can be constantly throttled, leading to high CPU Throttling.
• Instead, define only CPU Requests and leave the Limit open if the host has available resources.

2️⃣ Monitor and Adjust Assigned Resources

• Regularly review CPU Throttling metrics in Prometheus/Grafana.
• Ensure critical VMs have sufficient resources.

3️⃣ Avoid vCPU Overcommitment on Physical Nodes

• In OpenShift, assigning more vCPUs than the physical CPU capacity of the node can cause contention and throttling.
• Follow recommended vCPU to pCPU ratios.

📊 Thresholds for Interpreting CPU Throttling in OpenShift

🛠️ Quick Comparison: VMware vs. OpenShift Virtualization