Overview

A cluster is operationally a collection of ESXi hosts. As a result, the basic counters of CPU, memory, disk and network are basically the sum of the member host.

vSphere Cluster

Think of vSphere Cluster as the smallest logical building block. From operations management, it’s basically a single computer. Think of a huge and complex machine.

vSphere Cluster is much more than just a group of ESXi hosts sharing a common network and storage. What makes a cluster more complex than the sum of its hosts is the various cluster-level features and configuration.

Let’s start by looking its 2 most basic features:

You can see that the above complicate operations, especially in a very large environment with hundreds of clusters. If you add these features on top, you further increase complexity of your operations.

The above cover the standard vSphere cluster. There are 2 other variants, which take the operational complexity higher.

In addition, there are complication simply because there are multiple members in the cluster. For example, is cluster utilization simply the average of all its hosts? What if there is imbalanced? It will get buried if the cluster has many hosts.

While a cluster focuses on compute, it is where VM runs and consumes network and storage. This means network and storage counters must be considered as appropriate. If you’re using vSAN, then it’s mandatory.

Base Metrics

vSphere Client only displays basic set of metrics. They are grouped into 4, as shown in the following screenshot:

For each of the group, there is basic set of metrics. Here it is for memory:

The group Cluster Services only provides 3 metrics:

VM Operations

vSphere Cluster, being the main object where VM runs, has a set of event metrics. They count the number of times an event, such as a VM gets deleted, happens. This provides insight into the dynamics of the environment.

Take note that the metric is accumulative. So it starts since the day the cluster was created. VCF Operations converts into rate, and also make them available at higher level objects (Data Center, vCenter and vSphere World).

You certainly have some expectation on the dynamics of your environment. Does the reality match your expectation?

In production environment, these numbers should be low. Some numbers such as shutdown should also match the change request and happens during the green zone. Some exceptions apply, such as your VDI design includes scheduled reboot on the weekend.

Imbalance

Imbalance is the root cause behind why contention happens despite overall utilization has not reached 100%.

There are 3 types of imbalances in a vSphere Cluster.

Inter-Host Imbalance

You want clusters to be well utilized but balanced across all hosts in the cluster.

• High Utilisation 🡪 Performance risk.

• Low Utilization 🡪 Wastage capacity.

The cluster is imbalanced when there are High ESXi hosts and Low ESXi hosts.

Focus is imbalance at high utilization. Imbalance can happen at low utilization. While that’s mathematically true, it’s not operationally important.

The formula is basically High Hosts / Low Hosts.

Let’s visualize using a cluster of 6 hosts as an example:

Quantification Strategy

The formula uses a relative number, so we can compare across clusters of different sizes.

Looking at the values, we can see that each color falls within some specific ranges, that is intentionally designed not to overlap.

We can now translate the color into numbers.

Limitation

Summarizing always results in the loss of details.

For example, we don’t know if it’s 1 High Host over 2 Low Host, or 4 hosts over 8 hosts as both returns 0.5.

Alternatively, we can concanetate the numbers. So we get 1.2 and 4.8.

• Advantage: we know 1.2 is different to 4.8

• Disadvantage: the sort across many numbers will be less meaningful as the values are no longer relative.

Example

Let’s use a 12-node cluster as an example.

The following shows the complete permutation of the 169 possible Cluster Imbalance values.

Intra-Host Imbalance

You want all cores used first, before threads are sharing a core.

Since HT is dual thread, the worst score is 2.0.

When there are idle cores, no core should be running 2 threads. Since real world is not perfect world, the diagram shows gradient. It changes from red to green as Core Utilization hits 100%.

Imbalance Score of 1.5 at means half the cores are running 2 threads.

For an ESXi with 64 cores, 128 threads, a score of 1.5 at 40 cores means:

• 20 cores are running 2 threads

• 20 cores are running 1 thread

• 24 cores are idle.

Ideally, it should be 60 cores running 1 thread each.

The imbalance should be used together with Core Utilization.

ESXi CPU Imbalance = Count of Running Threads / Count of Running Cores

vSphere Cluster ESXi CPU Imbalance = Average (ESXi CPU Imbalance)