In a Private Cloud, there should be at least two SLAs: one for Availability, and one for Performance. The Availability SLA measures “downtime”. The Performance SLA measures “slow time”.

You could also consider an SLA for compliance. VCF Operations helps to track compliance of an environment against a number of predefined compliance baselines. We think the value of a compliance SLA, if one exists, should be 100%. In terms of Availability, or Performance, we know 100% is not realistic because they affected by factors like platform cost and efficiency. With compliance, however, we don’t believe there is any cost or efficiency justification for not keeping on top of compliance. We will not devote further time in this whitepaper to the subject of a compliance SLA, but this is an area that may be expanded upon and refined in future.

Measuring SLAs

SLAs should be measured over a monthly period. A monthly period for SLAs is industry standard, and allows for easier comparison against benchmarks and peers. Be aware also that a shorter (or longer) period of measurement can have a significant effect on the duration you have to fix a problem before the SLA is breeched.

SLA Leading Indicators

Adjacent to the SLO is the SLA Leading Indicator (SLI). Where the SLO (and thereby, the SLA) shows the customer the promise you are making, the SLI shows you the reality of how your platform is actually performing. It is a leading indicator of service quality because it shows you the current, real-time view of your environment. The SLA is a lagging indicator of service quality because it’s a counter of the average over the last month.

If the leading indicator is worse than the SLO, then your service runs the risk of violating its SLA. By providing insight into how the service is comparing to its SLA, it will give you an indication about whether you will meet your SLA at the end of the month.

Availability SLA

The Availability SLA measures downtime for a VM. As mentioned earlier, the Availability SLA may appear to be the simplest to measure because it’s binary. This depends a lot upon how and where you measure it. For example, if the only requirement in your business is that a VM is powered on, then it’s easy to measure. If you need to consider whether the network is functional, or that the guest OS is running, then it can become more complicated. You may also need to exclude scheduled changes for patching or reboots from the SLA measurement.

The majority of our customers manage the guest OS for their internal customers as part of their VMware service, therefore we recommend measuring availability via ping packets, which verify that the OS is running and available on the network.

When setting the SLO for Availability, keep in mind that achieving high levels of availability carry architectural and cost implications. A SLO of 99.999% uptime equates to 26 seconds of downtime per month, which is likely to require factors such as hardware with more redundancy or specific architectural configurations that are not necessary to support a 99.9% SLO (which equates to 43 minutes of downtime per month).

When documenting your availability SLA, make sure that your SLO and exclusions (i.e. what events are not measured towards downtime) are clearly defined.

Performance SLA

The performance SLA measures the performance of a VM across its four compute resources: CPU, memory, disk, and network. Performance problems in any one of these areas could cause the application to become unusably slow and cause the SLA to breech, so we need an indication of performance per-workload that is more complex than simply averaging the values of relevant performance metrics for each compute resource.

VCF Operations contains a built-in SLI metric which tracks a key metric for each of CPU, memory, Disk, and Network, on a per workload basis.

Figure 6: Breakdown of the Service Level Indicator metric in VCF Operations

This metric summarizes whether each of the metrics above is under or over threshold on a per-VM basis. On each VM, the value of the SLI metric will be between 0-4, with the number incrementing for each threshold breeched.

These values for performance are built into VCF Operations as a Service Level Indicator because we believe they are the best metrics for providing a reliable insight into VM performance.

We recommend measuring the performance SLA off these metrics. The performance SLA is a promise about the percentage of time per month that a VM’s performance will be at or under these thresholds (measured over 5-minute intervals).

One additional note: we don’t address multiple classes of service (e.g. gold, silver, bronze that have different SLAs) in this whitepaper. If you were to pursue different classes of service, we recommend that the threshold for performance above should be the same for each class. The differentiator between each class (from an SLA perspective) is the percentage of time that the VM may exceed the threshold within the 30-day period. If you instead provide class-based differentiation using different thresholds, this can result in a “better”, more expensive class receiving a lower percentage-based score than a lower class, even though the higher class’s actual performance might still be better. This is confusing and unclear for both users and operations teams.

Operations Management

Operations management refers to the processes and procedures which form operational capabilities required to run the IT infrastructure and provide service to the business.

While operations management spans multiple areas, the core is based upon three fundamental pillars: performance management, capacity management, and cost management.

These pillars of performance, capacity, and cost are interrelated. As you develop and mature your operational capability in one area, it will provide benefits and solve problems in the others. These pillars are also intrinsically linked and mutually supporting: more performance requires more resources, and more resources costs more money and consumes available capacity.

We have talked in earlier sections about availability, but availability is not something you manage directly. Availability (or lack thereof) is an outcome of configuration and capacity which you manage, as well as architectural decisions, and factors you don’t control such as hardware failures.

Self-service is a key part of providing a cloud-like service to your customers, and in this context these three pillars are even more important. When we talk to Broadcom customers who are hesitant about providing self-service to their own customers, a lot of the resistance comes down to the traditional mentality that IT are the gatekeepers of the IT infrastructure and play an irreplaceable role in ensuring that end-customers don’t consume at a pace which outstrips IT’s ability to manage capacity. We agree that governance around self-service is crucial, but effective cost management solves a big part of this capacity management challenge because your internal customers will have a new barrier to overprovisioning resources: their wallets.

As part of building operational maturity, you’re not just developing new capabilities but also refining existing ones and moving them from reactive and manual to proactive and automated. This maturation is important to provide the benefits around efficiency of both infrastructure and people that you need to run an efficient Private Cloud.

Figure 8: Cloud operations maturity phases

Performance Management

Overview

When a customer complains that their application or workload is slow, what do they mean?

Without an objective definition of “good” performance, discussions about slowness are mired in subjective language and relative expectations. We often try to solve the problem by providing more resources and hoping the problem goes away.

Put simply, performance is about providing workloads with the necessary compute resources. In the physical world, this is ensuring a workload has enough CPU, memory, storage, and network available to satisfy workload requirements. In the virtual world, where physical resources are abstracted and shared between multiple workloads, we also need to consider contention for physical resources between multiple virtual workloads: i.e., can the workload use the resources it has been given, or are the physical resources already consumed by other workloads running on the platform?

Performance troubleshooting is a complex topic when you consider that application performance is really a combination of the performance of several layers: the application itself, the virtual machine’s OS, the virtualization layer, and the physical layer (compute, storage, networking). This does not consider multi-tier applications with Kubernetes, or issues on the end user’s side, which increase the complexity further.

These layers can be broadly grouped into two buckets: application and infrastructure.