Business Units and Application level Sustainability Dashboards using Aria Operations/vRealize Operations.

le
Today, sustainability is a big concern for all of us and this is also the same for many VMware customers. We introduced our first sustainability dashboards in Aria Operations in 2021 and lot of customers were very happy about this first approach.

As always there is more to do, and having lot of meetings about this subject with many VMware customers bring some new subjects/needs to the table.

One of this need is to get the information not only on VMs but also at the Application and Business Units (group of applications) level. This is why these next generation dashboards for sustainability are made, providing data like scope2/scope3 CO2 emissions, visibility at the Business Unit/Applications/Datacenter/Cluster/ESXi level and Business and Applications summaries like these ones:  

Business Units and Applications CO2 Emissions dashboard

Datacenters CO2 Emissions

1. Some definitions :

CO2 carbon intensity :

The Carbon Intensity of electricity is a measure of how much CO2 emissions are produced per kilowatt hour of electricity consumed. It is a way to convert the power we measure to CO2 emissions.

Carbon intensity is measured in grams of carbon dioxide-equivalents emitted per kilowatt-hour of electricity.

Carbon Intensity example for some countries

PUE (Power Usage Effectiveness) :

The PUE is an indicator developed by the Green Grid to measure the energy efficiency of a data center. It is calculated by dividing the total energy consumed by the data center by the total energy used by IT equipment (server, storage, network). 

For example, for a datacenter that have a PUE of 2.5, which means that for 1 Watt consumed by IT, 2.5 Watts are needed at the entrance to the datacenter.

Without being a sufficient indicator to determine the efficiency of a datacenter, it is nonetheless a universally recognized indicator.

Scope2 and Scope3 : you can find a lot of definitions on the web (let’s take this one for instance : ).

Scope2 CO2 emission is the one which is generated by the host power usage on which the VM is running (note the keyword is “running” here). We also need to add the CO2 emissions for the total energy used to run the others Datacenter elements for this VMs (storage, network, cooling, etc.), this is where the PUE is interesting.

Scope3 CO2 emission is the one which is generated by the lifecycle of the host itself (raw material extraction, manufacturing, transportation, production, hardware disposition, …)

Product life cycle steps that result in Scope 3 emissions

2. Setting Power Usage Effectiveness (PUE) and Carbon intensity Factor for the Datacenters :

First set the PUE and Carbon Intensity values for each group of Datacenter (one group for all the Datacenters with the same values is better), these values will be stored into a Custom Group because we don’t have direct way to get them today and it is a nice way to store, centralized and easier to change if needed (with or without the API).

Note :

Carbon Intensity is in grams of CO2 emissions per kWh, but as we have our CO2 emissions stored in kg and the power in Wh, we will need to divide it by 106 (either on the Carbon Intensity value or in the next formula). 

Using the custom properties feature in the custom group, it will create a custom property for each Datacenter named in the custom group.

Add your Custom Groups here : Environment->Custom Groups->Add

Custom Group creation for the datacenters PUE and Carbon Intensity

Important : do not forget to select "Numeric" as the Custom Property Type for your custom group as it will be impossible (complicated) to change it once created.

These values will be easily accessible as seen now as metrics (under Environment->Object Browser->[your datacenter]->Metrics->Tags->Custom properties).

Custom properties created

Now let’s go to the maths part : what do we need to get all these calculated data ? of course … supermetrics !

3. Create Supermetrics :

We are going to prepare supermetrics. These metrics will allow us to calculate, store, and build dashboards based on new data. These supermetrics will have a different “Assigned Object Types” depending on where we want the metric to be stored as we would like to have them at Business Unit/Application/Datacenter/Cluster/host level for some of them.

Note :

After a setup of supermetrics for objects, we must choose the correct policies. if overlooked, the supermetrics will not be applied to those objects.
Once done, it's normal to experience a little delay, usually around one calculation cycle (around 5mn), before metric values will become visible.

 Here is the list of all the metrics used and their meanings/usage :

Supermetrics list and details

Formulas details for Power
Formulas details for CO2e

If you want to understand some calculations, here is some explanations/tricks :

  • to correct the PowerUsage for the VM as there is a known bug due to a conversion issue in ESXi 7.0, https://kb.vmware.com/s/article/92639, the supermetric VMPowerUsageCorrected is created (in the future, or if you haven’t the issue, you can use the Total Energy metric for the 2 others supermetrics or just get rid of the / 1000).
  • For the Scope2VMCO2, the hierarchical relationship of Aria operations is used to get the Datacenter’s PUE on which the VM is running (so doing a negative depth to go up to 3 levels of relationship, the sum is just a trick to access to the custom property value at parent level) :
    • {‌This Resource: ‌Super Metrics|Scope2VMCO2} * sum({‌Datacenter: ‌Custom Properties|DatacenterPUE, depth=-3})
  • Also we need to add the power usage of the ESXi host itself without any VM charge to the running VM, this power will be split across all running VMs on the host, this is the Scope3VMCO2. The average CO2 emission of 1,3T for the manufacturing, transportation and EOL of a server for its 4 years of life is a good value (Dell PowerEdge R640 as example, see : https://i.dell.com/sites/csdocuments/CorpComm_Docs/en/carbon-footprint-poweredge-r640.pdf, the CO2e is 17% of the total 7,7T of CO2e).

Estimated product carbon footprint distribution for Dell PowerEdge R640

    • As the given server’s life is 4 years (365.25 x 5 days, so 365.25 x 4 x 24 x 12 = 420 768), so there is 420 768 slices of 5mn on which the Scope3 CO2e for the server life will be allocated. 5mn is the sampling time to store the metric for the Scope3 on the VMs (checking if the VM is running to apply the weight of the scope3 is a good thing too).

    • In the future versions, the idea could be to have this info stored at the ESXi level and do a better accurate calculation using info coming from the manufacturer itself (and also when the info from the manufacturer will become more accurate as today some have 100% deviation possible).

All calculations are performed at the VM level and then aggregated to higher tiers. With our supermetrics in hand, our next step is to present them in informative & relevant dashboards.

4. Dashboards and Views :

When we design dashboards and views, prioritize simplicity. By creating fewer objects, we make management and the understanding easier.

There has been customer request to see both the current value and a month-to-date (MTD) value, this is why the view to get the MTD (as it is not possible to do it using a standard widget) has been created.

For step by step details about Month To Day View creation, refer to this other article on my blog : https://cloudetastuces.blogspot.com/2023/08/how-to-create-month-to-day-mtd-views-in.html.

Month to Day (MTD) View for VMs CO2e

The dashboards are built the same way to have visual consistency and are provided with the power and CO2e versions for combination of them.

The next question is how to group our VMs into applications and our applications into business units as this is the key here: being able to have an Application and Business Unit view of our CO2 emissions/power usage ?

5. Custom Groups :

As we can’t populate the applications automatically here (and of course not the Business units), we will do that manually (investigating how to do that using Aria Operations for Networks or API could be interesting for the next versions).

The tricky part is to use Group Types to be able to categorize our business units and applications custom group, it will be useful in the selectors and all the widgets we will use in the dashboards (as we don’t want to select only one app our business units but all of them).

Just create 2 Group Types : Business_Unit and Application to group the new Custom Groups into the same category (useful for filtering and building the views/widget in the dashboards), go to Environment->Custom Groups->Group Types->Add.

Now we can create all our applications, including the VMs we need using either the membership criteria or a direct inclusion, go to Environment->Custom Groups->Add.

Important : do not forget to select "Application" as the Group Type for your application custom group.

Application Custom Group Creation

And after Applications creations we can add them into our business units custom groups using the "object to always include" option to select them.

Important : do not forget to select "Business Unit" as the Group Type for your Business Unit custom group.

Business unit custom group creation

Note :

One interesting thing to do also is to create an "All Business Units" Custom Group to group all the business units in one group to have a global visibility. You just have to add the applications using the membership criteria as we now have the custom group type to filter on.

All Business Units Group creation

6. Conclusion

After setting up groups, and importing the supermetrics, views and dashboards, the new dashboards show insights from Applications, Business Units, and IT infrastructure.

The only thing you now have to do is to add your application and business units groups (and some Datacenter configurations if needed).

Of course, there is a lot to add and to enhance, and that will come in the next releases.

7. Resources

These dashboards can also be downloaded from the VMware Developer Exchange. All included assets are bundled in and can be installed in the customer environment. The developer exchange page also provides detailed installation instructions.

8. Acknowledgements

Thanks to Vincent Meoc, Bengt Gronas and Thomas Kopton who helped me discover the Sustainability world and corrected my article and designs.

9. References and sources :

  • Supermetrics, Views and Dashboards :

Note :

if you import each supermetric in the precedence order and wait the data to appear (usually 5mn after creation max), then you can import the next one without naming issues in the formula).

Import order for supermetrics

Your turn now !

Commentaires

Enregistrer un commentaire