Business Units and Application level Sustainability Dashboards using Aria Operations/vRealize Operations.
As always there is more to do, and having lot of meetings about this
subject with many VMware customers bring some new subjects/needs to the table.
One of this need is to get the information not only on VMs but also at
the Application and Business Units (group of applications) level. This is why these
next generation dashboards for sustainability are made, providing data like
scope2/scope3 CO2 emissions, visibility at the Business Unit/Applications/Datacenter/Cluster/ESXi
level and Business and Applications summaries like these ones:
1. Some definitions :
CO2 carbon intensity :
The Carbon Intensity of electricity is a measure of how much CO2
emissions are produced per kilowatt hour of electricity consumed. It is a way
to convert the power we measure to CO2 emissions.
Carbon intensity is measured in grams of carbon dioxide-equivalents
emitted per kilowatt-hour of electricity.
PUE (Power Usage Effectiveness) :
The PUE is an indicator developed by the Green Grid to measure the energy efficiency of a data center. It is calculated by dividing the total energy consumed by the data center by the total energy used by IT equipment (server, storage, network).
For example, for a datacenter that have a PUE of
2.5, which means that for 1 Watt consumed by IT, 2.5 Watts are needed at the
entrance to the datacenter.
Without being a sufficient indicator to determine the efficiency of a
datacenter, it is nonetheless a universally recognized indicator.
Scope2 and Scope3 : you can find a lot of definitions on the web (let’s take this one for instance : ).
Scope2 CO2 emission is the one which is generated by the host power
usage on which the VM is running (note the keyword is “running” here). We also
need to add the CO2 emissions for the total energy used to run the others
Datacenter elements for this VMs (storage, network, cooling, etc.), this is
where the PUE is interesting.
Scope3 CO2 emission is the one which is generated by the lifecycle of
the host itself (raw material extraction, manufacturing, transportation,
production, hardware disposition, …)
2. Setting Power Usage Effectiveness (PUE) and Carbon intensity Factor for the Datacenters :
First set the PUE and Carbon Intensity values for each group of Datacenter
(one group for all the Datacenters with the same values is better), these
values will be stored into a Custom Group because we don’t have direct way to
get them today and it is a nice way to store, centralized and easier to change
if needed (with or without the API).
Note :
Carbon Intensity is in grams of CO2 emissions per kWh, but as we have our CO2 emissions stored in kg and the power in Wh, we will need to divide it by 106 (either on the Carbon Intensity value or in the next formula).
Using the custom properties feature in the custom group, it will create
a custom property for each Datacenter named in the custom group.
Add your Custom Groups here : Environment->Custom
Groups->Add
Custom Group creation for the datacenters PUE and Carbon Intensity
Important : do not forget to select "Numeric" as the Custom Property Type for your custom group as it will be impossible (complicated) to change it once created.
These values will be easily accessible as seen now as metrics (under Environment->Object Browser->[your datacenter]->Metrics->Tags->Custom properties).
Custom properties created
Now let’s go to the maths part : what do we need to get all these calculated
data ? of course … supermetrics !
3. Create Supermetrics :
We are going to prepare supermetrics. These metrics will allow us to
calculate, store, and build dashboards based on new data. These supermetrics
will have a different “Assigned Object Types” depending on where we want the
metric to be stored as we would like to have them at Business Unit/Application/Datacenter/Cluster/host
level for some of them.
Note :
After a setup of supermetrics for objects, we
must choose the correct policies. if overlooked, the supermetrics will not be
applied to those objects.
Once done, it's normal to experience a little
delay, usually around one calculation cycle (around 5mn), before metric values
will become visible.
If you want to understand some calculations, here is some explanations/tricks :
- to correct the PowerUsage for the VM as there is a known bug due to a conversion issue in ESXi 7.0, https://kb.vmware.com/s/article/92639, the supermetric VMPowerUsageCorrected is created (in the future, or if you haven’t the issue, you can use the Total Energy metric for the 2 others supermetrics or just get rid of the / 1000).
- For the Scope2VMCO2, the hierarchical relationship of Aria operations is used to get the Datacenter’s PUE on which the VM is running (so doing a negative depth to go up to 3 levels of relationship, the sum is just a trick to access to the custom property value at parent level) :
- {This Resource: Super Metrics|Scope2VMCO2} * sum({Datacenter: Custom Properties|DatacenterPUE, depth=-3})
- Also we need to add the power usage of the ESXi host itself without any VM charge to the running VM, this power will be split across all running VMs on the host, this is the Scope3VMCO2. The average CO2 emission of 1,3T for the manufacturing, transportation and EOL of a server for its 4 years of life is a good value (Dell PowerEdge R640 as example, see : https://i.dell.com/sites/csdocuments/CorpComm_Docs/en/carbon-footprint-poweredge-r640.pdf, the CO2e is 17% of the total 7,7T of CO2e).
Estimated
product carbon footprint distribution for Dell PowerEdge R640
- As the given server’s life is 4 years (365.25 x 5 days, so 365.25 x 4 x 24 x 12 = 420 768), so there is 420 768 slices of 5mn on which the Scope3 CO2e for the server life will be allocated. 5mn is the sampling time to store the metric for the Scope3 on the VMs (checking if the VM is running to apply the weight of the scope3 is a good thing too).
- In the future versions, the idea could be to have this info stored at the ESXi level and do a better accurate calculation using info coming from the manufacturer itself (and also when the info from the manufacturer will become more accurate as today some have 100% deviation possible).
All calculations are performed at the VM level and then aggregated to higher tiers. With our supermetrics in hand, our next step is to present them in informative & relevant dashboards.
4. Dashboards and Views :
When we design dashboards and views, prioritize simplicity. By creating
fewer objects, we make management and the understanding easier.
There has been customer request to see both the current value and a
month-to-date (MTD) value, this is why the view to get the MTD (as it is not
possible to do it using a standard widget) has been created.
For step by step details about Month To Day View creation, refer to this other article on my blog : https://cloudetastuces.blogspot.com/2023/08/how-to-create-month-to-day-mtd-views-in.html.
Month to Day (MTD) View for VMs CO2e
The dashboards are built the same way to have visual consistency and are
provided with the power and CO2e versions for combination of them.
The next question is how to group our VMs into applications and our
applications into business units as this is the key here: being able to have an
Application and Business Unit view of our CO2 emissions/power usage ?
5. Custom Groups :
As we can’t populate the applications automatically here (and of course
not the Business units), we will do that manually (investigating how to do that
using Aria Operations for Networks or API could be interesting for the next
versions).
The tricky part is to use Group Types to be able to categorize our business
units and applications custom group, it will be useful in the selectors and all
the widgets we will use in the dashboards (as we don’t want to select only one
app our business units but all of them).
Just create 2 Group Types : Business_Unit and Application to group the
new Custom Groups into the same category (useful for filtering and building the
views/widget in the dashboards), go to
Note :
One interesting thing to do also is to create an "All Business Units" Custom Group to group all the business units in one group to have a global visibility. You just have to add the applications using the membership criteria as we now have the custom group type to filter on.
6. Conclusion
After setting up groups, and importing the supermetrics, views and dashboards, the new dashboards show insights from
Applications, Business Units, and IT infrastructure.
The only thing you now have to do is to add your application and business units groups (and some Datacenter configurations if needed).
Of course, there is a lot to add and to enhance, and that will come in the
next releases.
7. Resources
These dashboards can also be downloaded from the VMware Developer Exchange. All included assets are
bundled in and can be installed in the customer environment. The developer
exchange page also provides detailed installation instructions.
8. Acknowledgements
Thanks to Vincent Meoc, Bengt Gronas and Thomas Kopton who helped me discover the Sustainability world and corrected my article and designs.
9. References and sources :
- Supermetrics, Views and Dashboards :
- These sources are available here : Sources just upload and import them in your Aria Operations (may be you’ll encounter the known issue for the supermetrics : https://kb.vmware.com/s/article/2126552, but it will work anyway).
Note :
if you import each supermetric in the
precedence order and wait the data to appear (usually 5mn after creation max),
then you can import the next one without naming issues in the formula).
- using API to create Custom Groups :
- Create vRealize Operations Manager custom groups with REST API - Cloud Advisors
- Create Multiple Custom Groups using vROps REST API with vCenter Tags - Samples - VMware {code}
- Carbon intensity :
- https://ourworldindata.org/grapher/carbon-intensity-electricity
- https://www.nationalgrid.com/stories/energy-explained/what-is-carbon-intensity
- Scope definitions and details :
- https://boavizta.org/en/blog/empreinte-de-la-fabrication-d-un-serveur
- https://go.microsoft.com/fwlink/p/?linkid=2161861
- Dell servers carbon footprint info :
Commentaires
Enregistrer un commentaire