GB2485526A - Modeling data centre power consumption to determine if a physical server can be virtualized - Google Patents

Abstract

The power consumption of a data centre including a physical infrastructure with physical servers is estimated, non-intrusively without having to install additional hardware and/or software in the data centre. Software specification data is then used to determine if a physical server can be virtualized. If it is determined that it can, a model of the virtualized environment is used to estimate the new level of power consumption and is compared to the power consumption of the physical infrastructure. A proposed virtualization can be tested using a test bed, for example to test stability, compatibility and power consumption. At least some degree of virtualization of the data centre can then be carried out if required. Estimating the power consumption may include the power consumption of cooling apparatus. The proposed method may also estimate the carbon dioxide emissions of the data centre.

Description

VIRTUALISATION EMULATION METHOD

The present invention relates to a virtualisation emulation method.

The Focus on Green IT A United States physicist recently claimed that two search requests on Google produce as much carbon dioxide as boiling a kettle. See: BBC News. (2009, January 2009). "Carbon cost" of Google revealed. Available at BBC News Channel: http://news.bbc.co.uWllhi'7823387.stm (Accessed: 1 November, 2010).

The academic, Dr Alex Wissner-Gross, argues that Google operates large data centres around the world that add up to a lot of energy usage. The research paper added to the awareness of the demands made on the environment by modern day IT data centres.

The focus on green IT has increased considerably in recent years. Reducing carbon emissions can improve a company's reputation as a socially responsible business. It can also help to attract customers, investors and employees.

In August 2007, the United States environmental protection agency (EPA) compiled an extensive report to the Congress on server and data centre energy efficiency. See: Energy Star Program. (2007). Report to Congress on Server and Data Center Energy Efficiency, Public Law 109 -432. Energy Star.

The report focussed on the costs of IT data centres and servers and the power and cooling infrastructure that are required to maintain it. It found that data centres in the U.S. consumed 61 billion kWh of electricity in 2006. This represented 1.5% of all U.S. r 2 electricity consumption and was double the amount consumed in 2000. The EPA estimated that, based on current trends, the energy consumed by data centres will continue to grow by 12% per year.

In the United Kingdom, data centre energy efficiency is also widely encouraged. In September 2009, the British Computer Society (BCS) launched a new data centre related qualification called "Foundation Certificate in Green IT". See: BCS. (2009, September 21). BCS Launches Green IT Qualifications. Available at: BCS The Chartered Institute for IT: http://www.bcs.orglserver.php?show=corsWebDoc.32254 (Accessed: 1 November, 2010).

BCS designed the qualification to help iT professionals and corporations gain an in-depth understanding of the environmental issues associated with IT. This qualification is based on the EU Data Centre Code of Conduct and provides a framework to run an energy-efficient data centre. It is proving to be exceedingly popular with IT Professionals Underutilisation of Powerful Hardware Many factors contribute to the excessive energy consumption of data centres, but the underutilisation of x86 hardware is the most significant cause. Virtualisation has the potential to offer considerable energy savings.

A 2007 EPA report cites a personal communication by Jay Dietrich, the Program Manager at IBM's Corporate Affairs Group. See: Energy Star. (2007, August 2). Energy Star. Available at: Energy Star http:I/www.energystar.gov/index.cfm?c=prod development sen'er efficiency (Accessed: 1 November, 2010). r

In the communication, Dietrich states that volume servers typically operate at a processor utilisation level of only five to fifteen percent. The utilisation levels have a minimal effect on the power consumption of servers. At such low utilisation levels the server will still consume between 60 to 90% of its maximum system power, resulting in vast amounts of wasted energy.

Qatimising Hardware Utillsation Both the BCS and the EPA detail possible solutions to reduce the IT sectors carbon footprint. Server consolidation and virtualisation are cited as likely options that should be considered when reduction in energy consumption is considered. The BCS has compiled a "Data centre energy efficiency metrics" and states that a...the use of virtualisation technologies can also provide significant benefits in reducing the overall iT equipment power draw as well as the provisioned power to IT equipment'. See: Newcombe, L. (2009). Data Center Energy Efficiency Metrics. Available at: DatacenterDynamics: http://www. datacenterdynamics.com/MedialFormscontacg'Form/Datacenfreenegyeffi ciency_metrics.pdf (Accessed: 1 November, 2010).

Numerous case studies have proven that server virtualisation has the potential not only to reduce energy consumption, but also to save the company money.

In 2008, the Royal Borough of Windsor and Maidenhead (RBWM) reviewed its server infrastructure. The results of the review showed that the data centre capacity was at a maximum and the hardware was reaching the end of its five year lifecycle. The council employed a virtualisation specialist firm to implement virtualisation technology.

The council's energy bill was reduced by 44% following the migration to a virtual infrastructure. The head of ICT at RBWM, Keith Clark, also predicts a saving of r approximately £1.2 million on the project over a five-year period. This is as a result of the increased efficiency and productivity that the virtualisation brought to the council's IT infrastructure. See: Government Technology. (2009). Virtualisalion to save council £1.2M. Available at: Government Technology: http:/Iwww.governmenttechnology.co.uk/content/view/1572133/ (Accessed: October 8, 2009).

In August 2005, the energy company E.ON migrated its existing physical iT infrastructure to a virtualised IT infrastructure. The company required a flexible environment with the ability to accommodate all the individual 11 requirements of the separate business units. A virtual environment met these requirements. Following the move to virtualisation, EON reported a 56% reduction in power consumption. The heat in the data centre was also reduced by more than half. See: Simpson, G. (2007, April 23). Virtuailsation powers up energy group. Available at: Sillcon.com: http:/Ihardware.sillcon.com/storage/0,39024649,39166826,00.htrn (Accessed: I November, 2010).

In addition to the reduction of energy consumption, companies also report that a virtual infrastructure is easier to manage effectively. In his article "Virtual entropy" Pete Swabey reports that UK retailer Tesco is currently in the process of virtualising 1,500 physical servers onto 120 servers. IT director, Nick Folkes, explains that the virtual infrastructure is more effective and simpler to manage. See: Swabey, p. (2009, April 9).

Information Age. Virtual entropy, pp. 9-Il.

The "Virtual entropy" article discusses another benefit of virtualisation; the fact that a reduction in servers occupies less floor space in a data centre. The pension company Standard Life virtualised 70% of its physical servers. This enabled it to decommission r 143 of its 400 machines. Following the virtualisation, the occupied floor space was reduced from 1,400 square meters to 500 square meters. In addition, the data centre's annual energy consumption was reduced from 11 million kWh to 7 million kWh, resulting in a saving of £300,000.

Well-planned and suitably executed virtualisation brings about a more efficient data centre. It enables IT staff to respond more rapidly to a company's changing business needs and computing requirements. Responding to IT load demands in a physical IT infrastructure has proven to be a challenge. The static nature of a physical infrastructure makes it difficult to accommodate peak usage. A virtual infrastructure can accommodate the fluctuations in the IT load more dynamically.

Virtualisation Techno!oqy Virtualisation software makes it possible to run a number of virtual machines on a single physical machine. VMware gives an overview of virtualisation and the history that led up to the popularity of this technology. See: VMware. (2010, March). History of Virtuailzation. Available at VMware: http:I/www. vmware.com/virtualization/history.htm! (Accessed: 1 November, 2010).

In the 1960s, IBM implemented virtualisation as a way to partition mainframe computers. However, in the 1980s and 1990s the popularity of virtualisation diminished as more affordable hardware became available. Hardware continued to become more affordable and at the same time more powerful, and the utilisation levels dropped to around 10% and below. The increased focus on better hardware utilisation, while at the same time reducing the carbon footprint, revitalised the need for virtualisation technology. V 6

Various software packages are available on the market and VMware is one of the more popular packages available, with over 170,000 customers.

Virtualisation software creates an abstract layer between the virtual machine and the host operating system. This abstract layer, known as a "hypervisor", acts as a controller between the hardware and the virtual machines. Its function is to monitor the virtual machines, dynamically assigning the hardware to the virtual machines as and when the hardware is required.

Virtualisation software is based on one of three fundamental technologies, namely full virtualisation, para-virtualisation and operating system partitioning. McAllister describes the three technologies as follows (see: McAlllster, N. (2010, March). Server virtualization under the hood. Available at: Info World: htlp://www.infowodd.com/d/virtuailzation/seri, 'er-virtua/jzatjon-uncjer-hood-147 (Accessed: 1 November, 2010)): Full virtualisation Referring to FIG. 1, full virtualisation is believed to be the most popular form of virtualisation. With this technology, nearly all operating systems can be installed on a virtual machine without the need for modification. The hypervisor traps Cpu instructions and controls the virtual machine's access to the hardware. The operating system is not aware that it is running in a virtual environment and that its access is being controlled by the hypervisor. High processor overhead can be a disadvantage in some cases.

Para-virtualisation Referring to FIG. 2, in order to overcome the high processor overhead that is created in the full virtualisation technology, the operating system can be modified to r make it aware that it is running on a virtual machine. The virtual machine is then aware of the hypervisor and collaborates with the hypervisor to reduce the overhead on the processor.

Operating System Partitioning Referring to FIG. 3, some operating systems can include their own form of virtualisation at the kernel. If virtual servers are all running the same operating system, then they can be partitioned to be isolated from one another. This can yield performance in the virtual environment that is very similar to that of a physical environment.

Next, a possible methodology for calculating the power requirements of a data centre will be described.

Calculating data centre reqyirements important elements to consider ln his book "The Green and Virtual Data Centre", Greg Schulz identifies the following four elements as the primary issues that should be addressed: Power, cooling, floor space and environmental (PCFE). See: Schulz, G. (2009). The Green and Virtual Data Center. Boca Raton: Auerbach Publications.

FIG. 4 shows the typical energy consumption of the most commonly used data centre components. Focus currently centres on increased performance while reducing cooling requirements and power consumption in the data centre. However, there is a constant demand for improved computing, networks and storage resources. These demands mean that effective utilisation of the available PCFE resources is a concern. r 8

Energy consumption Case studies have shown that it is not always feasible to measure the exact energy consumption of data centre components. Numerous aspects play a role when elements of a data centre are to be assessed. There are security issues e.g. arranging access for staff to carry out surveys in the data centre. In many cases the data centre's consumption is measured and billed as part of a company's total energy bill, making it difficult to establish how much the data centre contributed to the energy usage.

Some vendors ship their server enclosures with intelligent monitors already installed. These monitors provide a multitude of administrative information and can report on real time temperature, power consumption and processor utilisation. See: Hewlett-Packard Development Company. (2009, September). HP i9ladeSystem Onboard Administrator. Available at: HP: http://h20000. www2.hp.corn/bc/docs/supporuSupportManual/c00705292/000705292.pdf (Accessed: 1 November, 2010).

Richard Sawyer, Senior Systems Application Engineer for American Power Conversion (APC) published a whitepaper titled "Calculating Total Power Requirements for Data Centres". See: Sawyer, R. (2004). American Power Conversion. Available at: Calculating Total Power Requirements for Data Centers: http://www.searchstorageasia.com/system/files/whitepaper/APC CalculateTotalPower.p df (Accessed: December 11, 2009) Sawyer's paper discusses factors that should be taken into consideration when calculating the power requirements for a data centre. Underestimating the power requirements can lead to power disruptions and overestimation leads to unwarranted installation cost and higher maintenance costs. Sawyer states that studies have shown r that nameplate ratings on IT devices are greatly in excess of the actual running load.

This is because nameplate ratings cater for worst-case power consumption.

Sawyer suggests that IT professionals use a power consumption calculator to approximate the data centre's power requirements. These calculators collect power consumption data from a wide range of IT equipment manufacturers.

FIG. 5 is an example of an APC calculator and the type of information that the user is required to record. The server's make and model, along with information about some components such as the CPUs and hard drives is required.

Referring to FIG. 6, based on the user's input, the calculator will state the power.

This is then used to calculate the Kilowatt-hour as shown in Equation A below.

aWatts * hours used kWh-I--- 1000 (Equation A) In the example, the server draws 348 Watt. If it is actively used for an hour, it will consume 0.348 kWh. (348W -1 000 = 0.3kWh) Carbon Dioxide (C02) emissions The U.S. Department of Energy and U.S. Environmental Protection Agency (EPA) has determined that one kWh generates about 950 gram of carbon dioxide emissions for coal generated power. See: U.S. Department of Energy and U.S. Environmental Protection Agency. (2000, July). Carbon Dioxide Emissions from the Generation of Electrical Power in the United States. Available at: Energy Information Administration: V 10 http:I/www. eia.doe.gov/cneaf/e!ectr!city/page/co2_reportlco2em/ss.pclf (Accessed: I November, 2010).

If a server that draws 348W is actively used for a week, the carbon dioxide emissions will amount to 47.88 kg of CO2 emissions. The calculation is as follows: Hours of active operation: 7 days * 24 hours = 168 hours Watts * hours used kWb=( 1000 -) = (348*168)/1,000 58.464 kWh per week 58.464 kWh * 0.95 kg of CO2 emissions = 55.54 kg of CO2 emissions per week.

Calculating Cooling Requirements Cooling and ventilation accounts for about half of the energy consumption in a data centre.

Equipment generates heat whenever power (Watts) is consumed. The capacity of cooling systems is measured in British Thermal Unit (BTU). In general, I BTU is generated from 0.293 Watt of energy. The power consumption of the data centre components determines the number of BTUs per hour that will be required.

Equation B can be used to calculate the required cooling is as follows: BTUs Wattss 3.42 hour (Equation B) V 11 Neil Rasmussen, founder and the Chief Technical Officer for APC suggests that the power consumption is calculated in Watts in order to determine the cooling requirements for a data centre. Rasmussen states that "Fortunately there is a worldwide trend among standard-selling organizations to move all power and cooling capacity measurements to a common standard, the Watt'. See: Rasmussen, N. (2007). APC Media. Available at: Calculating total cooling requfrements for Data Centers: http://www.apcmedia.com/salestools/NRAN-5TE6I-IEJ?2_EN.pdf (Accessed: 1 November, 2010).

Referring to FIG. 7, APC provides a Data Centre heat output calculation worksheet to aid with the calculation The following scenario is used to demonstrate the calculation: * A 250kW rated data centre with a floor space of 465m2 * Maximum staff of 20 * Data centre utilisation: 30% The table in FIG. 8 shows how the total thermal output is calculated.

Rasmussen advises that once the cooling requirements have been calculated, an air conditioning system can be sized. Energy Star provides a web based air conditioning search tool that can help determine a cooling unit based on the calculated BTUs. See: Energy Star. (2010, March). Find ENERGY STAR Qualified Room Air Conditioners.

Available at Energy Star: s (Accessed: 1 November, 2010).

V Data Centre Hardware In the corporate world, servers tend to have a relatively short useful life cycle.

Many large companies tend to refresh their servers and desktop machines around every three to four years. Properly maintained cooling equipment can have a functional lifespan of about ten to fifteen years. See: Schulz, 6. (2009). The Green and Virtual Data Center. Boca Raton: Auerbach Publications.

One has to bear in mind that carbon dioxide emissions are not only limited to the equipment that is installed and running in the data centre. The environmental impact starts long before a server is even installed and plugged in for the first time. Professors Rudiger Kuehr and Eric Williams at the United Nations University in Tokyo have carried out an exhaustive study on the impact that computer manufacturing has on the environment. The study found that the manufacturing phase of a desktop computer is exceptionally material-intensive and accounts for 80% of the carbon footprint in the product lifecycle. See: Kuehr, R., & Williams, E. (2003). Computers and the Environment Understanding and managing their impacts. Dordrecht: Kluwer Academic Publishers.

This establishes a strong argument to utilise the full potential of the technologies that are available in order to reduce the ever increasing demand for more computer hardware.

Virtualisation EnhancedfiroductMty Companies rarely have the time and budget to deploy environmentally friendly technologies and procedures purely for environmental reasons. The reality is that there has to be a business case or incentive before changes will be considered. r 13

Virtualisation technology has been part of the IT industry since the 1960$. It was initially used to partition large mainframe hardware to enable better hardware utilisation.

See: VMware. (2O09. Virtuafisation Basics. Available at VMware: http://www.vmware.com/virtualization/history.html (Accessed: 1 November, 2010).

Today, virtualisation software provides a more mature and robust product with a broader hardware support. With the increased focus to do more with the already available resources virtualisation has seen a renewed interest.

Virtualisation has been shown to improve overall effectiveness of the IT infrastructure while improving performance, availability, responsiveness and security.

These features help to maintain business growth in an economic and environmentally friendly manner.

Some IT professionals argue that a virtual IT infrastructure increases the management and maintenance overhead which, in turn, leads to reduced ability to deliver highly scalable systems. This hurdle can be overcome by effective automation and defined management processes and procedures. In general, one administration resource is required for every 50 to 100 servers in an average size enterprise. However, it is possible to enhance this even further. A good example is Google's automated data centres where there is only one administration resource for every 20,000 machines.

See: Swabey, P. (2009, April 9). Information Age. Virtual entropy, pp. 9-11.

Virtualisation enhances productivity by enabling transparency in many areas.

Virtualisation software such as VMware's "VMotion" allows administrators to move running machines from one physical server to another without disrupting the applications or end users. See: VMware. (2008). How VMware Virtualisation Right-sizes IT P 14 infrastructure to Reduce power consumption. Available at: VMware: http://www. vm ware. corn/file sfpdf/green_wp.pdf (Accessed: 1 November, 2010).

This technology facilitates technology upgrades, replacements and circumvents the "single point of failure" challenge that sometimes causes concern in a virtualised infrastructure.

VMware's Distributed Resource Scheduler (DRS) uses a feature called "Distributed Power Management". This feature has the ability to monitor virtual machines that are running across a resource pool of physical servers. If the software detects that not all servers are fully utilised, it rebalance the load and then it switches off the surplus servers. When the demand for servers increase, the software powers the servers back on.

Efficient utiilsation of resources Virtualisation enables integration and interoperability in the data centre. With virtualisation technologies a single physical server is able to host multiple operating systems, such as Windows and Linux. A single physical server can host in the order of to 40 virtual servers. This can result in more effective server utilisation of 70 to 80%, an increase from an average utilisation of 8 to 15%. See: VMware. (2008). How VMware Virtuailsation Right-sizes IT infrastructure to Reduce power consumption. Available at: VMware: http://www.vmware.com/files/pdf/green_wp.pcjf (Accessed: 1 November, 2010).

A reduction in hardware influences the amount of cooling that is required in the data centre. Hardware maintenance costs are reduced and the equipment will take up less floor space. A decrease in hardware will result in lowered energy consumption and carbon emissions. r__15

Determining when to virtuallse servers Reference is made to the whitepaper "Five Steps to Determine When to Virtualize Your Servers". See: Davis, D. (2008). Five Steps to Determine When to Virtualize Your.

Available at: VMware: http://t zr/net comlwhitepapers/VMware_Five_Steps to Determ me IA/hen to Virtua!ize_ Your Servers.pdf (Accessed: 1 November, 2010).

David Davis gives a short overview of factors that should be taken into account to determine when servers should be virtualised. Below is a discussion of the five guidelines when considering virtualisation.

* Verify the motivation for migrating to a virtual infrastructure.

o Most projects have to be justified to the company's management.

An IT professional should evaluate why virtualisation is necessary.

Reasons for virtualisation of an infrastructure can include: * To save tine with reduced administration.

* To save money -reduced administration time, lower infrastructure maintenance costs and reduced energy consumption.

* Simplified management -resource optimization and high availability.

* Disaster recovery.

* Evaluate a virtualisation solution.

o Look for a solution that has been tried and tested and that is flexible enough to fit the needs of your company.

o Tests with evaluation versions of software are strongly advised.

* Determine if applications are going to work well with virtualisation. P 16

o A common concern is compatibility.

o A small number of scenarios can or should not be virtualised. IT professionals are usually advised not to virtualise servers that are connected to special hardware such as backup drives or card readers.

See: Ou, G. (2006, May 22). Introduction to sewer

virtualization. Available at: TechRepublic: http:I/a rticles.techrepubilc.com.com/5100-1 087 11- 607494 lMtml (Accessed: 1 November, 2010).

a Compatibility guides, such as the "Guest/I-lost OS Compatibility Guide published by VMware, should be consulted. The guide lists the guest and host operating systems that are supported by VMware.

* See: VMware. (2010, March 16). Guest/Host OS Compatibility Guide. Available at: VMware: http://www. vm ware. com/resources/compatibi!ity/pdf/VMware _OS_Compatibility_Guide.pdf (Accessed: 1 November, 2010).

* Analyse the cost of virtualising the server infrastructure.

a The cost of virtualisation will depend on the infrastructure that is to be virtualised.

* Analyse the time and skill needed to virtualise the server infrastructure a Depending on the infrastructure that is to be virtualised Software ancLootentiaisavings In a virtualised environment there are potential savings to be made in the licensing costs. A good example of this is Windows Server 2003 R2 Datacenter Edition. Microsoft r 17 states that "...you can run one instance of the software in a physical operating system environment and an unlimited number of Thstances in virtual operating system environments". See: Microsoft. (2007, January 31). Microsoft Builds on Windows Seiver Datacenter Edition's Rellability and Scalability with Unilmited Virtuailzation Rights.

Available at: Microsoft: http://www.microsoft. com/windowsseiver2003/evaluation/newsjbuffetins/datacenterhigha vaiL mspx (Accessed: 1 November, 2010).

Another factor that may make a small difference is the choice of software. Some operating systems use the hardware in a more power efficient way than others. One only has to search the online IT forums to realise that laptop users are most likely to notice the differences in power requirements of the various operating systems. Software energy profiling has proved that different operating systems have dissimilar power requirements. See: Sinha, A., & Chandrakasan, A. p. (2003, September 9). Joule Track -A Web Based Tool for Software Energy Profiling. Available at IEEE: http://ieeexplore.ieee.org/stamplstamp.jsp?tp=&arnumber= 1225819 (Abstract Accessed: 1 November, 2010).

Michael Larabel published an article called "Linux vs. Windows Power Usage".

The tests analysed the power consumption of both Windows and Linux operating systems and found that Linux have slightly higher power consumption than Windows.

See: Larabel, M. (2007, October 16). Linux vs. Windows Power Usage. Available at: Phoronix: http://www.phoronix.com/scan.php?page=article&item=880&num=2 (Accessed: 1 November, 2010).

As explained above, at present, an engineer visits a data centre in person to obtain opertaing information in relation to the data centre. This may involve installing additional r 18 monitoring hardware and/or software. The potential security risks, particuilarly in smaller organisations, the inconvenience to those running the data centre, and the risk of monitoring inadvertently causing a fault in the data centre. The present invention is set against this backdrop. It is desirable to understand whether and how a data centre's physical hardware could be virtualised and what the potential savings are in terms of energy, cost, floor space and the like with minimal intrusion.

According to a first aspect of the invention, there is provided a virtualisation emulation method of modelling a virtualised infrastructure of a data centre having a physical infrastructure comprising at least two physical servers, the method comprising obtaining physical configuration data of the data centre's physical infrastructure, obtaining software specification data of software running on the at least two physical servers, using the physical configuration data to estimate the power consumption of the physical infrastructure in the data centre, using the software specification data to determine whether at least one of the physical servers can be virtualised, upon a positive virtualisation determination, emulating a model of the virtualised infrastructure and estimating the power consumption of the virtualised infrastructure, and comparing the power consumption of the physical infrastructure with the power consumption of the emulated virtualised infrastructure, whereby a test bed of the emulated virtualised infrastructure can be set-up to determine the effects of virtualisation on the physical infrastructure.

The method may comprise non-intrusively obtaining the physical configuration data

and the software specification data.

The physical configuration data may comprise vendor-supplied hardware information. r 19

The data centre may comprise at least one cooling apparatus and wherein the power consumption estimation includes estimating the power consumption of the at least one cooling apparatus.

The method may comprise using the estimated power consumption to determine the cooling requirements of the data centre.

The method may comprise using the estimated power consumption of the physical infrastructure to estimate the carbon dioxide emissions associated with the operation of the physical infrastructure.

The method may comprise using the estimated power consumption of the virtualised infrastructure to estimate the carbon dioxide emissions associated with the operation of the virtualised infrastructure.

The method may comprise setting up the test bed.

The method may comprise virtualising at least some of the physical infrastructure.

Various embodiments of the invention will now be described, with reference to the accompanying drawings, in which: Figure 1 shows a schematic representation of full virtualisation; Figure 2 shows a schematic representation of para-virtualisation; Figure 3 shows a schematic representation of operating system partitioning; Figure 4 shows typical data centre power consumption; Figure 5 shows a power calculator; r 20 Figure 6 shows a calculated power consumption; Figure 7 shows a data centre heat estimation worksheet; Figure 8 shows how a data centre thermal output can be calculated; Figure 9 shows a test bed setup for a home computer; Figure 10 is a graph of energy consumption and CPU usage for hosting hardware; Figure 11 shows how floor space for a home computer can be calculated; Figure 12 is a graph of energy consumption and CPU usage for a host computer running a Windows XP virtual machine; Figure 13 is a graph of energy consumption and CPU usage for a host computer running a Windows 7 Ultimate virtual machine; Figure 14 is a graph of energy consumption and CPU usage for a host computer running a Windows Server 2008 virtual machine; Figure 15 is a graph of energy consumption and CPU usage for a host computer running a Ubuntu Server virtual machine; Figure 16 is a graph of energy consumption and CPU usage for a host computer running a tibuntu Desktop virtual machine; Figure 17 is a graph of average energy consumption and CPU usage across different virtual machines; Figure 18 shows schematically a local and virtual operating system; Figure 19 shows CPU usage when running virtual machines; Figure 20 shows CMS server power consumption; and Figure 21 shows ILS server power consumption.

Virtual Machines -PC The design Technology V 21 A PC was analysed to determine how much energy could be saved by implementing different types of virtual machine. Some of the key considerations were: CPU Utilisation o The CPU utilisation was monitored with a sidebar monitor.

* Power Consumption o Power consumption was displayed in Watts on a plug-in power.

Only the power consumption of the computer was measured and care was taken to ensure that no other peripherals were plugged into the computer during the tests. A video camera recorded the power utilisation readings.

* Computer activity o CamStudio screen capturing software was used to capture all screen activity * Virtualisation software o VMware Server Version 2.0.2 was used to create virtual machines * Virtual machines o The following virtual machines were created for the analysis: * Ubuntu Desktop * Ubuntu Server (Command Line Interface) * Windows 7 Ultimate * Windows Server 2008 Enterprise * Windows XP To aid the analysis, recordings were made of the hardware energy consumption (measured at the power outlet) and the utilisation of the CPU.

FIG. 9 shows the set-up of the test bed. The test bed included: r 22 o a plug-in power meter 1 o a video camera 2 to capture power meter readings o a home computerS, used as server host o a performance monitor 4, monitoring: RAM and CPU usage o a clock 5, perfectly synchronized with time on plug-in power meter o CamStudio screen capturing software to capture activities The Virtualisation software used was VMware Server Version 2.

Video dllps Below is a list of the systems that were analysed on the home computer and the URL to the accompanying video clip: o Hosting hardware (Windows 7 Professional Operating System) o http://www.youtube.com/watch?v=lP2sgszXteY The effect of virtual machines on the abovementioned hosting machine was also tested and the video clips can be viewed by following the links provided: o Windows XP Virtual Machine o http:Ilwww.youtube.com/watch?v=a2dbkvDZmAQ&NR=1 o Windows 7 Ultimate Virtual Machine o http://www.youtube.com/watch?v=FNwkXJxHH4o o Windows Server 2008 Enterprise o http:I/www.youtube.corn/watch?v=51 bROV9EXx8 o Ubuntu Server (Command Line Interface) Virtual Machine o http://www.youtube.com/watclflv=hxUdoNzrCMo o Ubuntu Desktop Virtual Machine o http:IIwww.youtube.com/watch'pv=snb7mvu5Q V 23 Table 1 below provides an overview of the home system's hardware and software specifications that was used in the testing. A detailed graph and screen captures will be discussed later.

Hosting Virtual Virtual Virtual Virtual --Virtual Hardware Machine I Machine 2 Machine 3 Machine 4 Machine 5 Operating Windows 7 WindowsXP Windows 7 Windows Ubuntu -Ubuntu System Professional Professional Ultimate Server 2008 Server Desktop Processor 2.40 0Hz 1 x 2.4 GHz 2 x 4.693 2 x 4.693 1 x I x Speed GHz 0Hz 2.346GHz 2.346GHz Installed 3.00GB 512 MB 1.5GB 1.5GB 512 MB 512 MB Memory (RAM) Harddrive 297GB 8GB 8GB 16GB 8GB 8GB Size System Measurements The data that was gathered during the home testing formed the basis for the measurements that can be used to compare a physical infrastructure to a virtualised infrastructure. Using the PCFE (Power, Cooling, Floor space and Environment) approach as suggested by Schultz, the CPU utilisation and energy consumption of the home computer that was used in the testing was analysed. See: Schulz, G. (2009). The Green and Virtual Data Center, Boca Raton: Auerbach Publications. Power

r FIG. 10 shows the level of CPU utilisation and energy consumption of the host machine. Since no virtual machines formed part of this scenario, the baseline values were derived from this scenario to be used in later tests with virtual machines.

Using the hardware data that was gathered in the home tests, the average power requirement is calculated as shown in Table 2 below: Desktop LCD Monitor Total Activelyused --94W 18W -112W Activelyused 10 hours per 1.12 kWhper 0.94 kWh per day 0.18 kWh per day day day Standby 3.4W1W 4.4W Standby 14 hours per day 0.048 kWh per 0.014 kWh per 0.062 kWh per day day day The home system will typically use a total of 1.18 kWh in a 24 hour period, or 431.4 kWh over a single year.

Coo/Tho The power measurement is expressed in Watts as suggested by Neil Rasmussen in the whitepaper titled talculating Total Cooling Requirements for Data Centres". In Table 3 below, this measurement is used to calculate the cooling requirement as suggested by the same author.

Item Ebta lbquired Fbat output &dAotal Them on ckdI*lt baled cpadty (Uilisation) _________ _________ ______________ _______ ITEiuipment -1164 -116 116 Watt 18°A LwithBattery 04 -6 6'Watt 1°A Rwerflatribution 1164 3 3 watt VA Uiting(Roorspace) 25 538 538 Watt -8Th iop1e _____ ---Watt 0°A Total ---663.97Watt Total Thermal output In kW: 0.66 KW P 25 *The number of people is included in the calculation only if more than two people use the room where the IT equipment is kept.

The value calculated in Table 4.1-4 can now be used to calculate the BTUs/hr: Watts x 3.41 = BTUs/hr 660 W x 3.41 = 2250.6 BTUs /hr Due to the environment where this desktop is kept (large windows and little direct sunlight), no additional cooling is required for the home system. However, if a small portable Amcor ACP air conditioner was to be installed for cooling, then an additional 250 W would be consumed. This would add an additional 6 kWh per day to the electricity bill.

Floor £oace Referring to FIG. 11, the hosting computer has a width of 18 cm, length of 35 cm and a height of 35 cm. The area (floor space) occupied is therefore calculated as follows: Area = length x width =18x35 = 630 cm2 or 0.63 m2 Environment The U.S. Department of Energy and U.S. Environmental Protection Agency estimate that 950g of carbon dioxide is produced for every kWh. This means that the home computer and the monitor produce 409.83 kg of CO2 in a single year. (431.4 kWh x 0.95kg). See: U.S. Department of Energy and U.S. Environmental Protection Agency.

(2000, July). Carbon Dioxide Emissions from the Generation of Electrical Power in the P 26 United States. Available at Energy Information Administration: http://www.eia.doe.gov/cneaf/electricity/pagelco2_reportlco2emiss.pdf (Accessed: I November, 2010).

Virtual Machines -Analysis With reference to Figures 12 to 16 and as mentioned above, a number of virtual machines were tested. The hardware and software specifications were detailed earlier.

During the testing, the power consumption of the hosting machine and the CPU utilisation was recorded. Initially, the power consumption and CPU utilisation of the host machine was recorded. This was then used as a baseline for the later comparisons.

The virtual machines were started up individually and again the CPU utilisation and power consumption was recorded for each scenario.

Table 4 below shows the increase in utilisation and energy consumption for each virtual machine scenarios. The readings that were recorded in earlier tests on the host machine were used as a baseline in these calculations.

Windows Windows 7 Windows Server Ubuntu Ubuntu -XP VM Ultimate VM 2008 Enterprise Sewer Desktop

VM VM VM

%lncrease 12% -25% 20.50% 3.50% 4% CPU Utilisation % Increase 3.70% 3% 0.90% 2.60% 3.20% -Energy r_ 27 [ffnsurnPtion * Constant factor: Screen capturing software (CamStudio) was running throughout all tests CamStudio screen capturing software was used to capture all screen activity.

Since this was a factor that was present in all tests, it is not discussed as a separate influence on the hosting hardware.

The average values show that there is not a strong correlation between the increase in CPU utilisation and the increased power requirements when the virtual machine is actively used.

By comparing the virtual machine scenarios, it becomes clear that the Windows Server 2008 virtual machine noticeably require more processing resources than the Ubuntu server, for example, but the additional power requirements for Windows Server was insignificant.

In FIG. 17, the host machine baseline is used to compare graphically the results of the individual scenarios. The Linux operating systems clearly showed lower CPU utilisation than the Windows systems. However, the power consumption of Linux systems was higher than the Windows systems. The higher consumption by the Linux systems correlates with Larabel's findings that were discussed earlier.

Physical Machine Requirements vs. Virtual Machine Requirements One can conclude from the above tests that any of the virtual machines can run effortlessly on the hosting machine. This is more favourable than setting up an additional system with all the associated hardware. It was established above that the V 28 home computer and the monitor produces 409.83 kg of CO2 in a single year. By implementing a virtual environment rather than selling up additional hardware, the production of large amounts of additional CO2 is prevented.

The following scenario makes a small-scale comparison. A physical infrastructure is compared to a virtual environment. Two scenarios from the data that was recorded in the earlier tests is compared and analysed in more detail. The systems with the locally hosted Windows 7 operating system and a Windows 7 virtual machine with similar specification are analysed. Graphical representations of the scenarios are shown in FIG. 18.

In both scenarios a host machine was used with Windows 7 as the installed operating system.

The average CPU utilisation values and energy consumption and the average increase in each scenario is summarised in Table 5 below.

Averages System Cpu % Energy consumption % Increase (W) Increase *Hosting hardware 40 % -NA 92.8 W NA Wkidows 7 Ultimate Virtual 50 % 25% 955 W 3% Machine P 29 Referring to FIG. 19, in the tests it was possible to run up to four virtual machines at the same time on the host machine. The processor utilisation increased to 9%.

Data Centre Model The aforementioned analysis was performed on a home computer and albeit important, the potential benefits of virtualisation are relatively small and impractical.

The physical infrastructures of data centres at the University of GreenwIch were modelled based on the data that was provided by the University's IT staff. It was not possible to access the University's data centre because of security and safety concerns which may be similar to those of large corporations. Obtaining information about the data centre in this manner is referred to in herein as non-intrusive'.

The following questionnaire was devised to obtain server information, i.e. physical configuration data, of the data centre's physical infrastructure and software specification data of software running on the server for modelling the data centre's power requirements: Hardware 1.1 Servers Server I Server 2 Server Server Server Server 3 4 5 6 12 Make V'ModeF 1.4 RAM 1.5 Chipset(lntel/AMD) 1.6 Processor Speed -_________ _________ _______ _______ _______ _______ 13 Server's function 1.8 Number of power supplies 1.9 Average CPU -________ --______ ______ ______ ______ utilization(%) 1.10 *Server management software in use? *Eg. HP's ILO software that monitors & manages processor utilization, heat, power consumption etc 2 Software 2.1 OS (Name and version) -_________ ______ ______ ______ ______ 2.2 Service packs applied -__________ __________ _______ _______ _______ _______ r.31 Applications installed 2.4 Anyoftheaboveapps that can't run in Virtual Environment? 3 Storage 3.1 Number of hard drives 3.2 HDD Make 3.3 HDD Size 3.4 RAID 4 UPS 4.1 UPSSystems LJPS1 UPS2...

4.2 Make 4.3 Model 4.4 How many batteries? (if -_________ ______ ______ ______ _______ any) 4.5 Up-time in event of power failure Generators 5.1 Generators -Generator Generator 1 2 52 Make 5.3 Model 5.4 KW 6 Cooling 6.1 Cooling Systems -Cooling I Cooling 2 6.2 Make 6.3 Model 6.4 Whattemperatureis maintained? The aim was to establish if any of the servers could be virtualised and if some of the physical servers could be made redundant. Savings would be made, even if just one server were taken out of service. If a server that requires around 350W to operate is migrated to a virtual environment, then a 3,O66kWh per annum will be saved (8.4kWh r per day). This equates to a total of 2,913 kg of CO2 for the same year (excluding additional factors such as the cooling requirements).

This potential saving can be brought into perspective by comparing it to the CO2 emissions of a luxury V8 Jaguar model. Act on CO2 estimates that the V8 saloon have a carbon footprint of approximately 0.29 kg of CO2 per kilometre. Excluding all other factors such oil usage, replacement of tyres etc, this means that one can make four round trips from Plymouth to Thorso in Northern Scotland before the annual carbon footprint of the server is matched. See: Act on CO2. (2010, February). New car C02 emissions. Available at: Act on C02: http://actonco2.directgov. uk/actonco2/home/what-you-can-do/Compare-car-C02-emissjons/new-car-co2-emj ssjons-mode!-searchhtm/ (Accessed: 1 November, 2010).

This strongly supports the call for better hardware utilisation that was made by Energy Star. See: Energy Star Program. (2007). Report to Congress on Server and Data Center Energy Efficiency, Public Law 109 -432. Energy Star.

The data centre's power requirements For this project the physical configuration data of 21 servers was collected with the help of IT staff at the University of Greenwich. The Information and Library Services (ILS) provided details of five servers and the School of Computing and Mathematical Sciences (CMA) gave the details of sixteen servers. A model of the physical infrastructure was created based on the data.

The details of the ILS and CMS servers were collected by using the vendors' specification sheets, i.e. by using non-intrusively vendor-supplied hardware information,

V

rather than by having to install additional hardware and/or software. This information was then recorded in the power consumption calculator that was discussed above.

FIG. 20 shows a table that provides a detailed overview of the results as calculated by the power calculator. The total power requirement of the 16 servers in the CMS model added up to 5,727 Watts.

FIG. 21 shows a table that provides details of the five ILS servers that were also calculated by using the power calculator. The total energy requirements of the ILS model added up to 1,929 Watt.

The calculated values were then used to calculate the annual kWh and CO2 emissions and the BTUs that are produced by the power supplies and other internal components of the 21 servers. Table 6 below shows a summary of the power requirements and consequent carbon emissions and cooling requirements of the modelled CMS and ILS physical infrastructures.

CMS Model -ILS Model Physical Infrastructure Physical infrastructure Number of physical servers 16 Servers 5 Servers Power 5,727 Watt -1,929 Watt Annual kWh -50,168.52 kWh 16,898.04 kWh Annual CO2 47,660 kg -16,053 kg BTUIhour 19,559 BTUs/hr 6,587 BTUsIhr Table 6: Power consumption comparison

V

This calculation is based on the power consumption of the servers only and does not include peripherals such as monitors.

Not all scenarios are suitable for virtualisation, as discussed in "Determining when to virtualise servers'. It is generally advised not to virtualise servers that are connected to hardware such as backup drives. For this reason, four CMS servers and one ILS server that are used for backup in the CMS model were not considered for virtualisation.

The Data Centre's Cooilng Requfrements As discussed by Rasmussen (see above), cooling is calculated in BTUs/hour. The power (Wafts) that is drawn by a system is used to establish the thermal energy that will be generated by that system. (Wafts * 3.42 BIUs / hour) In the above models of the CMS and ILS data centres, it was established that the servers draw a total of 7,656 Waft.

This generates a total of 26,146 BTUs / hour.

By using Energy Star's web based Air Conditioning Unit search tool, it was found that a large air conditioning unit will be required to meet the cooling requirements of these two models. See: Energy Star. (2010, March). Find ENERGY STAR Quailfied Room Afr Conditioners. Available at Energy Star httpflwww.energystar.gov/index. cfm?fuseaction=roomac.searc h_Room_A fr_Conditioner s (Accessed: 1 November, 2010).

The power consumption of a suitable air conditioner would draw around 4.3kW during operation. See: Spot Coolers. (2010, March). Spot Coolers. Available at: Spot Coolers: http:I/spotcoolers.rtrk.com/?sckl= 1513915&rl_alt=http%253A%252F%252Fwww.spot..

coolers.com&rl_path=/se,ver.php (Accessed: 1 November, 2010).

The annual carbon emissions of the air conditioning unit can then be calculated as follows: (4,300 Watt * 8,760 hours) / 1,000 = 37,668 kWh.

CO2 Emissions: 37,668 kWh * 0.95 kg = 35,785 kg of CO2 The power consumption of the air conditioning unit is more than half the total power consumption of the two models of the physical infrastructure. This correlates with the statement by Rasmussen that cooling requirements account for a large portion of the data centre's power consumption.

Software Specifications and Compatibility

Virtualisation $upgtort for Guest OperatmgSystems It was explained earlier that compatibility of operating systems and applications should be determined as part of the assessment phase. As such, software specification data was collected to enable emulation of the virtualised server. The "Guest/Host OS Compatibility Guide" was consulted to determine the compatibility of the operating systems and the applications that are used in the CMS and 1LS models. it was established that VMware provides virtualisation support for all the Operating Systems used by the CMS and ILS departments. See: VMware. (2010, March 16). Guest/Host OS Compatibility Guide. Available at: VMware: http://www. vmware.com/resources/compatibfflty/pdfNMwarefiS_Compatibility_ciuide.p df or http://partnerweb. vmware.com/comp_guide/pdfNMware_OS_Compatibility_Guide.pdf (Latter accessed: 1 November, 2010).

Virtualisation Support for Software Applications The growing popularity of virtualisation also means that more and more applications are supported by the vendors of virtualisation software. However, with the number of software packages available on the market, it is not practically possible for virtualisation software vendors to maintain a comprehensive guide of compatible software. Instead, many vendors state the compatibility in the software specifications.

Table 7 below summarises the operating systems and the applications that are installed on the servers in the CMS and ILS models. Approximately half of the applications used by CMS and ILS are officially compatible in a VMware virtualised environment. Even though no official documents were readily available to confirm that the rest of the applications are compatible, a number of sources provide support for these applications in software environments. From this one could make the assumption that many companies do indeed use these applications in a virtualised environment.

VMware Software Applicable URLS Supported? Operating System Debian Yes 5.0.1 Microsoft Yes -http://www.vmware.com/resources/compatibilityl Windows pdfNMware_OS_Compatibility Guide.pdf Server 2003 Solaris 9 Yes -Solaris ID Yes Software Applications -______________________________________________ r__ 38 Apache Yes --http://www.kiv.zcu.czF'brada/doc/ Server man/apache/vhosts/virtual-host.html Backup Yes http://www.dell.com/downloads/global/solutionsl Exec vmware_softwarepompatibility_matrix_current.pdf Banner tNot (Forms I officially Reports) Banner *Not database officially Banner *Not Imaging officially Software Dovecot *Not officially Exchange Yes -http:Ilwww.vmware.cornlfileslpdf/partners/ 2007 netapp-vmware-exchange-tr-final.pdf http:Ilwww.vmware.comlfileslpdf/solutionsl 08Q4_VM_Exchange_ServerjOO7_VI3WP.pdf GCC MPI *Not officially Horde -*Not officially IIS *Not officially MySQL http://www.vmware.com/files/ pdfNirtualization-for-MySQL-on-VMware.pdf r_____ Oracle Yes http://www.vmware.com/solutions-application business-critical-apps/oracle/ server Oracle Yes http://www.vmware.com/solutionsl database business-critical-apps/oracle/ software Veritas Yes http://eval.symantec.com/mktginfo/ Netbackup enterprise/whitejapersfent-Software whitepaper_veritas_netBackup_6.Qymware_nov2007.pdf WebCT *Not Applications officially Table 7: Physical vs. Virtualised Infrastructure Comparison -Physical Infrastructure vs. Virtualised Infrastructure By considering the factors that should be taken into account when deciding which systems to virtualise, it was found that eight of the CMS servers and one of the ILS servers could be virtualised. Four of the CMS servers and one ILS server are used for backup which is usually not suitable for virtualisation. Consequently, these five servers would ideally not be virtualised. Of the five servers used in the ILS model, two were already being utilised at 60% and 80% and would not be virtualised. In the emulated model1 the two servers with the lowest CPU utilisation would be virtualised.

It was also determined that the emulated virtualised model requires less cooling. A suitable air conditioner would use 2.9 1KW to cool the twelve servers in the emulated model. See: Spot Coolers. (2010, March. Spot Coolers. Available at Spot Coolers: http:llspotcoolers.rtrk.coml?scid-1513915&rl_alt=http%253A%252F%252Fwww. spot-coolers. com&rpath=/server.php (Accessed: 1 November, 2010).

The calculation for the power consumption would then be as follows: (2,900 Watt * 8,760 hours) / 1,000 = 25,404 kWh.

CO2 Emissions: 17,520 kWh * 0.95 kg= 24,134 kg of CO2 Table 8 below compares the requirements of the physical infrastructure to an emulated virtualised infrastructure.

Physical Virtualised -CMS Model ILS Model CMS Model ILS Model Number of physical -___________ ____________ ____________ 16 Servers 5 Servers 8 Physical 4 Physical servers Power (Servers) 5j27 Watt 1,929 Waft 3,612 Watt -1,736 Waft -19,559 BTUs 6,587 BTUs 12,336BTUs 5,928BTUs STU/hour fhr Ihr I/jr /!ir Power (Cooling) 4,300 Watt 2,900 Waft -50,168.52 16,898.04 15,207.36 Annual kWh (Servers) 31,641 kWh kWh kWh kWh Annual kWh (Cooling) 37,668 kWh 25,404 kWh -Annual C02(Servers) 47,660 kg 16,053 kg 30,059 kg 14447k Annual CO2 (Cooling) 35,785kg -24,134 kg *Total Annual kWh 104,734.56 kWh 72,252.36 kWh tlotal Annual CO2 99,498 kg 68,640 kg -Table 8: Comparison -Physical Infrastructure vs. Virtualised Infrastructure *lncludes the servers of both models and the power consumption of the air conditioning system V 41 The emulated model of the virtualised infrastructure only has nine fewer servers than the model of the physical infrastructure. However, nine servers can make a surprisingly large difference in the energy requirements and the carbon emissions of a data centre.

By using a virtualisation technology, the virtualised model would use 32,482.2 kWh less than the physical infrastructure.

By using the British Gas Electricity rates on 1 November 2010, the potential savings on the electricity bill was calculated, inclusive of Value Added Tax (VAT) at 17.5%. The day rate tier I was calculated at 22.5419 pence and the day rate tier 2 at 10.8019 pence. The calculated consumption at the specified rate is shown in Table 9 below: -Infrastructure Rate -Total Physical Infrastructure -Day rate -tier I £112.71 -Day rate-tier2 £11,259.31 Physical Infrastructure Total £11,372.02 Virtualised Infrastructure Day rate -tier 1 £1 12.71 Day rate -tier 2 £7,750.62 Virtualised Infrastructure Total -£7,863.33 Potential savings by using virtualisation £3,508.69 Table 9: Potential savings on electricity V 42 The comparison of the two models shows that the University of Greenwich could make a potential saving of £3,508.69 per annum by virtualising nine of the twenty-one servers that were used in this model. In this model, it means a saving of 31% on the electricity bill.

A more comprehensive audit of an entity's data centre could help to identify more servers that would be suitable for virtualisation. This could potentially save large amounts of money in relation to that entity's electricity bill.

Using the above model of the data centre, a test bed of the emulated virtualised infrastructure can be set up for testing an monitoring.

Claims (9)

1. VClaims 1. A virtualisation emulation method of modelling a virtualised infrastructure of a data centre having a physical infrastructure comprising at least two physical servers, the method comprising: obtaining physical configuration data of the data centre's physical infrastructure; obtaining software specification data of software running on the at least two physical servers; using the physical configuration data to estimate the power consumption of the physical infrastructure in the data centre; using the software specification data to determine whether at least one of the physical servers can be virtualised; upon a positive virtualisation determination, emulating a model of the virtualised infrastructure and estimating the power consumption of the virtualised infrastructure; and comparing the power consumption of the physical infrastructure with the power consumption of the emulated virtualised infrastructure, whereby a test bed of the emulated virtualised infrastructure can be set-up to determine the effects of virtualisation on the physical infrastructure.
2. 2. A method according to claim 1, comprising non-intrusively obtaining the physical configuration data and the software specification data.
3. 3. A method according to claim 2, wherein the physical configuration data comprises vendor-supplied hardware information.
4. 4. A method according to any preceding claim, wherein the data centre comprises at least one cooling apparatus and wherein the power consumption r estimation includes estimating the power consumption of the at least one cooling apparatus.
5. 5. A method according to any preceding claim, comprising using the estimated power consumption to determine the cooling requirements of the data centre.
6. 6. A method according to any preceding claim, comprising using the estimated power consumption of the physical infrastructure to estimate the carbon dioxide emissions associated with the operation of the physical infrastructure.
7. 7. A method according to any preceding claim, comprising using the estimated power consumption of the virtualised infrastructure to estimate the carbon dioxide emissions associated with the operation of the virtualised infrastructure.
8. 8. A method according to any preceding claim, comprising setting up the test bed.
9. 9. A method according to any preceding claim, comprising virtualising at least some of the physical infrastructure.