Performance per watt

The performance and power consumption metrics used depend on the definition; reasonable measures of performance are FLOPS, MIPS, or the score for any performance benchmark. Several measures of power usage may be employed, depending on the purposes of the metric; for example, a metric might only consider the electrical power delivered to a machine directly, while another might include all power necessary to run a computer, such as cooling and monitoring systems. The power measurement is often the average power used while running the benchmark, but other measures of power usage may be employed (e.g. peak power, idle power).

For example, the early UNIVAC I computer performed approximately 0.015 operations per watt-second (performing 1,905 operations per second (OPS), while consuming 125 kW). The Fujitsu FR-V VLIW/vector processor system on a chip in the 4 FR550 core variant released 2005 performs 51 Giga-OPS with 3 watts of power consumption resulting in 17 billion operations per watt-second.[2][3] This is an improvement by over a trillion times in 54 years.

Most of the power a computer uses is converted into heat, so a system that takes fewer watts to do a job will require less cooling to maintain a given operating temperature. Reduced cooling demands makes it easier to quiet a computer. Lower energy consumption can also make it less costly to run, and reduce the environmental impact of powering the computer (see green computing). If installed where there is limited climate control, a lower power computer will operate at a lower temperature, which may make it more reliable. In a climate controlled environment, reductions in direct power use may also create savings in climate control energy.

Computing energy consumption is sometimes also measured by reporting the energy required to run a particular benchmark, for instance EEMBC EnergyBench. Energy consumption figures for a standard workload may make it easier to judge the effect of an improvement in energy efficiency.

Performance (in operations/second) per watt can also be written as operations/watt-second, or operations/joule, since 1 watt = 1 joule/second.

Exponential growth of supercomputer performance per watt based on data from the Green500 list. The red crosses denote the most power efficient computer, while the blue ones denote the computer ranked#500.

FLOPS per watt is a common measure. Like the FLOPS (Floating Point Operations Per Second) metric it is based on, the metric is usually applied to scientific computing and simulations involving many floating point calculations.

The Green500 list ranks computers from the TOP500 list of supercomputers in terms of energy efficiency, typically measured as LINPACK FLOPS per watt.[17][18]

As of November 2012, an Appro International, Inc. Xtreme-X supercomputer (Beacon) topped the Green500 list with 2499 LINPACK MFLOPS/W.[19] Beacon is deployed by NICS of the University of Tennessee and is a GreenBlade GB824M, Xeon E5-2670 based, eight cores (8C), 2.6 GHz, Infiniband FDR, Intel Xeon Phi 5110P computer.[20]

As of June 2013, the Eurotech supercomputer Eurora at Cineca topped the Green500 list with 3208 LINPACK MFLOPS/W.[21] The Cineca Eurora supercomputer is equipped with two Intel Xeon E5-2687W CPUs and two PCI-e connected NVIDIA Tesla K20 accelerators per node. Water cooling and electronics design allows for very high densities to be reached with a peak performance of 350 TFLOPS per rack.[22]

As of November 2014, the L-CSC supercomputer of the Helmholtz Association at the GSI in Darmstadt Germany topped the Green500 list with 5271 MFLOPS/W and was the first cluster to surpass an efficiency of 5 GFLOPS/W. It runs on Intel Xeon E5-2690 Processors with the Intel Ivy Bridge Architecture and AMD FirePro S9150 GPU Accelerators. It uses in rack watercooling and Cooling Towers to reduce the energy required for cooling.[23]

As of August 2015, the Shoubu supercomputer of RIKEN outside Tokyo Japan topped the Green500 list with 7032 MFLOPS/W. The then-top three supercomputers of the list used PEZY-SC accelerators (GPU-like that use OpenCL)[24] by PEZY Computing with 1024 cores each and 6–7 GFLOPS/W efficiency.[25][26]

As of June 2019, DGX SaturnV Volta, using "NVIDIA DGX-1 Volta36, Xeon E5-2698v4 20C 2.2GHz, Infiniband EDR, NVIDIA Tesla V100", tops Green500 list with 15,113 MFLOPS/W, while ranked only 469th on Top500.[27] It's only a little bit more efficient than the much bigger Summit ranked 2nd while 1st on Top500 with 14,719 MFLOPS/W, using IBM POWER9 CPUs while also with Nvidia Tesla V100 GPUs.

Top 10 positions of GREEN500 in November 2019[28]

Rank	Performance per Watt (GFLOPS/Watt)	Name	Model Processors, Interconnect	Vendor	Site Country, year	Rmax (PFLOPS)
1	16.876	A64FX prototype	Fujitsu A64FX Fujitsu A64FX 48C 2GHz, Tofu interconnect D	Fujitsu	Numazu   Japan, 2018	1.999
2	16.256	NA-1	ZettaScaler-2.2 Xeon D-1571 16C 1.3GHz, Infiniband EDR, PEZY-SC2 700Mhz	PEZY Computing K.K.	JAMSTEC Yokohama Institute for Earth Sciences, Yokohama   Japan, 2019	1.303
3	15.771	AiMOS	IBM Power System AC922 IBM POWER9 20C 3.45GHz, Dual-rail Mellanox EDR Infiniband, NVIDIA Volta GV100	IBM	Rensselaer Polytechnic Institute, Troy,  United States, 2018	8.045
4	15.574	Satori	IBM Power System AC922 IBM POWER9 20C 2.4GHz, Infiniband EDR, NVIDIA Tesla V100 SXM2	IBM	MIT/MGHPCC, Holyoke, Massachusetts,  United States,2018	1.464
5	14.719	Summit	IBM Power System AC922 IBM POWER9 22C 3.07GHz, NVIDIA Volta GV100, Dual-rail Mellanox EDR Infiniband	IBM	Oak Ridge National Laboratory, Oak Ridge, Tennessee   United States, 2018	148.600
6	14.423	AI Bridging Cloud Infrastructure (ABCI)	Primergy CX2570 M4 Xeon Gold, Tesla V100 SXM2,Infiniband EDR	Fujitsu	Joint Center for Advanced High Performance Computing, Kashiwa   Japan, 2018	19.880
7	14.131	MareNostrum P9 CTE	IBM Power System AC922 IBM POWER9 22C 3.1GHz, Dual-rail Mellanox EDR Infiniband, NVIDIA Tesla V100	IBM	Barcelona Supercomputing Center, Barcelona,   Spain,2019	1.145
8	13.704	TSUBAME3.0	SGI ICE XA IP139-SXM2, Xeon E5-2680v4 14C 2.4GHz, Intel Omni-Path, NVIDIA Tesla P100 SXM2	Hewlett-Packard	Tokyo Institute of Technology, Tokyo,   Japan,2017	8.045
9	13.065	PANGEA III	IBM Power System AC922 A III - IBM Power System AC922, IBM POWER9 18C 3.45GHz, Dual-rail Mellanox EDR Infiniband, NVIDIA Volta GV100	IBM	Total S.A., Pau,   France,2019	17.860
10	12.723	Sierra	IBM Power System AC922 - IBM Power System AC922, IBM POWER9 22C 3.1GHz, NVIDIA Volta GV100, Dual-rail Mellanox EDR Infiniband	IBM	Lawrence Livermore National Laboratory, Livermore,   United States, 2018	94.640

Graphics processing units (GPU) have continued to increase in energy usage, while CPUs designers have recently focused on improving performance per watt. High performance GPUs may draw large amount of power and hence, intelligent techniques are required to manage GPU power consumption.[29] Measures like 3DMark2006 score per watt can help identify more efficient GPUs.[30] However that may not adequately incorporate efficiency in typical use, where much time is spent doing less demanding tasks.[31]

With modern GPUs, energy usage is an important constraint on the maximum computational capabilities that can be achieved. GPU designs are usually highly scalable, allowing the manufacturer to put multiple chips on the same video card, or to use multiple video cards that work in parallel. Peak performance of any system is essentially limited by the amount of power it can draw and the amount of heat it can dissipate. Consequently, performance per watt of a GPU design translates directly into peak performance of a system that uses that design.

Since GPUs may also be used for some general purpose computation, sometimes their performance is measured in terms also applied to CPUs, such as FLOPS per watt.

While performance per watt is useful, absolute power requirements are also important. Claims of improved performance per watt may be used to mask increasing power demands. For instance, though newer generation GPU architectures may provide better performance per watt, continued performance increases can negate the gains in efficiency, and the GPUs continue to consume large amounts of power.[32]

Benchmarks that measure power under heavy load may not adequately reflect typical efficiency. For instance, 3DMark stresses the 3D performance of a GPU, but many computers spend most of their time doing less intense display tasks (idle, 2D tasks, displaying video). So the 2D or idle efficiency of the graphics system may be at least as significant for overall energy efficiency. Likewise, systems that spend much of their time in standby or soft off are not adequately characterized by just efficiency under load. To help address this some benchmarks, like SPECpower, include measurements at a series of load levels.[33]

The efficiency of some electrical components, such as voltage regulators, decreases with increasing temperature, so the power used may increase with temperature. Power supplies, motherboards, and some video cards are some of the subsystems affected by this. So their power draw may depend on temperature, and the temperature or temperature dependence should be noted when measuring.[34][35]

Performance per watt also typically does not include full life-cycle costs. Since computer manufacturing is energy intensive, and computers often have a relatively short lifespan, energy and materials involved in production, distribution, disposal and recycling often make up significant portions of their cost, energy use, and environmental impact.[36][37]

Energy required for climate control of the computer's surroundings is often not counted in the wattage calculation, but it can be significant.[38]

SWaP (space, wattage and performance) is a Sun Microsystems metric for data centers, incorporating energy and space:

${\displaystyle \mathrm {SWaP} ={\frac {\mathrm {Performance} }{\mathrm {Space} \cdot \mathrm {Power} }}}$

Where performance is measured by any appropriate benchmark, and space is size of the computer.[39]

Energy efficiency benchmarks

• Average CPU power (ACP) – a measure of power consumption when running several standard benchmarks
• EEMBC – EnergyBench
• SPECpower – a benchmark for web servers running Java (Server Side Java Operations per Joule)

Other

• Data center infrastructure efficiency (DCIE)
• Energy proportional computing
• GeForce 9 series – for GPU list, with energy use and theoretical FLOPS
• IT energy management
• Koomey's law
• Landauer's principle
• Low-power electronics
• Power usage effectiveness (PUE)

• Wu-Chun Feng (October 2003). "Making a case for Efficient Supercomputing". ACM Queue. 1 (7).
• Kirk W. Cameron (November 2013). "HPC Power Efficiency and the Green500". HPC Wire. 27 (11).
• Wu-Chun Feng and Kirk W. Cameron (December 2007). "The Green500 List: Encouraging Sustainable Supercomputing". IEEE Computer. 40 (12).

• 25 Energy Efficient Supercomputers
• The Green 500 Lists