Editing Supercomputer (section)

==Energy usage and heat management==
{{See also|Computer cooling|Green500}}
[[File:Summit (supercomputer).jpg|thumb|upright|220px|The [[Summit (supercomputer)|Summit]] supercomputer was as of November 2018 the fastest supercomputer in the world.<ref name ="nytimes">{{cite news|url=https://www.nytimes.com/2018/06/08/technology/supercomputer-china-us.html|title=Move Over, China: U.S. Is Again Home to World's Speediest Supercomputer|last=Lohr|first=Steve|date=8 June 2018|newspaper=New York Times|access-date=19 July 2018}}</ref> With a measured power efficiency of 14.668 GFlops/watt it is also the third most energy efficient in the world.<ref name="greenlistjune2018">{{cite web|url=https://www.top500.org/green500/lists/2018/11/|title=Green500 List - November 2018|website=TOP500|language=en|access-date=19 July 2018}}</ref>]]
Throughout the decades, the management of [[heat density]] has remained a key issue for most centralized supercomputers.<ref name=TH1 >{{cite journal |title=The TianHe-1A Supercomputer: Its Hardware and Software |author= Xue-June Yang |author2=Xiang-Ke Liao |display-authors=etal |journal=Journal of Computer Science and Technology | volume= 26 |issue= 3 |pages= 344–351 |doi= 10.1007/s02011-011-1137-8 |year= 2011 |s2cid= 1389468 }}</ref><ref name=Charley>''The Supermen: Story of Seymour Cray and the Technical Wizards Behind the Supercomputer'' by Charles J. Murray 1997, {{ISBN|0-471-04885-2}}, pages 133–135</ref><ref name=Rupak>''Parallel Computational Fluid Dyynamics; Recent Advances and Future Directions'' edited by Rupak Biswas 2010 {{ISBN|1-60595-022-X}} page 401</ref> The large amount of heat generated by a system may also have other effects, e.g. reducing the lifetime of other system components.<ref name=Huang313 >''Supercomputing Research Advances'' by Yongge Huáng 2008, {{ISBN|1-60456-186-6}}, pages 313–314</ref> There have been diverse approaches to heat management, from pumping [[Fluorinert]] through the system, to a hybrid liquid-air cooling system or air cooling with normal [[air conditioning]] temperatures.<ref name=Tokhi >''Parallel computing for real-time signal processing and control'' by M. O. Tokhi, Mohammad Alamgir Hossain 2003, {{ISBN|978-1-85233-599-1}}, pages 201–202</ref><ref name=sysx /> A typical supercomputer consumes large amounts of electrical power, almost all of which is converted into heat, requiring cooling. For example, [[Tianhe-1A]] consumes 4.04&nbsp;[[megawatt]]s (MW) of electricity.<ref>{{cite press release
 | url=http://pressroom.nvidia.com/easyir/customrel.do?easyirid=A0D622CE9F579F09&version=live&prid=678988&releasejsp=release_157
 | title=NVIDIA Tesla GPUs Power World's Fastest Supercomputer
 | publisher=Nvidia
 | date=29 October 2010
 | access-date=21 February 2011
 | archive-date=2 March 2014
 | archive-url=https://web.archive.org/web/20140302031237/http://pressroom.nvidia.com/easyir/customrel.do?easyirid=A0D622CE9F579F09&version=live&prid=678988&releasejsp=release_157
 | url-status=dead
 }}</ref> The cost to power and cool the system can be significant, e.g. 4&nbsp;MW at $0.10/kWh is $400 an hour or about $3.5 million per year.

[[File:IBM HS20 blade server.jpg|thumb|left|An [[IBM BladeCenter#HS20|IBM HS20]] [[blade server|blade]] ]]
Heat management is a major issue in complex electronic devices and affects powerful computer systems in various ways.<ref name=Spectrum >{{cite web |title=Better Computing Through CPU Cooling |first= Alexander A. |last=Balandin |publisher=[[IEEE]] |date= October 2009 |url=https://spectrum.ieee.org/semiconductors/materials/better-computing-through-cpu-cooling/0 |archive-url=https://archive.today/20120714070104/http://spectrum.ieee.org/semiconductors/materials/better-computing-through-cpu-cooling/0 |url-status=dead |archive-date=14 July 2012 }}</ref> The [[thermal design power]] and [[CPU power dissipation]] issues in supercomputing surpass those of traditional [[computer cooling]] technologies. The supercomputing awards for [[green computing]] reflect this issue.<ref>{{cite web | url = http://www.green500.org/ | title = The Green 500 | publisher = Green500.org | access-date = 14 August 2011 | archive-date = 26 August 2016 | archive-url = https://web.archive.org/web/20160826075608/http://www.green500.org/ | url-status = dead }}</ref><ref>{{cite web  | url = http://www.itnews.com.au/News/65619,green-500-list-ranks-supercomputers.aspx  | work = iTnews Australia  | title = Green 500 list ranks supercomputers  | url-status = dead  | archive-url = https://web.archive.org/web/20081022193316/http://www.itnews.com.au/News/65619,green-500-list-ranks-supercomputers.aspx  | archive-date = 22 October 2008 | df = dmy-all  }}</ref><ref name=WuFeng>{{cite journal |author=Wu-chun Feng |year=2003 |title=Making a Case for Efficient Supercomputing {{pipe}} ACM Queue Magazine, Volume 1 Issue 7, 10 January 2003 doi 10.1145/957717.957772 |journal=Queue |volume=1 |issue=7 |pages=54 |doi=10.1145/957717.957772 |s2cid=11283177 |doi-access=free }}</ref>

The packing of thousands of processors together inevitably generates significant amounts of [[heat density]] that need to be dealt with. The [[Cray-2]] was [[Computer cooling|liquid cooled]], and used a [[Fluorinert]] "cooling waterfall" which was forced through the modules under pressure.<ref name="Tokhi"/> However, the submerged liquid cooling approach was not practical for the multi-cabinet systems based on off-the-shelf processors, and in [[System X (supercomputer)|System X]] a special cooling system that combined air conditioning with liquid cooling was developed in conjunction with the [[Liebert (company)|Liebert company]].<ref name=sysx >''Computational science – ICCS 2005: 5th international conference'' edited by Vaidy S. Sunderam 2005, {{ISBN|3-540-26043-9}}, pages 60–67</ref>

In the [[Blue Gene]] system, IBM deliberately used low power processors to deal with heat density.<ref name="TheRegSC10">{{cite web
 |title=IBM uncloaks 20 petaflops BlueGene/Q super
 |website=The Register
 |date=22 November 2010
|url=https://www.theregister.co.uk/2010/11/22/ibm_blue_gene_q_super/
 |access-date=25 November 2010
}}</ref> The IBM [[Power 775]], released in 2011, has closely packed elements that require water cooling.<ref>{{cite web|last=Prickett |first=Timothy |url=https://www.theregister.co.uk/2011/07/15/power_775_super_pricing/ |title=''The Register'': IBM 'Blue Waters' super node washes ashore in August |publisher=Theregister.co.uk |date=15 July 2011 |access-date=9 June 2012}}</ref> The IBM [[Aquasar]] system uses hot water cooling to achieve energy efficiency, the water being used to heat buildings as well.<ref>{{cite web |url=https://www-03.ibm.com/press/us/en/pressrelease/32049.wss |title=IBM Hot Water-Cooled Supercomputer Goes Live at ETH Zurich |website=IBM News room |date=2 July 2010 |url-status=dead |archive-url=https://web.archive.org/web/20110110032000/https://www-03.ibm.com/press/us/en/pressrelease/32049.wss |archive-date=10 January 2011 |access-date=16 March 2020  }}</ref><ref>{{cite web |author=Martin LaMonica |url=http://news.cnet.com/8301-11128_3-20004543-54.html |title=CNet 10 May 2010 |publisher=News.cnet.com |date=10 May 2010 |access-date=9 June 2012 |archive-date=1 November 2013 |archive-url=https://web.archive.org/web/20131101060256/http://news.cnet.com/8301-11128_3-20004543-54.html |url-status=dead }}</ref>

The energy efficiency of computer systems is generally measured in terms of "[[FLOPS per watt]]". In 2008, [[Roadrunner (supercomputer)|Roadrunner]] by [[IBM]] operated at 376&nbsp;[[Performance per watt|MFLOPS/W]].<ref>{{cite news | url = http://www.cnn.com/2008/TECH/06/09/fastest.computer.ap/index.html  | work = CNN | title = Government unveils world's fastest computer |quote= performing 376 million calculations for every watt of electricity used. |archive-url = https://web.archive.org/web/20080610155646/http://www.cnn.com/2008/TECH/06/09/fastest.computer.ap/index.html |archive-date = 10 June 2008}}</ref><ref>{{cite web|url = http://www.hpcwire.com/topic/processors/IBM_Roadrunner_Takes_the_Gold_in_the_Petaflop_Race.html|title = IBM Roadrunner Takes the Gold in the Petaflop Race|url-status = live|archive-url = https://web.archive.org/web/20081217131938/http://www.hpcwire.com/topic/processors/IBM_Roadrunner_Takes_the_Gold_in_the_Petaflop_Race.html|archive-date = 17 December 2008|access-date=16 March 2020|df = dmy-all}}</ref>{{cbignore|bot=InternetArchiveBot}} In November 2010, the [[IBM Blue Gene#Blue Gene/Q|Blue Gene/Q]] reached 1,684&nbsp;MFLOPS/W<ref>{{cite web| url=http://www.serverwatch.com/hreviews/article.php/3913536/Top500-Supercomputing-List-Reveals-Computing-Trends.htm| title = Top500 Supercomputing List Reveals Computing Trends| date = 20 July 2010|quote=IBM... BlueGene/Q system .. setting a record in power efficiency with a value of 1,680&nbsp;MFLOPS/W, more than twice that of the next best system.}}</ref><ref>{{cite web| url=http://www.datacenterknowledge.com/archives/2010/11/18/ibm-system-clear-winner-in-green-500/|title = IBM Research A Clear Winner in Green 500|date = 18 November 2010}}</ref> and in June 2011 the top two spots on the [[Green 500]] list were occupied by [[Blue Gene]] machines in New York (one achieving 2097&nbsp;MFLOPS/W) with the [[DEGIMA (computer cluster)|DEGIMA cluster]] in Nagasaki placing third with 1375&nbsp;MFLOPS/W.<ref>{{cite web |url=http://www.green500.org/lists/2011/06/top/list.php |archive-url=https://web.archive.org/web/20110703094255/http://www.green500.org/lists/2011/06/top/list.php |url-status=live |archive-date=3 July 2011 |title=Green 500 list |publisher=Green500.org |access-date=16 March 2020 }}</ref>{{cbignore|bot=InternetArchiveBot}}

Because copper wires can transfer energy into a supercomputer with much higher power densities than forced air or circulating refrigerants can remove [[waste heat]],<ref>
Saed G. Younis.
[http://hdl.handle.net/1721.1/7058 "Asymptotically Zero Energy Computing Using Split-Level Charge Recovery Logic"].
1994.
page 14.
</ref> the ability of the cooling systems to remove waste heat is a limiting factor.<ref>
[http://www.pnl.gov/computing/resources/esdc/1_Cooling.stm "Hot Topic – the Problem of Cooling Supercomputers"] {{webarchive|url=https://web.archive.org/web/20150118222233/http://www.pnl.gov/computing/resources/esdc/1_Cooling.stm |date=18 January 2015 }}.
</ref><ref>
Anand Lal Shimpi.
[http://www.anandtech.com/show/6421/inside-the-titan-supercomputer-299k-amd-x86-cores-and-186k-nvidia-gpu-cores "Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K NVIDIA GPUs"].
2012.
</ref> {{As of|2015}}, many existing supercomputers have more infrastructure capacity than the actual peak demand of the machine{{snd}} designers generally conservatively design the power and cooling infrastructure to handle more than the theoretical peak electrical power consumed by the supercomputer. Designs for future supercomputers are power-limited{{snd}} the [[thermal design power]] of the supercomputer as a whole, the amount that the power and cooling infrastructure can handle, is somewhat more than the expected normal power consumption, but less than the theoretical peak power consumption of the electronic hardware.<ref>
Curtis Storlie; Joe Sexton; Scott Pakin; Michael Lang; Brian Reich; William Rust.
[https://arxiv.org/abs/1412.5247 "Modeling and Predicting Power Consumption of High-Performance Computing Jobs"].
2014.
</ref>
<!-- Editorial note: Can improve section by mentioning network interconnect technologies, e.g. >10 Gb ethernet, InfiniBand, FC SAN etc.-->