WO2006061404A1 - Cooling system and method - Google Patents

Cooling system and method Download PDF

Info

Publication number
WO2006061404A1
WO2006061404A1 PCT/EP2005/056581 EP2005056581W WO2006061404A1 WO 2006061404 A1 WO2006061404 A1 WO 2006061404A1 EP 2005056581 W EP2005056581 W EP 2005056581W WO 2006061404 A1 WO2006061404 A1 WO 2006061404A1
Authority
WO
WIPO (PCT)
Prior art keywords
coolant
cooling
cooling loop
heat exchanger
loop
Prior art date
Application number
PCT/EP2005/056581
Other languages
French (fr)
Inventor
Levi Campbell
Richard Chu
Michael Ellsworth
Madhusudan Iyengar
Roger Schmidt
Robert Simons
Original Assignee
International Business Machines Corporation
Ibm United Kingdom Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corporation, Ibm United Kingdom Limited filed Critical International Business Machines Corporation
Priority to EP05817223A priority Critical patent/EP1829441B1/en
Priority to JP2007544912A priority patent/JP4511601B2/en
Priority to DE602005005859T priority patent/DE602005005859T2/en
Publication of WO2006061404A1 publication Critical patent/WO2006061404A1/en

Links

Classifications

    • HELECTRICITY
    • H05ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05KPRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
    • H05K7/00Constructional details common to different types of electric apparatus
    • H05K7/20Modifications to facilitate cooling, ventilating, or heating
    • H05K7/20709Modifications to facilitate cooling, ventilating, or heating for server racks or cabinets; for data centers, e.g. 19-inch computer racks
    • H05K7/20763Liquid cooling without phase change
    • H05K7/2079Liquid cooling without phase change within rooms for removing heat from cabinets
    • HELECTRICITY
    • H01ELECTRIC ELEMENTS
    • H01LSEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L23/00Details of semiconductor or other solid state devices
    • H01L23/34Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements
    • H01L23/46Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids
    • H01L23/473Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids by flowing liquids
    • HELECTRICITY
    • H01ELECTRIC ELEMENTS
    • H01LSEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L23/00Details of semiconductor or other solid state devices
    • H01L23/34Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements
    • H01L23/46Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids
    • H01L23/473Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids by flowing liquids
    • H01L23/4735Jet impingement
    • HELECTRICITY
    • H01ELECTRIC ELEMENTS
    • H01LSEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L2924/00Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
    • H01L2924/0001Technical content checked by a classifier
    • H01L2924/0002Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00

Definitions

  • the present invention is directed to cooling assemblies and other apparatus used for removing heat from electronics devices, modules and systems. More particularly, this invention relates to an enhanced cooling system and method for extracting heat from heat generating components of one or more electronics subsystems of one or more electronics racks.
  • a cooling system which includes: a coolant distribution unit, the coolant distribution unit comprising a first heat exchanger, a first cooling loop and at least one second cooling loop, the first cooling loop receiving facility coolant and passing at least a portion thereof through the first heat exchanger, the at least one second cooling loop providing system coolant to at least one electronics subsystem, and expelling heat in the first heat exchanger from the at least one electronics subsystem to the facility coolant in the first cooling loop; and at least one thermal dissipation unit associated with the at least one electronics subsystem, each thermal dissipation unit comprising a second heat exchanger, a second cooling loop of the at least one second cooling loop, and a third cooling loop, the second cooling loop providing system coolant to the second heat exchanger, the third cooling loop circulating conditioned coolant within the at least one electronics subsystem and expelling heat in the second heat exchanger from the at least one electronics subsystem to the system coolant in the second cooling loop.
  • a cooled electronics system in another aspect, includes at least one electronics rack comprising a plurality of electronics subsystems, and a cooling system.
  • the cooling system includes a coolant distribution unit and multiple thermal dissipation units.
  • the coolant distribution unit includes a first heat exchanger, a first cooling loop and a plurality of second cooling loops.
  • the first cooling loop receives facility coolant and passes at least a portion thereof through the first heat exchanger.
  • the plurality of second cooling loops provide system coolant to at least some of the plurality of electronics subsystems, and expels heat in the first heat exchanger from the at least some electronics subsystems to the facility coolant in the first cooling loop.
  • Each thermal dissipation unit is associated with a respective one of the at least some electronics subsystems, and each unit includes a second heat exchanger, a second cooling loop of the plurality of second cooling loops, and a third cooling loop.
  • the second cooling loop provides system coolant to the second heat exchanger
  • the third cooling loop circulates conditioned coolant within the respective electronics subsystem and expels heat in the second heat exchanger from the electronics subsystem to system coolant in the second cooling loop.
  • FIG. 1 depicts a conventional coolant distribution unit, such as a computer room water conditioning unit (CRWCU) , for cooling one or more electronics racks of a computing environment;
  • CCWCU computer room water conditioning unit
  • FIG. 2 is a schematic of one embodiment of an electronics drawer of an electronics rack and a cooling system therefor employing a conventional coolant distribution unit with a facility coolant loop and a system coolant loop;
  • FIG. 3 is a schematic of one embodiment of a cooling system for an electronics subsystem of an electronics rack, which includes a coolant distribution unit and a thermal dissipation unit comprising a conditioned coolant loop within the electronics subsystem, in accordance with an aspect of the present invention
  • FIG. 4A depicts one embodiment of a method of filtering conditioned coolant within the conditioned coolant loop of the cooling system of FIG. 3, in accordance with an aspect of the present invention
  • FIG. 4B depicts one embodiment of the thermal dissipation unit components of FIG. 4A, shown coupled to an electronics module to be cooled, in accordance with an aspect of the present invention
  • FIG. 5A depicts an alternate embodiment of a method of filtering conditioned coolant within the conditioned coolant loop of the cooling system of FIG. 3, in accordance with an aspect of the present invention
  • FIG. 5B depicts one embodiment of the thermal dissipation unit components after filtering of the conditioned coolant, and shown coupled to an electronics module to be cooled, in accordance with an aspect of the present invention
  • FIG. 6A is a cross-sectional elevational view of one embodiment of a micro-scaled cooling structure coupled to an electronics module for indirect expelling of heat from the integrated circuit chips of the module to conditioned coolant within the micro-scaled cooling structure, in accordance with an aspect of the present invention
  • FIG. 6B is a cross-sectional elevational view of an alternate embodiment of a micro-scaled cooling structure coupled to a substrate having a plurality of integrated circuit chips thereon, with conditioned coolant being separated from the integrated circuit chips by an impermeable barrier layer, in accordance with an aspect of the present invention.
  • FIG. 6C is a cross-sectional elevational view of another embodiment of a micro-scaled cooling structure coupled to a substrate having a plurality of integrated circuit chips thereon, wherein the integrated circuit chips are cooled by direct conditioned coolant immersion, in accordance with an aspect of the present invention.
  • an electronics rack includes any frame, rack, blade server system, etc., having a heat generating component of a computer system or electronics system, and may be, for example, a stand alone computer processor having high, mid or low end processing capability.
  • an electronics rack may comprise multiple electronics subsystems, each having one or more heat generating components requiring cooling.
  • Each heat generating component may comprise an electronics device, an electronics module, an integrated circuit chip, etc.
  • micro-scaled cooling structure means a cooling structure with a characteristic dimension of 200 microns or less.
  • coolant within a cooling system in accordance with an aspect of the present invention is water.
  • the concepts disclosed herein are readily adapted to use with other types of coolant on the facility side, system side, and conditioned coolant side of the cooling system.
  • the coolants may comprise a brine, a fluorocarbon liquid, a liquid metal, or other similar coolant, or a refrigerant, while still maintaining the advantages and unique features of the present invention.
  • Heat dissipated by the processor can be transferred to water via a water cooled cold plate.
  • Facility water typically available at customer locations i.e., data centers
  • condensation formation is a concern as the temperature of the data center water, ranging from 7°C to 15°C, is far below the room dew point (typically 18-23°C) .
  • facility water or “facility coolant” refers to, in one example, this data center water or coolant
  • system coolant refers to cooled/conditioned coolant circulating between a coolant distribution unit and the electronics subsystems to be cooled
  • conditioned coolant refers to coolant circulating within a given electronics subsystem.
  • FIG. 1 depicts one embodiment of a coolant distribution unit 100 for a computer room.
  • the coolant distribution unit is conventionally a relatively large unit which occupies more than what would now be considered as two full electronics frames.
  • a power/control element 112 Within the cooling unit 100 is a power/control element 112, a reservoir/expansion tank 113, a heat exchanger 114, a pump 115 (often accompanied by a redundant second pump) , facility water (or site or customer service water or coolant) inlet 116 and outlet 117 supply pipes, a supply manifold 118 directing water to the electronics frames 130 via couplings 120 and lines 122, and a return manifold 119 directing water from the electronics frames 130, via lines 123 and couplings 121.
  • Each electronics rack includes multiple electronics drawers or multiple electronics subsystems 135.
  • FIG. 2 schematically illustrates operation of the cooling system of FIG. 1, wherein a liquid cooled cold plate 155 is shown coupled to an electronics module 150 of electronics drawer 135 within electronics rack 130. Heat is removed from electronics module 150 via the system coolant pumped via pump 115 through cold plate 155 within the system coolant loop defined by heat exchanger 114 of coolant distribution unit 100, lines 122, 123 and cold plate 155.
  • the system coolant loop and coolant distribution unit are designed to provide coolant of a controlled temperature and pressure, as well as controlled chemistry and cleanliness to the electronics.
  • the system coolant is physically separate from the less controlled facility coolant in lines 116, 117 to which heat is ultimately transferred to. Filtration has not been required in a system such as depicted in FIG.
  • system coolant loop has characteristic dimensions for fluid flow that are sufficiently large to allow residual particulate debris to flow freely through the loop.
  • a cold plate with 1.65 mm diameter channels was employed in the ES/9000 system offered by International Business Machines Corporation of Armonk, New York.
  • micro-scaled cooling structures are being developed. Two examples of such structures are marketed by Mikros Manufacturing, Inc., of Claremont, New Hampshire, and ATOTECH of Berlin, Germany. Other examples of micro-scaled cooling structures are also available in the art. These micro-scaled cooling structures have a characteristic dimension more than an order of magnitude less than the cold plates previously employed. Further, the micro-scaled structures have a minimum dimension on the order of or smaller than particulates that regularly circulate through the system coolant of a cooling system such as depicted in FIGS. 1 & 2.
  • the characteristic dimension currently ranges from 50 to 100 micrometers (microns), and could be further reduced as the technology matures. At these small width scales, liquid cleanliness is imperative. At such dimensions, the micro-scaled cooling structure could act more like a filter than a heat sink, thereby inhibiting cooling.
  • an objective of the present invention is to create an isolated subassembly associated with the electronics subsystem which is in thermal contact with the system coolant loop and which is designed and manufactured to accommodate the micro-scale aspects of a micro-scaled cooling structure such as described above.
  • FIG. 3 depicts one embodiment of a cooling system accomplishing this objective.
  • This cooling system or apparatus includes a coolant distribution unit 100 and one or more thermal dissipation units 195. Each thermal dissipation unit 195 is associated with a respective electronics subsystem or drawer 135 of an electronics rack 130 of the computing environment.
  • the coolant distribution unit 100 again includes a first heat exchanger 114, a first cooling loop 116, 117, and one or more second cooling loops 122, 123.
  • the first cooling loop 116, 117 receives facility coolant and passes at least a portion thereof through the first heat exchanger 114.
  • Each second cooling loop provides system coolant to at least one electronics subsystem 135 and expels heat in the first heat exchanger 114 from electronics subsystem 135 to the facility coolant in the first cooling loop 116, 117.
  • System coolant is circulated within the second cooling loop 122, 123 via a pump 115.
  • Each thermal dissipation unit 195 is associated with a respective electronics subsystem 135, and includes a second heat exchanger 160, a second cooling loop 122, 123 of the one or more second cooling loops, a third cooling loop 170, and a micro-scaled cooling structure 180.
  • the second cooling loop provides system coolant to the second heat exchanger 160
  • the third cooling loop circulates conditioned coolant within the at least one electronics subsystem through the micro-scaled cooling structure 180 and expels heat in the second heat exchanger 160 from a heat generating component 190 (e.g., electronics module) of the electronics subsystem 135.
  • the heat is expelled in the heat exchanger to the system coolant in the second cooling loop 122, 123.
  • Conditioned coolant circulates via a pump 175 through the third cooling loop 170 of the thermal dissipation unit 195.
  • a suitable pump 175 is provided in the initially incorporated, commonly assigned, co-filed application entitled: "Cooling Apparatus For An Electronics Subsystem
  • the third cooling loop is a closed loop fluid path, thereby minimizing the opportunity for particulate to enter the cooling loop once the conditioned coolant has been filtered as described below.
  • the third cooling loop is physically isolated from the system coolant of the cooling assembly.
  • the third cooling loop is a separate, dedicated loop or subassembly localized to the electronics subsystem, and to more particularly, the one or more heat generating components, such as an electronic module thereof, that is to be cooled.
  • the third cooling loop and associated components comprise a subassembly that is manufactured to create a pristine environment from both a particulate and materials compatibility (i.e., corrosion) viewpoint.
  • the cooling subassembly 195 is designed to be a closed system once operational (i.e., a system that is not opened in the field) . Being a closed subsystem in the field, particulate contamination can be managed during assembly.
  • FIGS. 4A & 5A depict alternate assemblies for filtering conditioned coolant within the thermal dissipation unit, for example, during fabrication of the unit.
  • the system subassembly 195 of FIG. 3 is shown associated with an electronics module 190, which may be integrated with or coupled to the micro-scaled cooling structure 180.
  • the subassembly includes two three-way valves 200, which are opened in this example to allow coolant flow through a filter 210 rather than through the micro-scaled cooling structure 180.
  • the conditioned coolant pumped 175 through the heat exchanger, the third cooling loop 170 and filter 210 via the three-way valves 200, is purified to a desired level for a particular application.
  • filter 210 can be any filtering mechanism designed to cleanse the conditioned coolant flowing through the third cooling loop 170 in a desired manner, and may include particulate filtering (resulting, e.g., from the manufacturing and assembly process), as well as chemical filtering (e.g., to remove undesired corrosive components from the coolant) .
  • valves 200 are either manually or automatically adjusted to remove filter 210 from the third cooling loop 170, thereby allowing the conditioned coolant to flow through the micro-scaled cooling structure 180 and heat exchanger 160 by means of pump 175 (see FIG. 4B) .
  • FIG. 5A depicts an alternate method for filtering conditioned coolant within loop 170 of the cooling subassembly 195 of FIG. 3.
  • connect/disconnect couplings 220 are employed to connect a filter 210 to the third cooling loop 170.
  • Filter 210 can again comprise any filtering mechanism for removing, for example, undesirable particulate and chemical components from the conditioned coolant flowing through the third cooling loop 170.
  • Coolant is pumped 175 through the heat exchanger 160, the third cooling loop 170 and filter 210 for a sufficient period of time to achieve the desired level of coolant purity.
  • filter 210 is removed and the micro-scaled cooling structure 180 is inserted into the third cooling loop, again using the couplings 220.
  • an electronic module 190 is assumed to be integrated with or coupled to the micro-scaled cooling structure 180.
  • FIGS. 6A-6C Various embodiments for coupling structure 180 to an electronic module are depicted in FIGS. 6A-6C and discussed further below.
  • a cooling assembly which employs three distinct cooling loops.
  • a first cooling loop and a second cooling loop are associated with a coolant distribution unit which includes a fluid-to-fluid heat exchanger to allow the transfer of heat from system coolant within the second cooling loop to facility coolant within the first cooling loop.
  • One or more thermal dissipation units or cooling subassemblies are associated with one or more electronics subsystems of, for example, an electronics rack.
  • Each thermal dissipation unit includes a respective second cooling loop and a third cooling loop, which in one example, comprises an isolated, closed loop flow path.
  • the thermal dissipation unit further includes a second fluid-to-fluid heat exchanger which allows heat to be expelled from conditioned coolant within the third cooling loop to the system coolant within the second cooling loop for transfer to the coolant distribution unit.
  • a second fluid-to-fluid heat exchanger which allows heat to be expelled from conditioned coolant within the third cooling loop to the system coolant within the second cooling loop for transfer to the coolant distribution unit.
  • Coolant purity which allows the use of higher purity coolant within the third cooling loop, less pure coolant within the system coolant loop, and still less pure coolant within the facility coolant loop.
  • High purity coolant is desirable in the third cooling loop of the thermal dissipation unit, particularly when used with small-scale cooling structures (i.e., channels, nozzles, orifices, fins, etc.) to prevent contaminants from interfering with operation of, for example, a micro-scaled cooling structure.
  • Coolant pressure which allows, for example, conditioned coolant within the third cooling loop to be at a pressure below atmospheric pressure, while system coolant and facility coolant in the second cooling loop and the first cooling loop remain at or above atmospheric pressure. This allows, for example, the conditioned coolant to have a different boiling point than the system coolant.
  • Coolant phase change - the third cooling loop allows a conditioned coolant to be employed in a two-phase cooling approach, while maintaining the system coolant and facility coolant as single-phase coolants.
  • Coolant flow rate which may be related to different pressures and phase change temperatures of the various coolants in the cooling system. Further, it may be desirable to employ a lower flow rate through the micro-scaled cooling structure than the flow rate through, for example, the second cooling loop containing the system coolant.
  • Coolant chemistry which allows different coolant fluid chemistries to be employed in the various cooling loops of the cooling system. For example, water could be employed in the first and second cooling loops as both the facility coolant and the system coolant, only of different purity, while the third cooling loop may employ a dielectric as the conditioned coolant. This may be advantageous, for example, in an embodiment where the conditioned coolant directly contacts one or more integrated circuit chips of the electronics subsystem being cooled.
  • FIGS. 6A-6C depict various embodiments for coupling a micro-scaled cooling structure 180 to one or more heat generating components of an electronics subsystem.
  • an electronic module 190 includes a substrate 191 having multiple integrated circuit chips 192 disposed thereon.
  • a module lid 193 encases the integrated circuit chips within the module 190.
  • Module 190 is shown mechanically coupled to a micro-scaled cooling structure 180 through which conditioned coolant (not shown) flows via an inlet 181 and outlet 184.
  • Module lid 193 and micro- scaled cooling structure 180 are fabricated of materials appropriate for facilitating thermal transfer from integrated circuit chips 192 to the conditioned coolant flowing through the micro-scaled structure.
  • the micro-scaled cooling structure 180 may be part of a thermal dissipation unit which is a field replaceable unit.
  • connect/disconnect couplings may be employed in the second cooling loop to couple the system coolant to the newly replaced thermal dissipation unit without requiring opening of the third cooling loop containing the conditioned coolant flowing through the micro-scaled cooling structure 180.
  • FIG. 6B depicts an alternate embodiment of a micro-scaled cooling structure coupled to a chip assembly, which is referred to herein as near- direct coolant immersion.
  • cooling structure 180' couples to a substrate 191 having multiple integrated circuit chips 192 thereon.
  • a multi-layer impermeable barrier 194 resides over the integrated circuit chips and protects the chips from the conditioned coolant flowing through the micro-scaled cooling structure 180'.
  • Conditioned coolant flows onto the impermeable barrier 194 via micro- scaled orifices 183 in fluid communication with a supply manifold 182 receiving conditioned coolant via an inlet 181.
  • Conditioned coolant flows from the integrated assembly via an outlet 184 in the micro-scaled cooling structure 180'.
  • any type of coolant may be employed, with water being one example.
  • liquid is in near-direct contact with the integrated circuit chips, but remains isolated therefrom.
  • near-direct integrated cooling structure and module assemblies are described in greater detail in commonly assigned United States Patent No. US 6,587,345 entitled “Electronic Device Substrate Assembly With Impermeable Barrier And Method Of Making” and United States Patent Application No. 2004/0012914 Al, entitled “Electronic Device Substrate Assembly With Multilayer Impermeable Barrier And Method Of Making", both of which are hereby incorporated herein by reference in their entirety.
  • FIG. 6C depicts still another attachment embodiment for integrating a micro-scaled cooling structure with an integrated circuit assembly.
  • This embodiment is referred to as direct coolant immersion since the conditioned coolant impinges directly onto the multiple integrated circuit chips 192 disposed on substrate 191.
  • the micro-scaled cooling structure 180' again includes micro-scale orifices 183 which provide conditioned coolant from a supply manifold 182 in fluid communication with an inlet 181 coupled to the third cooling loop of the respective thermal dissipation unit.
  • the micro-scaled cooling structure 180' in FIG. 6C includes an outlet 184, which is in fluid communication with the third cooling loop 170 of the thermal dissipation unit 195 of, for example, FIG. 3.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • General Physics & Mathematics (AREA)
  • Power Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Thermal Sciences (AREA)
  • Cooling Or The Like Of Electrical Apparatus (AREA)
  • Cooling Or The Like Of Semiconductors Or Solid State Devices (AREA)
  • Motor Or Generator Cooling System (AREA)

Abstract

A cooling approach is provided for one or more subsystems of an electronics rack. The cooling approach employs a coolant distribution unit (100) and a thermal dissipation unit (195). The coolant distribution unit has a first heat exchanger (114), a first cooling loop (116, 117) and a second cooling loop (122, 123). The first cooling loop passes facility coolant through the first heat exchanger, and the second cooling loop provides system coolant to an electronics subsystem, and expels heat in the first heat exchanger from the subsystem to the facility coolant. The thermal dissipation unit (195) is associated with the electronics subsystem and includes a second heat exchanger (160), the second cooling loop (122, 123), and a third cooling loop (170). The second cooling loop provides system coolant to the second heat exchanger, and the third cooling loop circulates conditioned coolant within the electronics subsystem and expels heat in the second heat exchanger from the electronics subsystem to the system coolant.

Description

COOLING SYSTEM AND METHOD
Technical Field
The present invention is directed to cooling assemblies and other apparatus used for removing heat from electronics devices, modules and systems. More particularly, this invention relates to an enhanced cooling system and method for extracting heat from heat generating components of one or more electronics subsystems of one or more electronics racks.
Background of the Invention
As is well known, as the circuit density of electronic chip devices increases in order to achieve faster and faster processing speed, there is a correspondingly increasing demand for the removal of heat generated by these devices. The increased heat demand arises both because the circuit devices are packed more closely together and because the circuits themselves are operated at increasingly higher clock frequencies. Nonetheless, it is also known that runaway thermal conditions and excessive heat generated by chips is a leading cause of failure of chip devices. Furthermore, it is anticipated that demand for heat removal from these devices will increase indefinitely. Accordingly, it is seen that there is a large and significant need to provide useful cooling mechanisms for electronic circuit devices.
Each new generation of computers continues to offer increased speed and function. In most cases, this has been accomplished by a combination of increased power dissipation and increased packaging density. The net result has been increased heat flux at all levels of packaging. For example, a common packaging configuration for many large computer systems today is a multi-drawer rack, with each drawer containing one or more processor modules along with associated electronics, such as memory, power and hard drive devices. These drawers are removable units so that in the event of failure of an individual drawer, the drawer may be removed and replaced in the field. The problem with this configuration is that the increase in heat flux at the electronics drawer level makes it increasingly difficult to dissipate heat by simple air cooling. Summary of the Invention
The shortcomings of the prior art are overcome and additional advantages are provided through a cooling system which includes: a coolant distribution unit, the coolant distribution unit comprising a first heat exchanger, a first cooling loop and at least one second cooling loop, the first cooling loop receiving facility coolant and passing at least a portion thereof through the first heat exchanger, the at least one second cooling loop providing system coolant to at least one electronics subsystem, and expelling heat in the first heat exchanger from the at least one electronics subsystem to the facility coolant in the first cooling loop; and at least one thermal dissipation unit associated with the at least one electronics subsystem, each thermal dissipation unit comprising a second heat exchanger, a second cooling loop of the at least one second cooling loop, and a third cooling loop, the second cooling loop providing system coolant to the second heat exchanger, the third cooling loop circulating conditioned coolant within the at least one electronics subsystem and expelling heat in the second heat exchanger from the at least one electronics subsystem to the system coolant in the second cooling loop.
In another aspect, a cooled electronics system is provided. The cooled electronics system includes at least one electronics rack comprising a plurality of electronics subsystems, and a cooling system. The cooling system includes a coolant distribution unit and multiple thermal dissipation units. The coolant distribution unit includes a first heat exchanger, a first cooling loop and a plurality of second cooling loops. The first cooling loop receives facility coolant and passes at least a portion thereof through the first heat exchanger. The plurality of second cooling loops provide system coolant to at least some of the plurality of electronics subsystems, and expels heat in the first heat exchanger from the at least some electronics subsystems to the facility coolant in the first cooling loop. Each thermal dissipation unit is associated with a respective one of the at least some electronics subsystems, and each unit includes a second heat exchanger, a second cooling loop of the plurality of second cooling loops, and a third cooling loop. The second cooling loop provides system coolant to the second heat exchanger, and the third cooling loop circulates conditioned coolant within the respective electronics subsystem and expels heat in the second heat exchanger from the electronics subsystem to system coolant in the second cooling loop.
Methods of fabricating a cooling system for an electronics subsystem and methods of cooling an electronics rack comprising a plurality of electronics subsystems are also described and claimed.
Further, additional features and advantages are realized through the techniques of the present invention. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention.
Brief Description of the Drawings
The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 depicts a conventional coolant distribution unit, such as a computer room water conditioning unit (CRWCU) , for cooling one or more electronics racks of a computing environment;
FIG. 2 is a schematic of one embodiment of an electronics drawer of an electronics rack and a cooling system therefor employing a conventional coolant distribution unit with a facility coolant loop and a system coolant loop;
FIG. 3 is a schematic of one embodiment of a cooling system for an electronics subsystem of an electronics rack, which includes a coolant distribution unit and a thermal dissipation unit comprising a conditioned coolant loop within the electronics subsystem, in accordance with an aspect of the present invention;
FIG. 4A depicts one embodiment of a method of filtering conditioned coolant within the conditioned coolant loop of the cooling system of FIG. 3, in accordance with an aspect of the present invention; FIG. 4B depicts one embodiment of the thermal dissipation unit components of FIG. 4A, shown coupled to an electronics module to be cooled, in accordance with an aspect of the present invention;
FIG. 5A depicts an alternate embodiment of a method of filtering conditioned coolant within the conditioned coolant loop of the cooling system of FIG. 3, in accordance with an aspect of the present invention;
FIG. 5B depicts one embodiment of the thermal dissipation unit components after filtering of the conditioned coolant, and shown coupled to an electronics module to be cooled, in accordance with an aspect of the present invention;
FIG. 6A is a cross-sectional elevational view of one embodiment of a micro-scaled cooling structure coupled to an electronics module for indirect expelling of heat from the integrated circuit chips of the module to conditioned coolant within the micro-scaled cooling structure, in accordance with an aspect of the present invention;
FIG. 6B is a cross-sectional elevational view of an alternate embodiment of a micro-scaled cooling structure coupled to a substrate having a plurality of integrated circuit chips thereon, with conditioned coolant being separated from the integrated circuit chips by an impermeable barrier layer, in accordance with an aspect of the present invention; and
FIG. 6C is a cross-sectional elevational view of another embodiment of a micro-scaled cooling structure coupled to a substrate having a plurality of integrated circuit chips thereon, wherein the integrated circuit chips are cooled by direct conditioned coolant immersion, in accordance with an aspect of the present invention.
Best Mode for Carrying Out the Invention
As used herein, "electronics subsystem" comprises any housing, compartment, drawer, blade, etc., containing one or more heat generating components of a computer system or other electronics system requiring cooling. The term "electronics rack" includes any frame, rack, blade server system, etc., having a heat generating component of a computer system or electronics system, and may be, for example, a stand alone computer processor having high, mid or low end processing capability. In one embodiment, an electronics rack may comprise multiple electronics subsystems, each having one or more heat generating components requiring cooling. Each heat generating component may comprise an electronics device, an electronics module, an integrated circuit chip, etc. As used herein, "micro-scaled cooling structure" means a cooling structure with a characteristic dimension of 200 microns or less.
One example of coolant within a cooling system in accordance with an aspect of the present invention is water. However, the concepts disclosed herein are readily adapted to use with other types of coolant on the facility side, system side, and conditioned coolant side of the cooling system. For example, one or more of the coolants may comprise a brine, a fluorocarbon liquid, a liquid metal, or other similar coolant, or a refrigerant, while still maintaining the advantages and unique features of the present invention.
As noted briefly above, power levels in computer equipment (primarily processors) are again rising to a level where they no longer can be simply air cooled. The components will likely be water cooled.
Heat dissipated by the processor can be transferred to water via a water cooled cold plate. Facility water typically available at customer locations (i.e., data centers) is not suitable for use in these cold plates. First, condensation formation is a concern as the temperature of the data center water, ranging from 7°C to 15°C, is far below the room dew point (typically 18-23°C) . Second, the relatively poor quality of the facility water (in chemistry, cleanliness, etc.) impacts system reliability. It is therefore desirable to utilize a water cooling/conditioning unit that circulates higher quality water to/from the electronics subsystems and rejects the heat to the data center water. As used herein, "facility water" or "facility coolant" refers to, in one example, this data center water or coolant, while "system coolant" refers to cooled/conditioned coolant circulating between a coolant distribution unit and the electronics subsystems to be cooled, and "conditioned coolant" refers to coolant circulating within a given electronics subsystem.
Reference is now made to the drawings, wherein the same reference numbers used throughout different figures designate the same or similar components. FIG. 1 depicts one embodiment of a coolant distribution unit 100 for a computer room. The coolant distribution unit is conventionally a relatively large unit which occupies more than what would now be considered as two full electronics frames. Within the cooling unit 100 is a power/control element 112, a reservoir/expansion tank 113, a heat exchanger 114, a pump 115 (often accompanied by a redundant second pump) , facility water (or site or customer service water or coolant) inlet 116 and outlet 117 supply pipes, a supply manifold 118 directing water to the electronics frames 130 via couplings 120 and lines 122, and a return manifold 119 directing water from the electronics frames 130, via lines 123 and couplings 121. Each electronics rack includes multiple electronics drawers or multiple electronics subsystems 135.
FIG. 2 schematically illustrates operation of the cooling system of FIG. 1, wherein a liquid cooled cold plate 155 is shown coupled to an electronics module 150 of electronics drawer 135 within electronics rack 130. Heat is removed from electronics module 150 via the system coolant pumped via pump 115 through cold plate 155 within the system coolant loop defined by heat exchanger 114 of coolant distribution unit 100, lines 122, 123 and cold plate 155. The system coolant loop and coolant distribution unit are designed to provide coolant of a controlled temperature and pressure, as well as controlled chemistry and cleanliness to the electronics. Furthermore, the system coolant is physically separate from the less controlled facility coolant in lines 116, 117 to which heat is ultimately transferred to. Filtration has not been required in a system such as depicted in FIG. 2 since the system coolant loop has characteristic dimensions for fluid flow that are sufficiently large to allow residual particulate debris to flow freely through the loop. For example, a cold plate with 1.65 mm diameter channels was employed in the ES/9000 system offered by International Business Machines Corporation of Armonk, New York.
As noted, processor power levels continue to rise as designers push for ever increasing computing performance. Electronic module power levels are expected to go well beyond conventional air cooling technologies, and even beyond conventional liquid cooling cold plate concepts. To address these future cooling needs, micro-scaled cooling structures are being developed. Two examples of such structures are marketed by Mikros Manufacturing, Inc., of Claremont, New Hampshire, and ATOTECH of Berlin, Germany. Other examples of micro-scaled cooling structures are also available in the art. These micro-scaled cooling structures have a characteristic dimension more than an order of magnitude less than the cold plates previously employed. Further, the micro-scaled structures have a minimum dimension on the order of or smaller than particulates that regularly circulate through the system coolant of a cooling system such as depicted in FIGS. 1 & 2. In available micro-scaled structures, the characteristic dimension currently ranges from 50 to 100 micrometers (microns), and could be further reduced as the technology matures. At these small width scales, liquid cleanliness is imperative. At such dimensions, the micro-scaled cooling structure could act more like a filter than a heat sink, thereby inhibiting cooling.
One solution to the problem would be to introduce a filter into the system coolant side of the cooling assembly of FIGS. 1 & 2. This, unfortunately, would be undesirable because it would add additional pressure drop and would require continuing maintenance. Thus, in one aspect, an objective of the present invention is to create an isolated subassembly associated with the electronics subsystem which is in thermal contact with the system coolant loop and which is designed and manufactured to accommodate the micro-scale aspects of a micro-scaled cooling structure such as described above.
FIG. 3 depicts one embodiment of a cooling system accomplishing this objective. This cooling system or apparatus includes a coolant distribution unit 100 and one or more thermal dissipation units 195. Each thermal dissipation unit 195 is associated with a respective electronics subsystem or drawer 135 of an electronics rack 130 of the computing environment. The coolant distribution unit 100 again includes a first heat exchanger 114, a first cooling loop 116, 117, and one or more second cooling loops 122, 123. The first cooling loop 116, 117 receives facility coolant and passes at least a portion thereof through the first heat exchanger 114. Each second cooling loop provides system coolant to at least one electronics subsystem 135 and expels heat in the first heat exchanger 114 from electronics subsystem 135 to the facility coolant in the first cooling loop 116, 117. System coolant is circulated within the second cooling loop 122, 123 via a pump 115. Each thermal dissipation unit 195 is associated with a respective electronics subsystem 135, and includes a second heat exchanger 160, a second cooling loop 122, 123 of the one or more second cooling loops, a third cooling loop 170, and a micro-scaled cooling structure 180. The second cooling loop provides system coolant to the second heat exchanger 160, and the third cooling loop circulates conditioned coolant within the at least one electronics subsystem through the micro-scaled cooling structure 180 and expels heat in the second heat exchanger 160 from a heat generating component 190 (e.g., electronics module) of the electronics subsystem 135. The heat is expelled in the heat exchanger to the system coolant in the second cooling loop 122, 123. Conditioned coolant circulates via a pump 175 through the third cooling loop 170 of the thermal dissipation unit 195. One example of a suitable pump 175 is provided in the initially incorporated, commonly assigned, co-filed application entitled: "Cooling Apparatus For An Electronics Subsystem
Employing A Coolant Flow Drive Apparatus Between Coolant Flow Paths". In one example, the third cooling loop is a closed loop fluid path, thereby minimizing the opportunity for particulate to enter the cooling loop once the conditioned coolant has been filtered as described below.
Advantageously, the third cooling loop is physically isolated from the system coolant of the cooling assembly. The third cooling loop is a separate, dedicated loop or subassembly localized to the electronics subsystem, and to more particularly, the one or more heat generating components, such as an electronic module thereof, that is to be cooled. The third cooling loop and associated components comprise a subassembly that is manufactured to create a pristine environment from both a particulate and materials compatibility (i.e., corrosion) viewpoint. The cooling subassembly 195 is designed to be a closed system once operational (i.e., a system that is not opened in the field) . Being a closed subsystem in the field, particulate contamination can be managed during assembly.
FIGS. 4A & 5A depict alternate assemblies for filtering conditioned coolant within the thermal dissipation unit, for example, during fabrication of the unit. In FIG. 4A, the system subassembly 195 of FIG. 3 is shown associated with an electronics module 190, which may be integrated with or coupled to the micro-scaled cooling structure 180. The subassembly includes two three-way valves 200, which are opened in this example to allow coolant flow through a filter 210 rather than through the micro-scaled cooling structure 180. The conditioned coolant pumped 175 through the heat exchanger, the third cooling loop 170 and filter 210 via the three-way valves 200, is purified to a desired level for a particular application. Note that filter 210 can be any filtering mechanism designed to cleanse the conditioned coolant flowing through the third cooling loop 170 in a desired manner, and may include particulate filtering (resulting, e.g., from the manufacturing and assembly process), as well as chemical filtering (e.g., to remove undesired corrosive components from the coolant) . Once filtered, valves 200 are either manually or automatically adjusted to remove filter 210 from the third cooling loop 170, thereby allowing the conditioned coolant to flow through the micro-scaled cooling structure 180 and heat exchanger 160 by means of pump 175 (see FIG. 4B) .
FIG. 5A depicts an alternate method for filtering conditioned coolant within loop 170 of the cooling subassembly 195 of FIG. 3. In this embodiment, connect/disconnect couplings 220 are employed to connect a filter 210 to the third cooling loop 170. Filter 210 can again comprise any filtering mechanism for removing, for example, undesirable particulate and chemical components from the conditioned coolant flowing through the third cooling loop 170. Coolant is pumped 175 through the heat exchanger 160, the third cooling loop 170 and filter 210 for a sufficient period of time to achieve the desired level of coolant purity.
As shown in FIG. 5B, after the conditioned coolant has been appropriately filtered, filter 210 is removed and the micro-scaled cooling structure 180 is inserted into the third cooling loop, again using the couplings 220. In the embodiment shown, an electronic module 190 is assumed to be integrated with or coupled to the micro-scaled cooling structure 180. Various embodiments for coupling structure 180 to an electronic module are depicted in FIGS. 6A-6C and discussed further below.
Those skilled in the art will note that provided herein is a cooling assembly which employs three distinct cooling loops. A first cooling loop and a second cooling loop are associated with a coolant distribution unit which includes a fluid-to-fluid heat exchanger to allow the transfer of heat from system coolant within the second cooling loop to facility coolant within the first cooling loop. One or more thermal dissipation units or cooling subassemblies are associated with one or more electronics subsystems of, for example, an electronics rack. Each thermal dissipation unit includes a respective second cooling loop and a third cooling loop, which in one example, comprises an isolated, closed loop flow path. The thermal dissipation unit further includes a second fluid-to-fluid heat exchanger which allows heat to be expelled from conditioned coolant within the third cooling loop to the system coolant within the second cooling loop for transfer to the coolant distribution unit. Advantageously, by separating the conditioned coolant, system coolant and facility coolant, each coolant loop can have coolant of different properties or characteristics. These different characteristics can include different:
• Coolant purity - which allows the use of higher purity coolant within the third cooling loop, less pure coolant within the system coolant loop, and still less pure coolant within the facility coolant loop. High purity coolant is desirable in the third cooling loop of the thermal dissipation unit, particularly when used with small-scale cooling structures (i.e., channels, nozzles, orifices, fins, etc.) to prevent contaminants from interfering with operation of, for example, a micro-scaled cooling structure.
• Coolant pressure - which allows, for example, conditioned coolant within the third cooling loop to be at a pressure below atmospheric pressure, while system coolant and facility coolant in the second cooling loop and the first cooling loop remain at or above atmospheric pressure. This allows, for example, the conditioned coolant to have a different boiling point than the system coolant.
• Coolant phase change - the third cooling loop allows a conditioned coolant to be employed in a two-phase cooling approach, while maintaining the system coolant and facility coolant as single-phase coolants.
• Coolant flow rate - which may be related to different pressures and phase change temperatures of the various coolants in the cooling system. Further, it may be desirable to employ a lower flow rate through the micro-scaled cooling structure than the flow rate through, for example, the second cooling loop containing the system coolant. • Coolant chemistry - which allows different coolant fluid chemistries to be employed in the various cooling loops of the cooling system. For example, water could be employed in the first and second cooling loops as both the facility coolant and the system coolant, only of different purity, while the third cooling loop may employ a dielectric as the conditioned coolant. This may be advantageous, for example, in an embodiment where the conditioned coolant directly contacts one or more integrated circuit chips of the electronics subsystem being cooled.
As noted, FIGS. 6A-6C depict various embodiments for coupling a micro-scaled cooling structure 180 to one or more heat generating components of an electronics subsystem. In FIG. 6A, an electronic module 190 includes a substrate 191 having multiple integrated circuit chips 192 disposed thereon. A module lid 193 encases the integrated circuit chips within the module 190. Module 190 is shown mechanically coupled to a micro-scaled cooling structure 180 through which conditioned coolant (not shown) flows via an inlet 181 and outlet 184. Module lid 193 and micro- scaled cooling structure 180 are fabricated of materials appropriate for facilitating thermal transfer from integrated circuit chips 192 to the conditioned coolant flowing through the micro-scaled structure. As one enhancement, the micro-scaled cooling structure 180 may be part of a thermal dissipation unit which is a field replaceable unit. In such a case, connect/disconnect couplings may be employed in the second cooling loop to couple the system coolant to the newly replaced thermal dissipation unit without requiring opening of the third cooling loop containing the conditioned coolant flowing through the micro-scaled cooling structure 180.
FIG. 6B depicts an alternate embodiment of a micro-scaled cooling structure coupled to a chip assembly, which is referred to herein as near- direct coolant immersion. In this embodiment, cooling structure 180' couples to a substrate 191 having multiple integrated circuit chips 192 thereon. A multi-layer impermeable barrier 194 resides over the integrated circuit chips and protects the chips from the conditioned coolant flowing through the micro-scaled cooling structure 180'. Conditioned coolant flows onto the impermeable barrier 194 via micro- scaled orifices 183 in fluid communication with a supply manifold 182 receiving conditioned coolant via an inlet 181. Conditioned coolant flows from the integrated assembly via an outlet 184 in the micro-scaled cooling structure 180'. With this embodiment, any type of coolant may be employed, with water being one example. Advantageously, liquid is in near-direct contact with the integrated circuit chips, but remains isolated therefrom. Examples of near-direct integrated cooling structure and module assemblies are described in greater detail in commonly assigned United States Patent No. US 6,587,345 entitled "Electronic Device Substrate Assembly With Impermeable Barrier And Method Of Making" and United States Patent Application No. 2004/0012914 Al, entitled "Electronic Device Substrate Assembly With Multilayer Impermeable Barrier And Method Of Making", both of which are hereby incorporated herein by reference in their entirety.
FIG. 6C depicts still another attachment embodiment for integrating a micro-scaled cooling structure with an integrated circuit assembly. This embodiment is referred to as direct coolant immersion since the conditioned coolant impinges directly onto the multiple integrated circuit chips 192 disposed on substrate 191. As shown, the micro-scaled cooling structure 180' again includes micro-scale orifices 183 which provide conditioned coolant from a supply manifold 182 in fluid communication with an inlet 181 coupled to the third cooling loop of the respective thermal dissipation unit. The micro-scaled cooling structure 180' in FIG. 6C includes an outlet 184, which is in fluid communication with the third cooling loop 170 of the thermal dissipation unit 195 of, for example, FIG. 3. With direct coolant immersion, the substrate and the cooling assembly are an integrated unit, and no fluid barrier exists between the conditioned coolant and the integrated circuit chips. This is possible by using a dielectric coolant selected so as not to damage the integrated circuit chips. A more detailed discussion of an integrated micro-scaled cooling structure and circuit subassembly is provided in the initially incorporated, commonly assigned, co-filed application entitled: "Cooling Apparatus And Method For An Electronics Module Employing An Integrated Heat Exchange Assembly".
Although preferred embodiments have been depicted and described in detail herein, it will be apparent to those skilled in the relevant art that various modifications, additions, substitutions and the like can be made without departing from the spirit of the invention and these are therefore considered to be within the scope of the invention as defined in the following claims.

Claims

1. A cooling system comprising: a coolant distribution unit, the coolant distribution unit comprising a first heat exchanger, a first cooling loop and at least one second cooling loop, the first cooling loop receiving facility coolant and passing at least a portion thereof through the first heat exchanger, the at least one second cooling loop providing system coolant to at least one electronics subsystem, and expelling heat in the first heat exchanger from the at least one electronics subsystem to the facility coolant in the first cooling loop; and at least one thermal dissipation unit associated with the at least one electronics subsystem, each thermal dissipation unit comprising a second heat exchanger, a second cooling loop of the at least one second cooling loop, and a third cooling loop, the second cooling loop providing system coolant to the second heat exchanger, the third cooling loop circulating conditioned coolant within the at least one electronics subsystem and expelling heat in the second heat exchanger from the at least one electronics subsystem to the system coolant in the second cooling loop.
2. The cooling system of claim 1, wherein the conditioned coolant in the third cooling loop and the system coolant in the second cooling loop differ by at least one characteristic, the at least one characteristic comprising at least one of: coolant purity; coolant pressure; coolant flow rate; coolant phase change temperature; and coolant chemistry.
3. The cooling system of claim 2, wherein the system coolant in the second cooling loop and the facility coolant in the first cooling loop differ by at least one characteristic, the at least one characteristic comprising at least one of: coolant purity; coolant pressure; coolant flow rate; coolant phase change temperature; and coolant chemistry.
4. The cooling system of claim 1, 2 or 3 wherein the third cooling loop of each thermal dissipation unit comprises an isolated, closed loop fluid path disposed within the at least one electronics subsystem.
5. The cooling system of claim 4, wherein each thermal dissipation unit further includes at least one micro-scaled cooling structure and a coolant pump for moving conditioned coolant through the closed loop fluid path of the third cooling loop between the second heat exchanger and the micro- scaled cooling structure.
6. The cooling system of claim 5, wherein the micro-scaled cooling structure is coupled to at least one heat generating component of the at least one electronics subsystem for facilitating expelling heat therefrom to the conditioned coolant within the closed loop fluid path of the third cooling loop.
7. The cooling system of claim 6, wherein each thermal dissipation unit comprises a field replaceable unit, and wherein the micro-scaled cooling structure mechanically couples to a surface of the heat generating component, the heat generating component comprising an electronic module.
8. The cooling system of claim 6 or 7, wherein the at least one heat generating component comprises at least one integrated circuit chip, and wherein the conditioned coolant within the third cooling loop directly contacts the at least one integrated circuit chip of the at least one electronics subsystem.
9. The cooling system of any of claims 1 to 8, wherein the conditioned coolant comprises a dielectric fluid or purified water.
10. The cooling system of claim 6, 7 or 8 wherein the at least one heat generating component comprises at least one integrated circuit chip, and wherein each thermal dissipation unit further includes a thermally conductive fluid barrier separating the conditioned coolant in the third cooling loop from the at least one integrated circuit chip.
11. A cooled electronics system comprising: at least one electronics rack comprising a plurality of electronics subsystems; and a cooling system comprising: a coolant distribution unit, the coolant distribution unit comprising a first heat exchanger, a first cooling loop and a plurality of second cooling loops, the first cooling loop receiving facility coolant and passing at least a portion thereof through the first heat exchanger, the plurality of second cooling loops providing system coolant to at least some of the plurality of electronics subsystems, and expelling heat in the first heat exchanger from the at least some electronics subsystems to the facility coolant in the first cooling loop; and multiple thermal dissipation units each associated with a respective one of the at least some electronics subsystems, each thermal dissipation unit comprising a second heat exchanger, a second cooling loop of the plurality of second cooling loops, and a third cooling loop, the second cooling loop providing system coolant to the second heat exchanger, the third cooling loop circulating conditioned coolant within the respective electronics subsystem and expelling heat in the second heat exchanger from the electronics subsystem to the system coolant in the second cooling loop.
12. A method of cooling at least one electronics subsystem, the method comprising: providing a coolant distribution unit, the coolant distribution unit comprising a first heat exchanger, a first cooling loop and at least one second cooling loop, the first cooling loop receiving facility coolant and passing at least a portion thereof through the first heat exchanger, the at least one second cooling loop providing system coolant to at least one electronics subsystem, and expelling heat in the first heat exchanger from the at least one electronics subsystem to the facility coolant in the first cooling loop; and providing at least one thermal dissipation unit associated with the at least one electronics subsystem, each thermal dissipation unit comprising a second heat exchanger, a second cooling loop of the at least one second cooling loop, and a third cooling loop, the second cooling loop providing system coolant to the second heat exchanger, the third cooling loop circulating conditioned coolant within the at least one electronics subsystem and expelling heat in the second heat exchanger from the at least one electronics subsystem to the system coolant in the second cooling loop.
PCT/EP2005/056581 2004-12-09 2005-12-07 Cooling system and method WO2006061404A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP05817223A EP1829441B1 (en) 2004-12-09 2005-12-07 Cooling system and method
JP2007544912A JP4511601B2 (en) 2004-12-09 2005-12-07 Cooling system and method
DE602005005859T DE602005005859T2 (en) 2004-12-09 2005-12-07 COOLING SYSTEM AND METHOD

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/008,711 2004-12-09
US11/008,711 US6973801B1 (en) 2004-12-09 2004-12-09 Cooling system and method employing a closed loop coolant path and micro-scaled cooling structure within an electronics subsystem of an electronics rack

Publications (1)

Publication Number Publication Date
WO2006061404A1 true WO2006061404A1 (en) 2006-06-15

Family

ID=35452409

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2005/056581 WO2006061404A1 (en) 2004-12-09 2005-12-07 Cooling system and method

Country Status (7)

Country Link
US (1) US6973801B1 (en)
EP (1) EP1829441B1 (en)
JP (1) JP4511601B2 (en)
CN (1) CN100556260C (en)
AT (1) ATE391406T1 (en)
DE (1) DE602005005859T2 (en)
WO (1) WO2006061404A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10772234B2 (en) 2017-02-28 2020-09-08 Fujitsu Limited Cooling device and electronic device system

Families Citing this family (107)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8464781B2 (en) 2002-11-01 2013-06-18 Cooligy Inc. Cooling systems incorporating heat exchangers and thermoelectric layers
US7836597B2 (en) 2002-11-01 2010-11-23 Cooligy Inc. Method of fabricating high surface to volume ratio structures and their integration in microheat exchangers for liquid cooling system
US7508672B2 (en) * 2003-09-10 2009-03-24 Qnx Cooling Systems Inc. Cooling system
JP4468379B2 (en) * 2004-11-29 2010-05-26 三菱重工業株式会社 Air refrigerant type refrigeration heating equipment
US8424885B2 (en) 2005-12-22 2013-04-23 Elliptical Mobile Solutions, LLC Method and apparatus for an environmentally-protected electronic equipment enclosure
US7578337B2 (en) * 2005-04-14 2009-08-25 United States Thermoelectric Consortium Heat dissipating device
US7298617B2 (en) * 2005-10-25 2007-11-20 International Business Machines Corporation Cooling apparatus and method employing discrete cold plates disposed between a module enclosure and electronics components to be cooled
US7298618B2 (en) * 2005-10-25 2007-11-20 International Business Machines Corporation Cooling apparatuses and methods employing discrete cold plates compliantly coupled between a common manifold and electronics components of an assembly to be cooled
US7814965B1 (en) 2005-10-27 2010-10-19 United States Thermoelectric Consortium Airflow heat dissipation device
US7992625B1 (en) 2006-08-18 2011-08-09 United States Thermoelectric Consortium Fluid-operated heat transfer device
US7365973B2 (en) 2006-01-19 2008-04-29 American Power Conversion Corporation Cooling system and method
US8672732B2 (en) 2006-01-19 2014-03-18 Schneider Electric It Corporation Cooling system and method
TW200813695A (en) * 2006-03-30 2008-03-16 Cooligy Inc Integrated liquid to air conduction module
DK2032907T3 (en) 2006-06-01 2018-07-02 Google Llc Hot cooling for electronics
US8322155B2 (en) 2006-08-15 2012-12-04 American Power Conversion Corporation Method and apparatus for cooling
US8327656B2 (en) 2006-08-15 2012-12-11 American Power Conversion Corporation Method and apparatus for cooling
US9568206B2 (en) 2006-08-15 2017-02-14 Schneider Electric It Corporation Method and apparatus for cooling
US20080043442A1 (en) * 2006-08-16 2008-02-21 Strickland Travis C Computer system with thermal conduction
US7665325B2 (en) * 2006-09-12 2010-02-23 International Business Machines Corporation Multi-fluid cooling system and method with freeze protection for cooling an electronic device
US7681404B2 (en) 2006-12-18 2010-03-23 American Power Conversion Corporation Modular ice storage for uninterruptible chilled water
US8425287B2 (en) 2007-01-23 2013-04-23 Schneider Electric It Corporation In-row air containment and cooling system and method
US8141620B1 (en) 2007-02-26 2012-03-27 United States Thermoelectric Consortium (USTC) Method for conditioning a cooling loop of a heat exchange system
US7511959B2 (en) * 2007-04-25 2009-03-31 Hewlett-Packard Development Company, L.P. Scalable computing apparatus
US7477514B2 (en) * 2007-05-04 2009-01-13 International Business Machines Corporation Method of facilitating cooling of electronics racks of a data center employing multiple cooling stations
EP2147585B1 (en) 2007-05-15 2016-11-02 Schneider Electric IT Corporation Method and system for managing facility power and cooling
US7602609B2 (en) * 2007-05-31 2009-10-13 Liebert Corporation Cooling system and method of use
TW200934352A (en) * 2007-08-07 2009-08-01 Cooligy Inc Internal access mechanism for a server rack
US9496200B2 (en) * 2011-07-27 2016-11-15 Coolit Systems, Inc. Modular heat-transfer systems
US9943014B2 (en) 2013-03-15 2018-04-10 Coolit Systems, Inc. Manifolded heat exchangers and related systems
US20090229283A1 (en) * 2007-08-24 2009-09-17 Joseph Marsala Method and apparatus for isothermal cooling of hard disk drive arrays using a pumped refrigerant loop
US7963119B2 (en) * 2007-11-26 2011-06-21 International Business Machines Corporation Hybrid air and liquid coolant conditioning unit for facilitating cooling of one or more electronics racks of a data center
US8553416B1 (en) * 2007-12-21 2013-10-08 Exaflop Llc Electronic device cooling system with storage
US8175753B2 (en) * 2008-02-15 2012-05-08 The Pnc Financial Services Group, Inc. Systems and methods for computer equipment management
US20090225514A1 (en) 2008-03-10 2009-09-10 Adrian Correa Device and methodology for the removal of heat from an equipment rack by means of heat exchangers mounted to a door
US8299604B2 (en) 2008-08-05 2012-10-30 Cooligy Inc. Bonded metal and ceramic plates for thermal management of optical and electronic devices
US20100085708A1 (en) * 2008-10-07 2010-04-08 Liebert Corporation High-efficiency, fluid-cooled ups converter
US7983040B2 (en) 2008-10-23 2011-07-19 International Business Machines Corporation Apparatus and method for facilitating pumped immersion-cooling of an electronic subsystem
US7961475B2 (en) 2008-10-23 2011-06-14 International Business Machines Corporation Apparatus and method for facilitating immersion-cooling of an electronic subsystem
US7885070B2 (en) * 2008-10-23 2011-02-08 International Business Machines Corporation Apparatus and method for immersion-cooling of an electronic system utilizing coolant jet impingement and coolant wash flow
US7916483B2 (en) 2008-10-23 2011-03-29 International Business Machines Corporation Open flow cold plate for liquid cooled electronic packages
US7944694B2 (en) 2008-10-23 2011-05-17 International Business Machines Corporation Liquid cooling apparatus and method for cooling blades of an electronic system chassis
US7903404B2 (en) * 2009-04-29 2011-03-08 Hewlett-Packard Development Company, L.P. Data centers
US8219362B2 (en) 2009-05-08 2012-07-10 American Power Conversion Corporation System and method for arranging equipment in a data center
US8369090B2 (en) 2009-05-12 2013-02-05 Iceotope Limited Cooled electronic system
WO2010130993A2 (en) * 2009-05-12 2010-11-18 Iceotope Limited Cooled electronic system
US8081461B2 (en) * 2009-06-25 2011-12-20 International Business Machines Corporation Cooling apparatus with thermally conductive porous material and jet impingement nozzle(s) extending therein
US8490419B2 (en) * 2009-08-20 2013-07-23 United States Thermoelectric Consortium Interlocked jets cooling method and apparatus
US8522569B2 (en) 2009-10-27 2013-09-03 Industrial Idea Partners, Inc. Utilization of data center waste heat for heat driven engine
US8189334B2 (en) 2010-05-26 2012-05-29 International Business Machines Corporation Dehumidifying and re-humidifying cooling apparatus and method for an electronics rack
US7905096B1 (en) * 2010-05-26 2011-03-15 International Business Machines Corporation Dehumidifying and re-humidifying air cooling for an electronics rack
US8144467B2 (en) 2010-05-26 2012-03-27 International Business Machines Corporation Dehumidifying and re-humidifying apparatus and method for an electronics rack
US9038406B2 (en) 2010-05-26 2015-05-26 International Business Machines Corporation Dehumidifying cooling apparatus and method for an electronics rack
US8179677B2 (en) 2010-06-29 2012-05-15 International Business Machines Corporation Immersion-cooling apparatus and method for an electronic subsystem of an electronics rack
US8184436B2 (en) 2010-06-29 2012-05-22 International Business Machines Corporation Liquid-cooled electronics rack with immersion-cooled electronic subsystems
US8351206B2 (en) 2010-06-29 2013-01-08 International Business Machines Corporation Liquid-cooled electronics rack with immersion-cooled electronic subsystems and vertically-mounted, vapor-condensing unit
US8369091B2 (en) 2010-06-29 2013-02-05 International Business Machines Corporation Interleaved, immersion-cooling apparatus and method for an electronic subsystem of an electronics rack
US8345423B2 (en) 2010-06-29 2013-01-01 International Business Machines Corporation Interleaved, immersion-cooling apparatuses and methods for cooling electronic subsystems
US8514575B2 (en) 2010-11-16 2013-08-20 International Business Machines Corporation Multimodal cooling apparatus for an electronic system
US8274790B2 (en) 2010-11-16 2012-09-25 International Business Machines Corporation Automatically reconfigurable liquid-cooling apparatus for an electronics rack
US8688413B2 (en) 2010-12-30 2014-04-01 Christopher M. Healey System and method for sequential placement of cooling resources within data center layouts
US10365667B2 (en) 2011-08-11 2019-07-30 Coolit Systems, Inc. Flow-path controllers and related systems
US8760863B2 (en) 2011-10-31 2014-06-24 International Business Machines Corporation Multi-rack assembly with shared cooling apparatus
US8817474B2 (en) 2011-10-31 2014-08-26 International Business Machines Corporation Multi-rack assembly with shared cooling unit
US20130112378A1 (en) * 2011-11-04 2013-05-09 Dell Products L.P. System and Method for Cooling Information Handling Resources
CN104137660B (en) 2011-12-22 2017-11-24 施耐德电气It公司 System and method for the predicting temperature values in electronic system
AU2011384046A1 (en) 2011-12-22 2014-07-17 Schneider Electric It Corporation Analysis of effect of transient events on temperature in a data center
US9167728B2 (en) * 2012-07-18 2015-10-20 International Business Machines Corporation Information technology equipment cooling method
US8964384B2 (en) * 2012-07-27 2015-02-24 Hewlett-Packard Development Company, L.P. Component cooling
US20140048993A1 (en) * 2012-08-17 2014-02-20 Apple Inc. Selective placement of coolant to undercut areas
US8925333B2 (en) 2012-09-13 2015-01-06 International Business Machines Corporation Thermoelectric-enhanced air and liquid cooling of an electronic system
US9351431B2 (en) 2012-10-11 2016-05-24 International Business Machines Corporation Cooling system with automated seasonal freeze protection
US9648784B2 (en) 2013-03-15 2017-05-09 Inertech Ip Llc Systems and assemblies for cooling server racks
US10364809B2 (en) 2013-03-15 2019-07-30 Coolit Systems, Inc. Sensors, multiplexed communication techniques, and related systems
WO2014158196A1 (en) * 2013-03-29 2014-10-02 Hewlett-Packard Development Company, L.P. Electronic apparatus having a cooling apparatus
TW201515563A (en) * 2013-10-08 2015-04-16 Hon Hai Prec Ind Co Ltd Heat dissipating system
US10197310B2 (en) 2014-06-20 2019-02-05 Nortek Air Solutions Canada, Inc. Systems and methods for managing conditions in enclosed space
US10168061B2 (en) * 2014-11-11 2019-01-01 Emerson Network Power S.R.L. Conditioning system of the free cooling type for environments, method of operation of a said conditioning system, and apparatus for carrying out such method
US10638641B2 (en) 2014-11-14 2020-04-28 Hewlett Packard Enterprise Development Lp Cooling rack
US11092349B2 (en) 2015-05-15 2021-08-17 Nortek Air Solutions Canada, Inc. Systems and methods for providing cooling to a heat load
CN107850335B (en) 2015-05-15 2021-02-19 北狄空气应对加拿大公司 Liquid cooling using liquid-gas membrane energy exchangers
WO2017052628A1 (en) * 2015-09-25 2017-03-30 Intel Corporation Computing apparatus with closed cooling loop
CA3010515C (en) 2016-01-08 2023-03-21 Nortek Air Solutions Canada, Inc. Integrated make-up air system in 100% air recirculation system
KR102599450B1 (en) 2016-03-16 2023-11-09 이너테크 아이피 엘엘씨 Systems and methods for performing series heat dissipation and coordinated cooling using fluid coolers and chillers
EP3236725B8 (en) * 2016-04-18 2020-03-11 Rolls-Royce plc Power electronics module
US11452243B2 (en) 2017-10-12 2022-09-20 Coolit Systems, Inc. Cooling system, controllers and methods
DE202018102582U1 (en) 2018-01-23 2018-05-22 MEGWARE Computer Vertrieb und Service GmbH High temperature direct liquid cooled high performance computer system
US10681846B2 (en) 2018-04-19 2020-06-09 Google Llc Cooling electronic devices in a data center
US10645847B2 (en) 2018-04-20 2020-05-05 Google Llc Cooling electronic devices in a data center
US10966352B2 (en) 2018-09-24 2021-03-30 Google Llc Cooling electronic devices in a data center
US10548239B1 (en) * 2018-10-23 2020-01-28 Google Llc Cooling electronic devices in a data center
GB201916771D0 (en) 2019-11-18 2020-01-01 Iceotope Group Ltd Heat sink for liquid cooling
US10548240B1 (en) 2019-01-11 2020-01-28 Google Llc Cooling electronic devices in a data center
US11662037B2 (en) 2019-01-18 2023-05-30 Coolit Systems, Inc. Fluid flow control valve for fluid flow systems, and methods
WO2020210587A1 (en) 2019-04-10 2020-10-15 Jetcool Technologies, Inc. Thermal management of electronics using co-located microjet nozzles and electronic elements
CN113994772B (en) 2019-04-14 2023-05-16 捷控技术有限公司 Cooling module based on direct contact fluid
US11473860B2 (en) 2019-04-25 2022-10-18 Coolit Systems, Inc. Cooling module with leak detector and related systems
CA3140764A1 (en) * 2019-06-27 2020-12-30 Hypertechnologie Ciara Inc. Microgap system for cooling electronics with direct contact
US11801733B2 (en) 2019-09-25 2023-10-31 Baidu Usa Llc Liquid cooling loop design for high performance processors in harsh vehicle environment
US10959352B1 (en) * 2020-01-03 2021-03-23 Quanta Computer Inc. Cooling system with floating cold plate with single pipes
WO2021229365A1 (en) 2020-05-11 2021-11-18 Coolit Systems, Inc. Liquid pumping units, and related systems and methods
US11963341B2 (en) 2020-09-15 2024-04-16 Jetcool Technologies Inc. High temperature electronic device thermal management system
US11211538B1 (en) 2020-12-23 2021-12-28 Joseph L. Pikulski Thermal management system for electrically-powered devices
US11903166B2 (en) * 2021-02-01 2024-02-13 Microsoft Technology Licensing, Llc Systems and methods for immersion cooling with subcooled spray
US12082382B2 (en) * 2021-04-23 2024-09-03 Nvidia Corporation Intelligent in-rack pump or compressor unit for datacenter cooling systems
US11812589B2 (en) * 2021-05-12 2023-11-07 Nvidia Corporation Intelligent refrigerant distribution unit for datacenter cooling systems
US12048118B2 (en) * 2021-08-13 2024-07-23 Jetcool Technologies Inc. Flow-through, hot-spot-targeting immersion cooling assembly
TW202407925A (en) 2022-03-04 2024-02-16 美商捷控技術有限公司 Actively cooled heat-dissipation lids for computer processors and processor assemblies

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5903433A (en) * 1994-12-14 1999-05-11 Telefonaktiebolaget Lm Ericsson Cooling system for telecommunications equipment arranged in a cabinet or similar
US20020104646A1 (en) * 2001-02-08 2002-08-08 Jeong In Kwon Multi-channel temperature control system for semiconductor processing facilities
US20020134544A1 (en) * 2000-09-07 2002-09-26 Thermotek, Inc. Passive cooling system and method

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3032590A1 (en) * 1980-08-27 1982-04-01 Siemens AG, 1000 Berlin und 8000 München Cooling system for electrical machines - has prim. cooling agent in direct heat exchange contact with evaporator of heat pump
JPS62119620A (en) * 1985-11-20 1987-05-30 Fujitsu Ltd Constitution system for cooling water supply equipment
US5131233A (en) 1991-03-08 1992-07-21 Cray Computer Corporation Gas-liquid forced turbulence cooling
JP2874684B2 (en) * 1997-03-27 1999-03-24 日本電気株式会社 Heat dissipation structure of plug-in unit
US5924482A (en) 1997-10-29 1999-07-20 Motorola, Inc. Multi-mode, two-phase cooling module
US6019165A (en) 1998-05-18 2000-02-01 Batchelder; John Samuel Heat exchange apparatus
US6408937B1 (en) 2000-11-15 2002-06-25 Sanjay K. Roy Active cold plate/heat sink
US6604370B2 (en) 2001-02-22 2003-08-12 Hewlett-Packard Development Company, L.P. Variably configured sprayjet cooling system
US6498725B2 (en) 2001-05-01 2002-12-24 Mainstream Engineering Corporation Method and two-phase spray cooling apparatus
TW561226B (en) 2001-09-25 2003-11-11 Matsushita Electric Ind Co Ltd Ultra-thin pump and cooling system including the pump
US6587345B2 (en) 2001-11-09 2003-07-01 International Business Machines Corporation Electronic device substrate assembly with impermeable barrier and method of making
US6628520B2 (en) * 2002-02-06 2003-09-30 Hewlett-Packard Development Company, L.P. Method, apparatus, and system for cooling electronic components
US6625023B1 (en) 2002-04-11 2003-09-23 General Dynamics Land Systems, Inc. Modular spray cooling system for electronic components
US6604571B1 (en) 2002-04-11 2003-08-12 General Dynamics Land Systems, Inc. Evaporative cooling of electrical components
US6940712B2 (en) 2002-07-17 2005-09-06 International Business Machines Corporation Electronic device substrate assembly with multilayer impermeable barrier and method of making
US6807056B2 (en) * 2002-09-24 2004-10-19 Hitachi, Ltd. Electronic equipment
US7146822B2 (en) 2002-12-30 2006-12-12 Intel Corporation Centrifugal liquid pump with perimeter magnetic drive
CN100338983C (en) * 2003-12-03 2007-09-19 国际商业机器公司 Cooling system and method employing for ensuring cooling of multiple electronics subsystems
US7106590B2 (en) * 2003-12-03 2006-09-12 International Business Machines Corporation Cooling system and method employing multiple dedicated coolant conditioning units for cooling multiple electronics subsystems

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5903433A (en) * 1994-12-14 1999-05-11 Telefonaktiebolaget Lm Ericsson Cooling system for telecommunications equipment arranged in a cabinet or similar
US20020134544A1 (en) * 2000-09-07 2002-09-26 Thermotek, Inc. Passive cooling system and method
US20020104646A1 (en) * 2001-02-08 2002-08-08 Jeong In Kwon Multi-channel temperature control system for semiconductor processing facilities

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10772234B2 (en) 2017-02-28 2020-09-08 Fujitsu Limited Cooling device and electronic device system
US11317534B2 (en) 2017-02-28 2022-04-26 Fujitsu Limited Cooling device and electronic device system

Also Published As

Publication number Publication date
ATE391406T1 (en) 2008-04-15
DE602005005859T2 (en) 2009-05-20
JP4511601B2 (en) 2010-07-28
DE602005005859D1 (en) 2008-05-15
CN100556260C (en) 2009-10-28
JP2008523599A (en) 2008-07-03
EP1829441B1 (en) 2008-04-02
CN101091424A (en) 2007-12-19
EP1829441A1 (en) 2007-09-05
US6973801B1 (en) 2005-12-13

Similar Documents

Publication Publication Date Title
EP1829441B1 (en) Cooling system and method
US7272005B2 (en) Multi-element heat exchange assemblies and methods of fabrication for a cooling system
US7274566B2 (en) Cooling apparatus for an electronics subsystem employing a coolant flow drive apparatus between coolant flow paths
US7184269B2 (en) Cooling apparatus and method for an electronics module employing an integrated heat exchange assembly
US10070560B2 (en) Drawer-level immersion-cooling with hinged, liquid-cooled heat sink
US7298618B2 (en) Cooling apparatuses and methods employing discrete cold plates compliantly coupled between a common manifold and electronics components of an assembly to be cooled
US9743561B2 (en) Liquid-cooled heat sink configured to facilitate drainage
US8245401B2 (en) Casted heat sink and tube cold plate with peritectically reacted metals
US9282678B2 (en) Field-replaceable bank of immersion-cooled electronic components and separable heat sinks
JP5671731B2 (en) Liquid cooling system, electronic equipment rack, and manufacturing method thereof
US9253921B2 (en) Coolant-conditioning unit with automated control of coolant flow valves

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KN KP KR KZ LC LK LR LS LT LU LV LY MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2007544912

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 200580042255.3

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2005817223

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2005817223

Country of ref document: EP

WWG Wipo information: grant in national office

Ref document number: 2005817223

Country of ref document: EP