WO2022126937A1 - Boîtier de plateforme en nuage de fpga haute densité de type tiroir - Google Patents
Boîtier de plateforme en nuage de fpga haute densité de type tiroir Download PDFInfo
- Publication number
- WO2022126937A1 WO2022126937A1 PCT/CN2021/085808 CN2021085808W WO2022126937A1 WO 2022126937 A1 WO2022126937 A1 WO 2022126937A1 CN 2021085808 W CN2021085808 W CN 2021085808W WO 2022126937 A1 WO2022126937 A1 WO 2022126937A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- fpga
- drawer
- cloud platform
- chassis
- power supply
- Prior art date
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 4
- 238000001816 cooling Methods 0.000 claims description 6
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 claims description 4
- 229910052802 copper Inorganic materials 0.000 claims description 4
- 239000010949 copper Substances 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 2
- 230000003993 interaction Effects 0.000 abstract description 7
- 238000012423 maintenance Methods 0.000 abstract description 6
- 238000011161 development Methods 0.000 abstract description 4
- 238000007726 management method Methods 0.000 description 14
- 238000012544 monitoring process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000000034 method Methods 0.000 description 5
- 230000017525 heat dissipation Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 2
- 229910052737 gold Inorganic materials 0.000 description 2
- 239000010931 gold Substances 0.000 description 2
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
- H05K7/00—Constructional details common to different types of electric apparatus
- H05K7/14—Mounting supporting structure in casing or on frame or rack
- H05K7/1485—Servers; Data center rooms, e.g. 19-inch computer racks
- H05K7/1487—Blade assemblies, e.g. blade cases or inner arrangements within a blade
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
- H05K7/00—Constructional details common to different types of electric apparatus
- H05K7/14—Mounting supporting structure in casing or on frame or rack
- H05K7/1485—Servers; Data center rooms, e.g. 19-inch computer racks
- H05K7/1488—Cabinets therefor, e.g. chassis or racks or mechanical interfaces between blades and support structures
- H05K7/1489—Cabinets therefor, e.g. chassis or racks or mechanical interfaces between blades and support structures characterized by the mounting of blades therein, e.g. brackets, rails, trays
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
- H05K7/00—Constructional details common to different types of electric apparatus
- H05K7/14—Mounting supporting structure in casing or on frame or rack
- H05K7/1485—Servers; Data center rooms, e.g. 19-inch computer racks
- H05K7/1488—Cabinets therefor, e.g. chassis or racks or mechanical interfaces between blades and support structures
- H05K7/1491—Cabinets therefor, e.g. chassis or racks or mechanical interfaces between blades and support structures having cable management arrangements
Definitions
- the invention relates to the field of computer architecture and FPGA (Field Programmable Gate Array) heterogeneous acceleration, in particular to a cloud platform server based on a programmable gate array.
- FPGA Field Programmable Gate Array
- FPGA has gradually become one of the application solutions considered in various fields of computer due to its advantages of high energy efficiency, parallel computing and multiple programming, especially in media compression, encryption and decryption, AI, big data processing and other fields, FPGA solutions are more traditional CPU and GPU can often achieve several times or even dozens of times of energy efficiency improvement. Due to this trend, the FPGA cloud platform came into being.
- the FPGA chassis is to the FPGA cloud platform what the commercial standard x86 chassis is to the cloud computing platform.
- the FPGA chassis is mainly based on a standard x86 server, supplemented by an FPGA hardware card in the form of a PCIe interface on the x86 motherboard inside the chassis.
- power supplies, fans, motherboards, and service boards are usually placed on the same layer or on the same plane.
- the modules are generally placed in front of each other and connected by cables. Such an organizational structure complicates chassis assembly.
- the circuit board components in the chassis are mostly composed of an x86 server motherboard (control board) and multiple FPGA node boards (service boards).
- the FPGA node board is inserted into the x86 server control board through the gold finger of the PCIe interface. data interaction.
- the FPGA cloud platform chassis can deploy no more than 8 FPGA nodes according to its own chassis size and the number of PCIe interfaces of the motherboard x86 server, so the FPGA node resources that can provide services in a single chassis are limited.
- the existing cloud computing framework and commercial x86 server chassis if a large-scale deployment is carried out, a large number of x86 servers are required to carry FPGA node boards, which leads to higher deployment costs and lower utilization of physical space in the computer room. Therefore, in this traditional FPGA cloud platform chassis mode, it is difficult for FPGA nodes to be deployed on a large scale and at high density.
- the current commercial FPGA cloud platforms usually provide x86 servers and FPGA nodes as resources to cloud users. Users can develop their own application software in the x86 server, and can also complete the development of FPGA acceleration logic in the x86 server. This method will result in waste of resources and increased cost for users who only need FPGA resources.
- the existing FPGA node boards are managed and configured through the x86 server motherboard in the chassis. Users must first access the x86 server, and then start the relevant processes in the x86 server to manage and configure the FPGA node boards. Configuration, flexibility is not strong.
- the invention effectively reduces the cost of FPGA cloud platform chassis deployment and maintenance; in the limited space chassis, the deployment density of FPGA node boards is greatly improved; since x86 servers are not used as FPGA nodes Management boards, which can save more economic costs when deploying large-scale FPGA nodes;
- the FPGA node boards in the chassis can be managed and configured more reasonably and efficiently, and the resource utilization efficiency of the cloud platform can be improved.
- the present invention proposes a drawer-type high-density FPGA cloud platform chassis, which includes:
- a switch module located at the bottom of the chassis, a power supply module located above the switch module, and a drawer structure located above the power supply module;
- the drawer structure is provided with a control board and an FPGA node board, and the FPGA node board is plugged into the control board through a preset interface;
- the power transmission end of the power supply module is electrically connected to the power input interface of the switch module and the control board, and the network switch interface of the switch module is connected to the network interface of the FPGA node board, and is used to exchange the communication between the FPGA node boards. data.
- the drawer structure further includes a cooling fan, and a handle for loading and unloading is installed at the rear of the drawer structure.
- the FPGA node board further has a PCIe interface, and the FPGA node board can be connected to the x86 server through the PCIe interface.
- the power supply module includes an electronic power supply module for backup power supply.
- the chassis is a standard 5U server chassis.
- the inner side of the side wall of the chassis is provided with a slide rail, which is connected to the drawer structure through the slide rail.
- the power supply module of the FPGA node board card is connected to the power take-off clip of the control board card through a power supply copper bar.
- the drawer-type high-density FPGA cloud platform chassis further includes a restraint device for fixing the FPGA node board, and the restraint device includes: an opening mesh plate located on one side of the FPGA node board, located on the other side of the FPGA node board.
- the front panel on one side, the cover above the FPGA node board.
- the chassis includes a plurality of the drawer structures arranged side by side on the power supply module.
- control board controls and manages the FPGA node board through the preset interface and monitors the working state of the board in real time.
- the present invention has the advantages that the present invention will greatly increase the deployable density of the FPGA node boards in the FPGA cloud platform chassis. Reduce the wiring cost in the chassis, simplify the assembly complexity, and reduce the maintenance difficulty. By using the self-developed control management system, it provides users with a more comprehensive and convenient development environment. Real-time monitoring of the chassis and board status and reduce unnecessary wiring, improve the reliability of the FPGA cloud platform chassis.
- Fig. 1 is the overall structure diagram of the present invention
- Fig. 2 is the internal structure diagram of the single drawer structure of the present invention.
- Figure 3 is a side view of the present invention.
- FIG. 4 is a partial view of the chassis of the present invention.
- the present invention effectively reduces the deployment and maintenance costs of the FPGA cloud platform chassis 1; the stacking layout and the double-drawer structure can greatly improve the deployment of the FPGA node board 8 in a limited chassis space Density; through the control management system designed in "a cloud platform computing system and its application method” (application number 201810532745.0) and “a method, device and system for implementing an FPGA server” (application number 202010019013.9), more efficient Manage and configure FPGA node boards, providing users with a convenient, fast and cheaper FPGA resource usage environment.
- a cloud platform computing system and its application method application number 201810532745.0
- a method, device and system for implementing an FPGA server application number 202010019013.9
- the technical difficulty of the present invention lies in how to realize the deployment of high-density FPGA nodes in the limited cloud platform chassis space, and how to effectively manage, configure and use the high-density FPGA nodes in the chassis.
- this application includes the following key points:
- the integrated drawer structure track is shown in Figure 4.
- the drawer structure 2 When the drawer structure 2 is installed, first put down along the vertical part of the track 13, and then push it inward along the horizontal part of the track 13.
- the rails 13 are located on both sides of the inner wall of the chassis 1 .
- the drawer structure 2 has all the working elements (control board 6, high-density business board, cooling fan 12, etc.), and can work as an independent system after external power supply; technical effect: easy to use and fast to assemble;
- Key point 2 Stacked structure layout, the drawer structure 2, power supply module 3, and 100G switching module 4 in the chassis 1 are arranged up and down, saving the depth space of the chassis 1;
- Technical effect efficient use of space and increase the node density in the chassis 1;
- Key point 3 Independent control and management unit, through the control board 6 to complete the control and management of the FPGA node board 8 and real-time monitoring of the working status of the board; technical effect: rational allocation of resources, monitoring board status;
- Key point 4 The design of the preset interface 10, the status information, control information and debugging interface of the FPGA node board 8 can be interacted with through the preset interface 10; technical effect: Simplify the internal wiring of the chassis 1.
- the chassis 1 is composed of a double drawer structure 2 , a power supply module 3 and a 100G switching module 4 respectively from top to bottom.
- the two drawer structures 2 have the same structure, and are mainly used for placing high-density service boards and their cooling systems, and two handles 5 are installed at the rear for loading and unloading.
- the power supply module 3 is composed of three sub-power supply modules and adopts a dual-use and one-standby mode for power supply.
- the 100G switching module 4 is placed at the bottom of the chassis 1 for data interaction between the FPGA node boards 8 .
- the power supply module 3 needs to supply power to the control board 6 and the FPGA node board 8. They are both located on the upper part of the chassis 1. Placing it in the middle will be closer to the board, which can shorten the length of the power supply copper bar 14.
- the exchange module 4 is mainly connected with the network interface of the FPGA node board 8, and is used for data interaction between the FPGA node boards 8.
- each drawer structure 2 constitutes a small system with independent heat dissipation and management methods.
- the chassis 1 only needs to provide the power supply module 3 for the drawer structure 2, which is convenient for maintenance. , the replacement is simple, the assembly and disassembly are easy, the power supply module 3 is connected to the power taking clip of the control board 6 of the drawer structure 2 through the power supply copper bar 14, and the specific form is shown in Figure 4.
- the FPGA node board 8 in the drawer structure 2 also has a standard PCIe x16 interface. This interface is used for control management and data interaction of the board. The FPGA node board 8 can directly use this interface. It is plugged into the common x86 server on the market and has strong compatibility.
- the drawer structure 2 integrates a control board 6 inside, and the FPGA node board 8 is inserted into the control board 6 through a preset interface 10 .
- the preset interface 10 also integrates an Ethernet channel (which can support Gigabit, 10 Gigabit and higher-speed networks), monitoring and management channels, configuration and debugging channels, etc. Card power supply, monitoring, configuration debugging and other functions.
- the control board 6 may also be additionally equipped with a PCIe switch chip and reserve multiple PCIe interfaces for PCIe data exchange between the FPGA node boards.
- the FPGA node board can be plugged into the preset interface and the PCIe interface at the same time, and the drawer structure 2 does not need manual wiring, and has high reliability.
- the drawer structure 2 only needs to provide a network interface to the outside, so as to realize the management and use of the resources of the FPGA node board 8.
- the control board 6 integrates an intelligent management system, which can monitor the temperature, power consumption and other information in the chassis 1 in real time, and automatically trigger the safety mechanism when the limit is exceeded.
- the intelligent management system can dynamically schedule 8 resources of the FPGA node board to provide users with a safe and reliable use environment. Structurally, the control board 6 can also play the role of supporting the FPGA node board 8 .
- the left side has the opening mesh board 11 on the chassis 1
- the right side has the front panel 7 of the FPGA node board 8, and the PCIe interface and The preset interface 10 can be supported, and the cover plate 9 is fixed above the board card, and the structure is firm.
- Each drawer structure 2 has its own independent cooling system fan 12, and the cooling air ducts are unobstructed.
- the control board 6 does not have a high-performance CPU, so there is no need to think too much about the heat dissipation of the control board 6.
- Two drawer structures 2 are placed in the chassis 1. When the two-stage fans 12 work together to dissipate heat, the heat dissipation performance is improved.
- the key point of the present invention is that the density of the FPGA node board 8 in the limited space in the chassis is greatly improved through two integrated drawer structures.
- the drawer structure 2 itself has a way of dissipating heat and supplying power to the FPGA node board 8, which can be separated from the
- the chassis 1 works alone (external power supply is required), and the control board 6 in the drawer structure 2 and the FPGA node board 8 are interconnected through a custom gold finger (preset interface 10), which removes a large number of power supply modules 3 inside the traditional chassis and management network wiring for easy installation and commissioning.
- the present invention can deploy 32 high-performance FPGA node boards 8 with full height and three-quarter length in a standard 5U server chassis.
- the chassis 1 can be adjusted to a single-drawer or multi-drawer structure 2 according to the actual requirements of the number of boards or the physical size of the FPGA boards. For example, if the lateral dimension of the FPGA board is shorter, the drawer structure 2 can also be shortened accordingly. Then, multiple (greater than or equal to three) drawer structures 2 can be placed in succession in a standard server chassis, thereby improving the physical space utilization of the server rack to a greater extent.
- the invention provides a drawer-type high-density FPGA cloud platform chassis, comprising: a switch module located at the bottom of the chassis, a power supply module located on the switch module, and a drawer structure located on the power supply module; the drawer structure is provided with a control board card and an FPGA node The board, the FPGA node board is plugged into the control board through a preset interface; the power transmission end of the power supply module is electrically connected to the switching module and the power input interface of the control board, and the network switching interface of the switching module is connected to the network of the FPGA node board.
- the interfaces are connected to exchange data between the FPGA node boards.
- the present invention will greatly improve the deployable density of the FPGA node boards in the FPGA cloud platform chassis.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Mounting Of Printed Circuit Boards And The Like (AREA)
- Cooling Or The Like Of Electrical Apparatus (AREA)
Abstract
L'invention concerne un boîtier de plateforme en nuage FPGA haute densité de type tiroir, comprenant : un module d'échange situé au fond du boîtier, un module d'alimentation électrique situé sur le module d'échange, et une structure de tiroir située sur le module d'alimentation électrique, une carte de carte de commande et des cartes de carte de nœud FPGA étant agencées dans la structure de tiroir, et les cartes de carte de nœud FPGA étant insérées dans la carte de carte de commande au moyen d'interfaces prédéfinies ; une extrémité de transmission de puissance du module d'alimentation électrique est électriquement connectée au module d'échange et une interface d'entrée de puissance de la carte de carte de commande ; et une interface d'échange de réseau du module d'échange est connectée à des interfaces réseau des cartes de carte de nœud FPGA et utilisée pour effectuer une interaction de données entre les cartes de carte de nœud FPGA. Au moyen de la présente invention, la densité de déploiement des cartes de carte de nœud FPGA dans le boîtier de plateforme de nuage FPGA est considérablement améliorée, ce qui permet de réduire le coût de câblage, la complexité de montage et de démontage et la difficulté de maintenance dans le boîtier ; un environnement de développement complet et pratique est fourni pour un utilisateur à l'aide d'un système de commande et de gestion qui est indépendamment recherché et développé ; et le boîtier et l'état des cartes de carte sont surveillés en temps réel, et le nombre de fils connectés manuellement est réduit au moyen des interfaces prédéfinies, ce qui permet d'améliorer la fiabilité du boîtier de plateforme en nuage FPGA.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011478372.7 | 2020-12-15 | ||
CN202011478372.7A CN112512262A (zh) | 2020-12-15 | 2020-12-15 | 一种抽屉式高密度fpga云平台机箱 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022126937A1 true WO2022126937A1 (fr) | 2022-06-23 |
Family
ID=74973767
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/085808 WO2022126937A1 (fr) | 2020-12-15 | 2021-04-07 | Boîtier de plateforme en nuage de fpga haute densité de type tiroir |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112512262A (fr) |
WO (1) | WO2022126937A1 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115442207A (zh) * | 2022-07-29 | 2022-12-06 | 中电科思仪科技股份有限公司 | 一种基于BMC+SoC+网络交换模块的硬件运维管理系统 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112512262A (zh) * | 2020-12-15 | 2021-03-16 | 中国科学院计算技术研究所 | 一种抽屉式高密度fpga云平台机箱 |
CN113434348A (zh) * | 2021-06-25 | 2021-09-24 | 成都中微达信科技有限公司 | 一种芯片互联测试平台 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN203520283U (zh) * | 2013-10-18 | 2014-04-02 | 纬创资通股份有限公司 | 抽取式框架及服务器 |
CN204180403U (zh) * | 2014-09-23 | 2015-02-25 | 安徽四创电子股份有限公司 | Ip多媒体处理机箱及装置 |
CN206042618U (zh) * | 2016-08-22 | 2017-03-22 | 苏州博思得电气有限公司 | 一种电源模块安装结构、电源机箱和医疗设备 |
US20190339747A1 (en) * | 2018-05-02 | 2019-11-07 | Wiwynn Corporation | Power distribution board, modular chassis system and operating method thereof |
CN112512262A (zh) * | 2020-12-15 | 2021-03-16 | 中国科学院计算技术研究所 | 一种抽屉式高密度fpga云平台机箱 |
-
2020
- 2020-12-15 CN CN202011478372.7A patent/CN112512262A/zh active Pending
-
2021
- 2021-04-07 WO PCT/CN2021/085808 patent/WO2022126937A1/fr active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN203520283U (zh) * | 2013-10-18 | 2014-04-02 | 纬创资通股份有限公司 | 抽取式框架及服务器 |
CN204180403U (zh) * | 2014-09-23 | 2015-02-25 | 安徽四创电子股份有限公司 | Ip多媒体处理机箱及装置 |
CN206042618U (zh) * | 2016-08-22 | 2017-03-22 | 苏州博思得电气有限公司 | 一种电源模块安装结构、电源机箱和医疗设备 |
US20190339747A1 (en) * | 2018-05-02 | 2019-11-07 | Wiwynn Corporation | Power distribution board, modular chassis system and operating method thereof |
CN112512262A (zh) * | 2020-12-15 | 2021-03-16 | 中国科学院计算技术研究所 | 一种抽屉式高密度fpga云平台机箱 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115442207A (zh) * | 2022-07-29 | 2022-12-06 | 中电科思仪科技股份有限公司 | 一种基于BMC+SoC+网络交换模块的硬件运维管理系统 |
CN115442207B (zh) * | 2022-07-29 | 2024-01-26 | 中电科思仪科技股份有限公司 | 一种基于BMC+SoC+网络交换模块的硬件运维管理系统 |
Also Published As
Publication number | Publication date |
---|---|
CN112512262A (zh) | 2021-03-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022126937A1 (fr) | Boîtier de plateforme en nuage de fpga haute densité de type tiroir | |
CN105717991B (zh) | 电子装置 | |
US20110310550A1 (en) | Rack server | |
US20180027687A1 (en) | Technologies for sled architecture | |
CN107656588B (zh) | 一种优化散热的服务器系统及安装方法 | |
CN203745989U (zh) | 一体化高密度服务器机箱 | |
TWI724739B (zh) | 電腦電源供應組件及其製造方法 | |
CN102625608A (zh) | 一种大规模多节点服务器机柜的设计方法 | |
TWM458020U (zh) | 電池備援系統及其電池備用模組 | |
CN211656673U (zh) | 一种工控机 | |
CN102932156A (zh) | 一种微服务器及微服务器集群系统 | |
CN202443354U (zh) | 多节点无线缆模块化计算机 | |
CN214896436U (zh) | 一种模块化多计算节点gpu服务器结构 | |
WO2019100701A1 (fr) | Architecture d'extension de contrôleur de commutateur sas et son procédé de conception | |
CN103605413A (zh) | 机架式服务器系统的机柜、机架式服务器系统及其管理方法 | |
CN103970713A (zh) | 一种高扩展性1u服务器 | |
CN210075453U (zh) | 一种用于有线数字电视监测前端高集成度硬件平台 | |
CN214545185U (zh) | 一种抽屉式高密度fpga云平台机箱 | |
CN111427833A (zh) | 服务器集群 | |
CN212009564U (zh) | 服务器集群 | |
CN203241863U (zh) | 一种高可用多单元携行服务器系统 | |
CN210605597U (zh) | 电子计算设备 | |
CN204374858U (zh) | 一种能够高效协同办公的云存储一体机 | |
CN210428236U (zh) | 一种高密度八路服务器 | |
CN113220080A (zh) | 一种模块化多计算节点gpu服务器结构 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21904876 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21904876 Country of ref document: EP Kind code of ref document: A1 |