WO2013088018A1

WO2013088018A1 - Method and computer program for configuring devices in a computer infrastructure having infiniband architecture

Info

Publication number: WO2013088018A1
Application number: PCT/FR2012/052730
Authority: WO
Inventors: Jean-Vincent Ficet; Sébastien DUGUE; Yann Kalemkarian
Original assignee: Bull Sas
Priority date: 2011-12-13
Filing date: 2012-11-27
Publication date: 2013-06-20
Also published as: FR2984052B1; FR2984052A1

Abstract

The invention relates in particular to the configuration of devices in a cluster having InfiniBand architecture after said devices have been started up. According to the invention, the cluster includes a plurality of devices, at least one administration device of which is configured such as to access data for configuring the cluster. The administration device can be notified when devices are started up and can explore the cluster in order to set up at least one path of direct routes for accessing a device. After receiving (430) a notification that a device has been started up, the notification including a path of direct routes for accessing said device in the cluster, at least one characteristic of said device is obtained (435) from the data for configuring the cluster along the path for access to the device. The latter is then configured (475) according to said at least one obtained characteristic.

Description

Method and computer program for configuring equipment in a computer infrastructure before an Infiniband type architecture

The present invention relates to the management of computer infrastructures such as clusters and data centers, and more particularly to a method and a computer program for configuring equipment in a computer infrastructure having a data architecture. Infiniband type (Infiniband is a brand).

High Performance Computing, also known as High Performance Computing (HPC), is developing for both academic research and industry, particularly in technical fields such as automotive, aeronautics, energy , climatology and life sciences. In particular, modeling and simulation make it possible to reduce development costs and speed up the launch of innovative, more reliable and less energy-consuming products. For researchers, high performance computing has become an indispensable means of investigation.

These calculations are generally implemented on data processing systems called clusters (sometimes translated "clusters of servers"). A cluster typically includes a set of interconnected nodes. Some nodes are used to perform compute tasks (compute nodes), others to store data (storage nodes), and one or more others manage the cluster (administration nodes). Each node is for example a server implementing an operating system such as Linux (Linux is a brand). The connection between the nodes is, for example, carried out using Ethernet communication links and interconnection networks (for example Infiniband) (Ethernet and Infiniband are trademarks).

Figure 1 schematically illustrates an example of a topology 100 of a cluster, type fat-tree. The latter includes a set of nodes The nodes belonging to the set 1 10 are here computation nodes while the nodes of the set 1 15 are service nodes (storage nodes and administration nodes). The calculation nodes can be grouped into subsets 120 called computing islands, the set 15 being called service island.

The nodes are connected to each other by switches (called switches in English terminology), for example hierarchically. In the example illustrated in FIG. 1, the nodes are connected to first level switches 125 which are themselves connected to second level switches 130 which are in turn connected to third level switches 135. The switches route messages from their sources to their destinations according to routing tables programmed during cluster initialization. For these purposes, an output port is generally associated in each switch with one or more destinations.

A cluster can be decomposed into elementary infrastructures also called subnets in English terminology. Each subnet is managed by managers and agents including a master subnet manager such as the one known as OpenSM.

A master subnet manager can periodically scan the corresponding basic infrastructure to determine whether devices, typically nodes, have been added or removed. Such modifications are notably related to equipment startup and breakdowns. In addition, asynchronous notification messages, called traps in English terminology, can be automatically sent to the main manager of subnet to inform him of such a change without it having to scan the basic infrastructure. An asynchronous notification message is typically issued by a switch to which the added or removed equipment is connected. A corresponding message is then usually sent to a cluster administrator who can then, if necessary, install a new node after obtaining its logical address (usually available in a cluster configuration database).

As illustrated in FIG. 2, each node typically comprises one or more microprocessors, local memories as well as a communication interface. More specifically, the node 200 here comprises a communication bus 202 to which are connected:

central processing units or microprocessors 204 (CPU, acronym for Central Processing Unit in English terminology);

components of RAM 206 (Random Access Memory in English) with registers adapted to record variables and parameters created and modified during the execution of programs (as illustrated, each memory component alive can be associated with a microprocessor); and,

communication interfaces 208 adapted to transmit and receive data.

In this case, the node 200 also has internal storage means 210, such as hard disks, which can notably comprise the executable code of programs.

The communication bus allows the communication and interoperability between the different elements included in the node 200 or connected to it. The microprocessors 204 control and direct the execution of the instructions or portions of software code or programs. When powering up, the program or programs that are stored in a non-volatile memory, for example a hard disk, are transferred into the RAM 206.

When installing a cluster, each node is typically tested before being configured. For these purposes, each time a new node is started, a pre-loaded test routine in the image node is installed and executed, and then, when the node has been tested and considered stable, an image related to the role of the node in the cluster is obtained, typically via the communication network to which the node is connected, and is then installed. Obtaining and installing an image, usually called deployment, is often done automatically. After installation of this image, the node is operational within the cluster. However, if the deployment can be performed automatically, it is implemented by an operator or administrator who identifies the nodes to be deployed and obtains their characteristics necessary for the deployment, including their logical address in the cluster.

The size of clusters is increasing day by day. Therefore, their installation and the associated software deployment, implemented, in particular, to install the necessary operating systems and software applications, require a longer and longer time.

There is therefore a need to improve the configuration of equipment in an IT infrastructure such as a cluster, in particular an IT infrastructure having an Infiniband type architecture.

The subject of the invention is thus a method for configuring an equipment in a computer infrastructure having an Infiniband type architecture, following the startup of said equipment, said computing infrastructure comprising a plurality of equipment including at least one configured administration equipment. to access configuration data of said computing infrastructure, said at least one administration equipment being able to be notified of the startup of said equipment and being able to explore said IT infrastructure to establish at least one direct route path to access said equipment, which method comprises the following steps,

receiving a start-up notification of said equipment, said notification comprising a direct route path for accessing said equipment in said computer infrastructure;

obtaining at least one characteristic of said equipment from said configuration data of said computer infrastructure according to said access path to said equipment;

configuration of said equipment according to said at least one characteristic. The method of the invention thus facilitates the startup, configuration and / or deployment of equipment in a computer infrastructure having an Infiniband type architecture, typically nodes in a cluster. For these purposes, the method according to the invention combines characteristics of Infiniband type architectures for obtaining route information and the availability of configuration data of IT infrastructures.

Said at least one feature of said equipment preferably comprises a logical address of said equipment in said computing infrastructure. Such a feature allows the execution of commands (or scripts) aimed, for example, the installation of an image to the identified logical address. Advantageously, said at least one characteristic of said equipment further comprises information of the type, role, function and / or state of said equipment in said computer infrastructure making it possible to choose specific commands to be executed, determined according to the role, the function and / or or the state of said equipment.

According to a particular embodiment, the method further comprises a step of obtaining an image to be deployed, said configuration step comprising a step of deploying said image obtained in said equipment. Advantageously, the method further comprises a step of identifying an image to be deployed, said identification using said information of type, role, function and / or state of said equipment.

Still according to a particular embodiment, the method further comprises a step of updating said configuration data of said IT infrastructure following the configuration of said equipment. The method according to the invention thus makes it possible to maintain coherence between configuration data and the configuration of said IT infrastructure.

Advantageously, the method further comprises a step of testing said equipment, said testing step being performed before said step of configuring said equipment. Said equipment is thus configured only under certain conditions, for example if it can actually be used in said Informatique infrastructure. Advantageously, the method further comprises a step of updating said configuration data of said IT infrastructure following the testing of said equipment in order to maintain consistency between configuration data and the configuration of said IT infrastructure.

The invention also relates to a computer program comprising instructions adapted to the implementation of each of the steps of the method described above when said program is executed on a computer and a device comprising means adapted to the implementation of each of the steps of the method described above. The benefits provided by this computer program and device are similar to those discussed above.

Other advantages, aims and features of the present invention will emerge from the detailed description which follows, given by way of non-limiting example, with reference to the accompanying drawings in which:

FIG. 1 illustrates an exemplary topology of a cluster;

FIG. 2 illustrates an exemplary architecture of a node of a cluster;

FIG. 3 schematically illustrates an example of an elementary infrastructure in which topological information of a node added to this elementary infrastructure can be automatically obtained by an administration node; and,

FIG. 4 schematically illustrates certain steps implemented in accordance with the invention for configuring a piece of equipment in an IT infrastructure following the start-up of the latter.

In general, the invention aims at obtaining topological information of an equipment such as a node belonging to a computer infrastructure having an Infiniband type architecture, typically a cluster, when it is launched, in order to obtain characteristics of this equipment. Such characteristics can in particular be obtained from a database describing the IT infrastructure. These features are then used to automatically configure the equipment. Such topological information typically is the location or location of equipment within an IT infrastructure.

It is first of all reminded here that a particularity of Infiniband architectures is the possibility of indicating, in a management message of a basic infrastructure (subnef), called SMP (acronym for Subnet Management).

Packet in English terminology), the number of the port by which it is sent by each switch crossed during its transmission.

Such indications thus make it possible, in a computer infrastructure having an Infiniband type architecture, to determine the relative position of an equipment with respect to a second one, that is to say a path between two pieces of equipment.

In the Infiniband context, indications defining a path between a given piece of equipment and a reference piece of equipment are called

Direct Route path. The reference equipment is typically an administration node, for example the node in which the main manager of the subnet concerned is executed.

Depending on the topology of the IT infrastructure, there may be several direct route paths between a given device and a reference device.

In other words, a direct route path of a given piece of equipment describes the successive links that a packet of data sent by the reference equipment must use to reach that given piece of equipment. It is expressed as a suite of switch output port references that the packet must use to reach its destination.

Direct route paths between reference equipment and other equipment are usually established upon discovery of the basic infrastructure by the corresponding subnet master manager. Such a discovery typically uses conventional tree exploration algorithms.

The establishment of direct route paths can also be performed following receipt of an asynchronous notification message (trap) in particular, to determine the direct route (s) of a newly added equipment to the basic infrastructure.

As indicated above, an asynchronous notification message is generally issued, in a computer infrastructure having an Infiniband type architecture, by a switch to which a device is directly connected when it is started. Upon receipt of such a message, the main subnet manager can then rediscover the corresponding basic infrastructure to identify the equipment and establish their direct route paths.

It is thus possible to automatically obtain topological information of equipment added to a basic infrastructure.

The main manager of subnet is here configured to alert a specific agent, for example an equipment configuration agent, the start of a device and transmit him or the direct route paths.

By way of illustration, FIG. 3 schematically illustrates an example of elementary infrastructure 300 in which topological information of a node added to this elementary infrastructure can be automatically obtained by an administration node implementing a main subnet manager. (master subnet manager).

As illustrated, the basic infrastructure 300 here comprises a management node 305, a set of nodes referenced 310-1 to 310-15 and the switches 315-1 to 315-6. Each switch here has six bidirectional ports numbered 1-6.

Switches 315-1 through 315-4 are each connected to four different nodes. Thus, for example, the switch 315-1 is connected to the nodes 305 and 310-1 to 310-3. Switches 315-1 through 315-4 are further connected to switches 315-5 and 315-6. Each node is here connected to each of the other nodes via at most three switches.

As an illustration, the node 305 is connected to the node 310-14 via the switches 315-1, 315-5 and 315-4. More specifically, the output port of the node 305 is connected to the input port 1 of the switch 315-1 whose port output 5 is connected to the input port 1 of the switch 315-5 whose output port 4 is itself connected to the input port 5 of the switch 315-4 whose output port 3 is connected to the port of entry of node 310-14. Similarly, since the links here are bidirectional, the output port of the node 30-14 is connected to the input port 3 of the switch 315-4 whose output port 5 is connected to the input port 4 of the switch 315. 5 whose output port 1 is itself connected to the input port 5 of the switch 315-1 whose output port 1 is connected to the input port of the node 305.

When the primary manager of the basic infrastructure 300 explores the latter, he establishes the direct route paths between the administration node 305 and each of the other nodes and therefore, in particular, at least one path of direct routes between the node 305. and the node 310-14.

By convention, it is accepted that a direct route path always starts with the identifier 0. In addition, for the sake of clarity, it is assumed here that each node comprises only one input and output port having the reference 1.

As described above, the establishment of a path of direct routes between two devices and therefore between the node 305 and the node 310-14, is performed according to the references of the output ports used. Thus, a direct route path between the node 305 and the node 310-14 begins with the expression {0, 1} in which the reference 0 designates the beginning of a direct route path and the reference 1 designates the reference of the first output port (the output port of the reference node, i.e., the administration node), as shown. It is observed here that information for establishing links between switch ports and between switch and node ports is generally stored in routing tables and / or in a configuration database of the IT infrastructure. .

The direct route path considered here between node 305 and node 310-14 (shown in bold lines in FIG. 3) using the output port 5 of switch 315-1, reference 5 is added to the expression {0 , 1} representing the direct route path that becomes {0, 1, 5} as shown. Similarly, the direct route path considered here using the output port 4 of the switch 315-5, the reference 4 is added to the expression {0, 1, 5} which becomes {0, 1, 5, 4} as illustrated. In a similar way again, the direct route path considered here then using the output port 3 of the switch 315-4, the reference 3 is added to the expression {0, 1, 5, 4} which becomes {0, 1, 5, 4, 3} as illustrated. This last expression thus represents an example of a direct route path of the node 310-14 with respect to the reference node, here the administration node 305.

It is assumed here that node 310-14 is started at a time f. Following this startup and as previously described, an asynchronous notification message is issued by the switch 315-4 (to which the node 310-14 is connected). Upon receipt of this message, the main manager of the basic infrastructure 300 executed by the node 305 can re-explore the basic infrastructure 300 and thus determine that the node whose path of direct routes is {0, 1, 5, 4, 3} has just started.

According to the invention, this information is transmitted to an equipment configuration agent in order to configure the target node. For these purposes, the configuration agent uses configuration information from the IT infrastructure to identify the node that has a direct route path established. Typically, such configuration information is gathered in a database defining all the characteristics of the IT infrastructure and, in particular, its topology as well as the characteristics of each node. These characteristics are, for example, the type, role and / or function of each node, for example if it is a compute node or an input / output node, their name and their state as well as their logical address in the IT infrastructure, for example their IP address (abbreviation of Internet Protocol in English terminology).

From this information, the configuration agent can automatically launch, if necessary, image installations and / or tests allowing the configuration of the nodes.

It is observed here that if there are several direct route paths for a given node, the choice of one path or another is unimportant. in that they make it possible to identify the same node, by its position, in the infrastructure.

FIG. 4 schematically illustrates certain steps implemented in accordance with the invention for configuring a piece of equipment in an IT infrastructure following the start-up of the latter. As an illustration, Figure 4 is more specifically the configuration of a node in a cluster.

After starting a node (step 400), an asynchronous notification message {trap) is typically transmitted (step 405) by a switch to which this node is connected. This asynchronous notification message is received (step 410) by the main manager of the corresponding elementary infrastructure which can then re-explore the latter (step 415) in order, in particular, to establish direct route paths of the newly started node. A notification is then sent (step 420) by the manager to alert a configuration agent of the start of a node. Such notification includes a direct route route.

For the sake of clarity, the steps implemented here by the configuration agent are referenced 425.

Following the reception (step 430) of a notification sent by a main manager of a basic infrastructure alerting the startup configuration agent of a node, the configuration agent queries (step 435) a database of configuring the computer infrastructure, referenced herein 440, to obtain characteristics of the node of which a direct route path has been received with the notification. Typically, these features include the node name, type, role, function, state, and logical address, such as its IP address, in the IT infrastructure.

It is observed here that if such data are generally stored in a database, they may also be available in other forms.

According to a particular embodiment, this request may comprise a local identifier of the node, for example an identifier known under the name of GUID (acronym obtained from Globally Unique IDentifier in English terminology), obtained in a standard way by the main manager of the basic infrastructure, in order to update the interrogated database.

A test is then performed to determine if the node for which a start notification has been received is a new node (step 445), that is to say a blank node started for the first time in this context of the IT infrastructure. . Such a test may notably consist in comparing the state of this node with a predetermined state, for example the "NEW" state.

If so, the configuration agent launches a deployment tool to install an image for testing the node (step 450). Such an image is preferably preinstalled in the node. After the image has been installed, the corresponding test is started (step 455). Such a test consists for example in filling the working memory of the node to a very high level of filling and performing matrix calculations. It can also be a test of its network equipment (for example a network card) and / or its mass memory (for example a hard disk).

A test is then performed to determine if the tests of the node have been passed successfully (step 460). If not, an error is reported to the administrator, for example in a log file.

If, on the contrary, the tests of the node have been passed successfully, the configuration database of the IT infrastructure is preferably updated (step 465) to indicate that the node has been successfully tested. . For these purposes, the state of the node in the database can be changed, the new state being, for example, "TESTED". The node is then restarted and the algorithm loops back on itself to process a new node start (as shown, the algorithm returns to step 430).

If the node for which a start notification was received is not a new node (step 445), another test is performed to determine if the node has been tested and the tests passed successfully (step 470) , that is, if it is the first start of the node after installation during which it has been tested and needs to be deployed. Such test may include comparing the state of the node with the predetermined state "TESTED".

If the node has been tested and the tests passed successfully, the configuration agent launches a standard deployment tool to install an application image (step 475). For these purposes, when several images are likely to be deployed, the image to be deployed is advantageously identified according to characteristics of the node previously obtained (as a function of the path of direct routes), in particular according to its type, its role and / or its function in the IT infrastructure. The image or images that can be deployed are here stored in storage means accessible in the IT infrastructure. To allow the deployment of the identified image, the logical address of the node obtained among the characteristics of the node is used here. The deployment of the image is then performed.

The IT infrastructure configuration database is then preferably updated (step 480) to indicate that the node has been installed. For these purposes, the state of the node in the database can be changed, the new state being, for example, "DEPLOYED".

The algorithm then loops back on itself to process a new node start (as shown, the algorithm returns to step 430).

Similarly, if the node for which a start notification has been received is not a new node (step 445) and the node is not in a state that it is its first start after installation (step 470), this means that the node has already been deployed and does not require any particular processing. The algorithm then loops back on itself to process a new node start (as shown, the algorithm returns to step 430).

An exemplary device adapted to implement the invention, in particular the steps described with reference to FIG. 4, more particularly the steps of the configuration agent (referenced 425), is a cluster node, for example the node described in FIG. reference to Figure 2. Naturally, to meet specific needs, a person skilled in the field of the invention may apply modifications in the foregoing description.

Claims

A method of configuring a device in a computing infrastructure having an Infiniband architecture, following the startup of said equipment, said computing infrastructure comprising a plurality of devices including at least one administration equipment configured to access configuration data. said computer infrastructure, said at least one administration equipment being able to be notified of the start of said equipment and being able to explore said computing infrastructure to establish at least one direct route path to access said equipment, said method being characterized in that it comprises the following steps,

receiving (430) a start-up notification of said equipment, said notification comprising a direct route path for accessing said equipment in said computing infrastructure;

obtaining (435) at least one characteristic of said equipment from said configuration data of said IT infrastructure according to said access path to said equipment;

- configuration (475) of said equipment according to said at least one characteristic.

The method of claim 1 wherein said at least one feature of said equipment comprises a logical address of said equipment in said computing infrastructure.

3. The method of claim 2 wherein said at least one feature of said equipment further comprises information of type, role, function and / or state of said equipment in said computer infrastructure.

4. The method of claim 2 or claim 3 further comprising a step of obtaining (475) an image to be deployed, said configuration step comprising a step of deploying said image obtained in said equipment.

5. The method of claim 4, dependent on claim 3, further comprising a step of identifying an image to be deployed, said identification using said information type, role, function and / or state of said equipment.

6. Method according to any one of claims 1 to 5 further comprising a step of updating (480) said configuration data of said IT infrastructure following the configuration of said equipment.

7. Method according to any one of the preceding claims further comprising a step of testing said equipment, said testing step being performed before said step of configuring said equipment.

8. The method of claim 7 further comprising a step of updating (465) said configuration data of said computer infrastructure following the testing of said equipment.

9. Computer program comprising instructions adapted to the implementation of each of the steps of the method according to any one of the preceding claims when said program is executed on a computer.

10. Device comprising means adapted to the implementation of each of the steps of the method according to any one of claims 1 to 8.