KR102419015B1 - Interrupt distribution method for numa-based devices, recording medium storing program for executing the same, and recording medium storing program for executing the same - Google Patents

Abstract

The present invention relates to a method for efficiently distributing (allocating) interrupt requests (IRQs) of non-uniform memory access (NUMA)-based equipment and, specifically, to an interrupt distribution method for NUMA-based equipment, a recording medium in which a program for performing the same is stored, and a computer program stored in a medium to perform the same, which can improve system performance by modifying a source code in a Linux kernel and efficiently allocate IRQs in a vector space of each core in a central processing unit (CPU).

Description

Interrupt distribution method of NUMA-based equipment, a recording medium storing a program for implementing the same, and a computer program stored in the medium for implementing the same PROGRAM FOR EXECUTING THE SAME}

The present invention relates to a method for efficiently distributing (assigning) an interrupt (Interrupt request, IRQ) of a NUMA (Non-Uniform Memory Access)-based device, and in detail, a source code ( source code) to efficiently distribute interrupts in the vector space of each core within the central processing unit (CPU), thereby improving system performance. Interrupt distribution method of NUMA-based equipment , relates to a recording medium storing a program for implementing the same, and a computer program stored in the medium for implementing the same.

Recently, a multi-core processor for high performance computing has been widely used. In a multi-core system having such a multi-core processor, how to handle interrupt requests generated between peripheral devices and cores to improve performance is emerging as one of the important tasks.

In general, the method of allocating interrupts in the Linux kernel is performed in a space called a vector of each core of the central processing unit. When an interrupt occurs during system operation according to the assigned IRQ number, the vector space of each core is used.

For example, the following two methods are known for the IRQ allocation method for the interrupt request in the conventional Ethernet driver. The first method is a method in which each port creates as many IRQs as the number of cores during the system booting process, and sequentially fills them from 'Core (0)' to 'Core (K)' in the core vector. The second method is to allocate the desired IRQ to the desired core by directly modifying the 'smp_affinity' value by the system operator.

1 is a flowchart illustrating an example of an allocation method when an interrupt is requested in a conventional Ethernet driver. Here, N, M, and K are natural numbers.

Referring to FIG. 1, the method of allocating an IRQ when an interrupt is requested in an Ethernet driver according to the prior art sequentially increases the IRQs (0~N) of 'Port (M)' in the Ethernet port 'Port (0)' to the core. Allocate from 'Core (0) Vector' to 'Core (K) Vector' (S1~S9).

In this way, the method of allocating IRQs to the vector space of each core in the Linux kernel simply takes the method of sequentially allocating IRQs to the core vectors regardless of efficiency.

2 is an exemplary diagram of an interrupt assignment according to the prior art.

As in FIG. 2 , for example, all IRQ lists of the Ethernet port 'Port (0)' are allocated to 'Core (0)'. In addition, all IRQ lists of the Ethernet port 'Port (1)' are allocated to 'Core (0)' and 'Core (1)', and the IRQ lists of the Ethernet port 'Port (2)' are also sequentially allocated.

However, in the interrupt allocation method according to the prior art, when redistribution is attempted while the IRQ lists of each port are allocated, a failure occurs.

FIG. 3 is an exemplary diagram of a core vector when an interrupt is allocated as shown in FIG. 2 .

As shown in Figure 3, in the interrupt allocation method according to the prior art, when the user tries to redistribute the IRQ of 'Port (0)' to improve performance, there is no empty vector in 'Core (1)', An error occurs in the process of moving the IRQ to the vector of 'Core (1)'. After all, only cores with room for a smooth transition to IRQs are possible.

As such, in the interrupt allocation method according to the prior art, an error may occur during IRQ redistribution. This phenomenon can be alleviated little by little when the number of cores is small, but there is a problem that becomes more serious as the number of cores and ports increases. This causes a bottleneck in the system and increases the interrupt latency, so resource utilization in the system is not effective. there was.

KR 10-2013-0049110 A, 2013. 05. 13., "Method and device for allocating interrupts" KR 10-1717494 B1, 2017. 03. 13., "Interrupt handling apparatus and method"

The problems that occur in the interrupt allocation method according to the prior art described above occur when an equipment using a multi-port Ethernet card is used for a multi-core CPU, but in particular, a multi-core CPU, NUMA structure, and high-performance Ethernet are used in server-class equipment. It occurs easily when using a chipset (Ethernet chipset (ixgbe, i40e, Mellanox, etc.) at the same time).

Therefore, the present invention prevents the problems pointed out in the interrupt allocation method according to the prior art by distributing IRQs for each core during initial IRQ allocation, rather than simply sequentially allocating IRQs to the core vectors as in the interrupt allocation method according to the prior art. Interrupt distribution method of NUMA-based equipment that can improve performance by specifying receive/transmit queues (Rx/Tx Queue) in the Ethernet driver afterwards, a recording medium storing a program for implementing it, and a computer program stored in the medium to implement it Its purpose is to provide

The present invention for achieving the above object is a plurality of core vectors each configured to a plurality of central processing units (Central Processing Unit) a plurality of IRQ (Interrupt request) for each port in a NUMA (Non-Uniform Memory Access) environment equipment In the interrupt distribution method of NUMA-based equipment assigned to the process of allocating; (b) the process of increasing the number of the IRQ of the corresponding port to '1'; (c) the process of increasing the number of the core vector of the central processing unit by '1'; and (d) allocating the IRQ of the next number increased by '1' in the step (b) to the core vector of the next number increased by '1' in the step (c). Provides an interrupt distribution method for the underlying equipment.

In addition, (e) the number of the core vector increased by '1' in the process (c) by comparing the number of the core vector increased by '1' in the process (c) with the number of the last core vector of the central processing unit. and in the process (b) until the number of the current IRQ increased by '1' in the process (b) exceeds the number of the last IRQ if it does not exceed the number of the final th core vector of the central processing unit. When the process (d) is repeatedly performed to distribute the IRQs included in the corresponding port, and the core vector increased by '1' in the process (c) exceeds the final th core vector of the corresponding central processing unit After returning to the first core vector of the corresponding central processing unit, in step (b), in step (d), until the number of the current IRQ increased by '1' in step (b) exceeds the number of the last IRQ. The method may further include a process of performing distribution of IRQs included in the corresponding port by repeating the process until the end of the process.

In addition, the present invention for achieving the above object is a NUMA (Non-Uniform Memory Access) environment equipment in a plurality of IRQ (Interrupt request) for each port in a plurality of central processing units (Central Processing Unit) each configured In the interrupt distribution method of NUMA-based equipment that is assigned to a core vector, (a) among the ports, according to the NUMA environment, the first IRQ among the IRQ list configured in the corresponding port connected to the central processing unit is assigned to the first core of the central processing unit. the process of assigning to a vector; (b) the process of increasing the number of the IRQ of the corresponding port to '1'; (c) comparing the number of the current IRQ increased by '1' in step (b) with the number of the last IRQ of the corresponding port; (d) the process of increasing the number of the core vector by '1' when, as a result of the comparison of (c), the number of IRQs increased by '1' in step (b) does not exceed the number of the last IRQ; (e) comparing the number of the core vector increased by '1' in the process (c) and the number of the final th core vector; and (f) as a result of comparison in step (e), when the number of core vectors increased by '1' in step (d) does not exceed the number of the final core vector, '1' in step (b) If the increased current IRQ is assigned to the current core vector increased by '1' in the process (d), and the number of the core vector increased by '1' in the process (d) exceeds the number of the final th core vector, Returning to the first core vector of the corresponding central processing unit and allocating the current IRQ increased by '1' in the process (b) to the first core vector of the corresponding central processing unit. Provides an interrupt distribution method.

In addition, (g) after step (f), return to step (b) and repeat the steps from step (b) to step (f), but as a result of comparison in (c), (b) ) may further include a process of repeatedly performing until the number of the current IRQ increased by '1' in the process exceeds the number of the last IRQ.

In addition, (h) as a result of the comparison of (c) in step (d), when the number of IRQs increased by '1' in step (b) exceeds the number of the last IRQ, the port number is set to '1'. It may be characterized in that it further includes the process of performing the same from (a) to (f) of the IRQ of the current port increased by '1' by 'increasing'.

In addition, the present invention for achieving the above object provides a computer-readable recording medium storing a program for implementing the interrupt distribution method of the above-described NUMA-based equipment.

In addition, the present invention for achieving the above object provides a computer program stored in a computer-readable recording medium for implementing the interrupt distribution method of the above-described NUMA-based equipment.

As described above, according to the interrupt distribution method of a NUMA-based device according to the present invention, the IRQ is not simply allocated sequentially to the core vector, but is distributed to each core during initial IRQ allocation, as indicated in the interrupt allocation method according to the prior art. This can prevent problems and improve performance through efficient memory usage.

In addition, according to the interrupt distribution method of the NUMA-based equipment according to the present invention, the space utilization of the core vector of the central processing unit is improved by allocating and distributing the IRQs in a manner that sequentially allocates the IRQs for each port one by one for each core during IRQ initialization. Through this, efficient IRQ distribution and smooth interrupt handling and management are possible, other business capabilities can be improved by minimizing the resources required for traffic management, and flexible interrupt handling is possible.

1 is a flowchart illustrating an allocation method when an interrupt is requested in a conventional Ethernet driver.
2 is an exemplary diagram of an interrupt assignment according to the prior art;
FIG. 3 is an exemplary diagram of a core vector upon assignment of an interrupt shown in FIG. 2 .
4 is an exemplary environment diagram of a NUMA-based device to which an interrupt distribution method according to an embodiment of the present invention is applied.
5 is a flowchart of an interrupt distribution method according to an embodiment of the present invention;
6 is an exemplary diagram of an interrupt allocation process according to an embodiment of the present invention;

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but does not exclude that they are implemented in various different forms, only the embodiments of the present invention make the disclosure of the present invention complete, and those of ordinary skill in the art It is provided to fully inform the person of the scope of the invention. In the drawings, like reference numerals refer to like elements.

4 is an exemplary environment diagram of a NUMA-based device to which an interrupt distribution method according to an embodiment of the present invention is applied. 5 is a flowchart of an interrupt distribution method according to an embodiment of the present invention.

As shown in Fig. 4, it is a NUMA-based device to which the interrupt distribution method according to the embodiment of the present invention is applied. For example, it can be applied to a device using a multi-port Ethernet card for a multi-core CPU, and a multi-core CPU, NUMA to a server-class device. Structure and high-performance Ethernet chipset (Ethernet chipset (ixgbe, i40e, Mellanox, etc.)) can be applied to equipment using simultaneously.

For example, the interrupt distribution method according to the embodiment of the present invention is based on the environment of 64 Core CPU (+Hyper Threading) and i40e multiport in a NUMA structure, and the OS is based on Fedora. However, depending on the version, the file/function name containing the code may be different.

5 is a flowchart of an interrupt distribution method of a NUMA-based device according to an embodiment of the present invention, and FIG. 6 is an exemplary diagram of an interrupt allocation process. Here, N, M, and K are natural numbers.

5 and 6 , in the interrupt distribution method according to the embodiment of the present invention, each IRQ for each port is sequentially allocated to the core vector one by one in sequence. In this case, the allocation of IRQs for each port is sequentially allocated to all core vectors, but sequentially allocated to the core vectors of the corresponding central processing unit according to the NUMA-based environment.

Hereinafter, an interrupt distribution method according to an embodiment of the present invention will be described in detail.

5 and 6, each IRQ list (IRQ (0 to N)) of the Ethernet port (Port (0 to M)) corresponding to 'CPU (0)', which is the central processing unit, is divided by core (Core ( 0~K) is assigned as Vector).

Specifically, first, 'IRQ (0)', which is the first in the IRQ list of 'Port (0)', which is the first port among the Ethernet ports corresponding to 'CPU (0)', which is the first central processing unit, is processed by the corresponding central processing unit. It is assigned to the 'Core (0) Vector', which is the first core vector of the device 'CPU (0)' (S11).

Subsequently, when the allocation for the first 'IRQ (0)' is completed, the IRQ number in the 'Port (0)', which is the first port, is incremented by '1' (S12), and then the IRQ number increased by '1' is It is compared whether it exceeds 'IRQ (N)', which is the last IRQ number in the IRQ list (S13). That is, when the IRQ number (IRQ (x)) increased by '1' is 'IRQ (0) < IRQ (x) ≤ IRQ (N)', the core vector is increased by '1' (S14).

Next, it is compared whether the '1' increased core vector exceeds the 'Core (K) Vector', which is the last 'Core (K) Vector' among the core vectors in the 'CPU (0)' (S15), that is, the '1' increased core vector (Core). If (x) Vector) is 'Core (0) Vector < Core (x) Vector ≤ Core (K) Vector', IRQ increased by '1' in process 'S12' is increased by '1' in process 'S14'. It is assigned to a core vector (S16).

'S16' → 'S12' → 'S13' → 'S14' → 'S15' In the process, the core vector (Core (x) Vector) increased by '1' from 'S14' is the final core vector 'Core (K)'. Vector' is repeatedly performed, and through this process, the IRQ of each port is sequentially assigned from the first core vector 'Core (0) Vector' to the last core vector 'Core (K) Vector', respectively. .

On the other hand, if the core vector (Core (x) Vector) increased by '1' in process 'S14' exceeds the 'Core (K) Vector', which is the final number among the core vectors, that is, 'Core (x) Vector > Core (K) Vector' (when x = K+1), returns to the first core vector, 'Core (0)', and allocates the IRQ to be currently allocated to the first core vector, 'Core (0)'. (S17, S16).

In addition, in process 'S13', if the IRQ number (IRQ (x)) increased by '1' exceeds 'IRQ (N)', which is the last IRQ number (in the case of X = N+1), the port '1' is increased (S18), and the '1' increased port (Port (x)) exceeds the last 'Port (M)', that is, 'Port (x) > Port (M)' (x = M +1), repeat 'S11' → 'S12' → 'S13' → 'S18' → 'S19'.

In this way, the interrupt distribution method according to the embodiment of the present invention is different from the conventional interrupt distribution method shown in FIG. N)' is assigned sequentially from 'Core (0) Vector' to 'Core (K) Vector' to each core. At this time, in 'Port (0) IRQ (0~N)', 'Port (M) IRQ (0~N)' is assigned from 'Core (0) Vector' to 'Core (K) Vector', but NUMA structure Accordingly, among the Ethernet ports, the Ethernet port corresponding to 'CPU (0)' allocates an IRQ only to the core vector of 'CPU (0)'.

That is, as shown in FIG. 6 , 'IRQ (0)' of 'Port (0)', which is the first port, is assigned to 'Core (0) Vector', which is the first core vector, and the second of 'Port (0)' 'IRQ (1)' is assigned to the second core vector, 'Core (1) Vector', and the third 'IRQ (2)' of 'Port (0)' is the third core vector, 'Core (2) Vector'. ' is assigned to Assign sequentially in this way.

On the other hand, when the number of IRQs of 'Port (0)' is greater than the number of core vectors, that is, when the number of IRQs of 'Port (0)' exceeds the number of core vectors of the CPU (IRQ (N) > Core ( K) Vector) is assigned to the 'Core (0) Vector', which is the first core vector from the IRQ that is exceeded. And, when the IRQ assignment of the first port, 'Port (0)', is completed, the next port is assigned in the same way as the IRQ assignment of the first port.

As such, in the interrupt distribution method according to the embodiment of the present invention, unlike the interrupt distribution method according to the prior art, the Ethernet port 'Port (0) IRQ (0~N)' to 'Port (M) IRQ (0~N)' By sequentially allocating from 'Core (0) Vector' to 'Core (K) Vector' to each core, the IRQ of each port is uniformly allocated and distributed to one core vector. An advantage of easy redistribution can be provided compared to a method of allocating IRQs by filling all the core vectors from the second core vector.

In addition, in the interrupt distribution method according to the embodiment of the present invention, as the IRQ of each port is uniformly distributed in the core vector, a free space exists for each core. Therefore, during redistribution, redistribution can be performed by mapping the core and IRQ 1:1. For example, the first 'IRQ (0)' of the first port 'Port (0)' is redistributed to the first core 'Core (0)', the second 'IRQ (1)' is the second core 'Core (1)' )' can be redistributed and used.

And, in the interrupt distribution method according to the embodiment of the present invention, the IRQ of each port of the Ethernet card corresponding to the first 'CPU (0)' is assigned to the core of 'CPU (0)' so as to conform to the NUMA structure. . This includes the process of transferring data coming into the Ethernet port from the memory of 'CPU (0)' to the memory space of 'CPU (1)' through the bus in the process of data processing when crossing the central processing units, so performance degradation is reduced. because it occurs

In addition, in the interrupt distribution method according to the embodiment of the present invention, the receive queue (Rx queue) and the transmit queue (Tx queue) in the Ethernet driver use the same queue to process the same data. deterioration can be prevented.

On the other hand, the interrupt distribution method according to the embodiment of the present invention can be implemented by modifying the source code in the Linux kernel (Linux Kernel) in the development stage. And, of course, a computer-readable recording medium in which a program for implementing the same is stored and a computer program stored in the computer-readable recording medium can also be implemented.

That is, the interrupt distribution method according to an embodiment of the present invention may be implemented in the form of a computer-readable programming language code recorded on a computer-readable recording medium. Here, any computer-readable recording medium may be computer-readable, and any device capable of storing data may be used. For example, the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, flash memory, solid state disk (SSD), or the like. Also, here, the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.

As described above, although preferred embodiments of the present invention have been described and illustrated using specific terms, such terms are only for clearly describing the present invention. In addition, it is obvious that various changes and changes can be made to the embodiments and described terms of the present invention without departing from the spirit and scope of the following claims. Such modified embodiments should not be separately understood from the spirit and scope of the present invention, but should be considered to fall within the scope of the claims of the present invention.

CPU (0), CPU (1): Central processing unit
Core (0) Vector~ Core (K) Vector : Core Vector
Port (0) ~ Port (M) : Ethernet port
IRQ (0) to IRQ (N): Interrupt request

Claims (7)

Interrupt distribution method of NUMA-based equipment that allocates multiple IRQs (Interrupt requests) for each port within NUMA (Non-Uniform Memory Access)-based equipment to a plurality of core vectors configured in a plurality of central processing units, respectively In
(a) allocating, by the NUMA-based equipment, a first IRQ among the IRQ lists configured in the corresponding port connected to the corresponding CPU according to the NUMA environment among the ports to the first core vector of the corresponding CPU;
(b) the NUMA-based device increases the number of the IRQ of the corresponding port by '1';
(c) the process by which the NUMA-based equipment increases the number of the core vector of the corresponding central processing unit by '1'; and
(d) allocating, by the NUMA-based device, the IRQ of the next number increased by '1' in the step (b) to the core vector of the next number increased by '1' in the step (c);
Interrupt distribution method of NUMA-based equipment comprising a.
The method of claim 1,
(e) the NUMA-based equipment compares the number of the core vector increased by '1' in the process (c) with the number of the final th core vector of the central processing unit, and the number of the core vector increased by '1' in the process (c) If it does not exceed the number of the core vector and the number of the last core vector of the central processing unit, the number of the current IRQ increased by '1' in the process (b) exceeds the number of the last IRQ ( In the process b), the process (d) is repeatedly performed to distribute the IRQs included in the corresponding port, and the core vector increased by '1' in the process (c) is the last core vector of the corresponding central processing unit. If it exceeds, after returning to the first core vector of the central processing unit, in the process (b), until the number of the current IRQ increased by '1' in the process (b) exceeds the number of the last IRQ. Interrupt distribution method for NUMA-based equipment, characterized in that it further comprises the step of repeatedly performing up to (d) to distribute IRQs included in the corresponding port.
Interrupt distribution method of NUMA-based equipment that allocates multiple IRQs (Interrupt requests) for each port within NUMA (Non-Uniform Memory Access)-based equipment to a plurality of core vectors configured in a plurality of central processing units, respectively In
(a) allocating, by the NUMA-based equipment, a first IRQ among the IRQ lists configured in the corresponding port connected to the corresponding CPU according to the NUMA environment among the ports to the first core vector of the corresponding CPU;
(b) the NUMA-based device increases the number of the IRQ of the corresponding port by '1';
(c) comparing, by the NUMA-based device, the number of the current IRQ increased by '1' in the step (b) with the number of the last IRQ of the corresponding port;
(d) As a result of the comparison of (c), the NUMA-based device sets the number of the core vector to '1' when the number of IRQs increased by '1' in the process (b) does not exceed the number of the last IRQ. process of increasing;
(e) a process in which the NUMA-based device compares the number of the core vector increased by '1' in the process (c) and the number of the final th core vector; and
(f) When the NUMA-based device compares the step (e), the number of the core vector increased by '1' in the step (d) does not exceed the number of the final th core vector, the step (b) The current IRQ increased by '1' is assigned to the current core vector increased by '1' in the process (d), and the number of the core vector increased by '1' in the process (d) is the number of the final th core vector the process of allocating the current IRQ increased by '1' in the process (b) to the first core vector of the corresponding central processing unit by returning to the first core vector of the corresponding central processing unit if it is exceeded;
Interrupt distribution method of NUMA-based equipment comprising a.
4. The method of claim 3,
(g) the NUMA-based device returns to the process (b) after the process (f) and repeats the process from the process (b) to the process (f), but the comparison result of (c); The interrupt distribution method of a NUMA-based device, characterized in that it further comprises the step of repeatedly performing until the number of the current IRQ increased by '1' in the step (b) exceeds the number of the last IRQ.
5. The method of claim 4,
(h) When the NUMA-based device compares (c) in the step (d), the number of the IRQ increased by '1' in the step (b) exceeds the number of the last IRQ, the port number The interrupt distribution method of a NUMA-based device, characterized in that it further comprises the step of performing the IRQ of the current port increased by '1' by '1' in the same manner from the steps (a) to (f).
A computer-readable recording medium storing a program for implementing the method of any one of claims 1 to 5 for distributing an interrupt of a NUMA-based device.
A computer program stored in a computer-readable recording medium for implementing the method of any one of claims 1 to 5 for distributing an interrupt of a NUMA-based device.

Images