CN113535370A - Method and equipment for realizing multiple RDMA network card virtualization of load balancing - Google Patents

Method and equipment for realizing multiple RDMA network card virtualization of load balancing Download PDF

Info

Publication number
CN113535370A
CN113535370A CN202010277822.XA CN202010277822A CN113535370A CN 113535370 A CN113535370 A CN 113535370A CN 202010277822 A CN202010277822 A CN 202010277822A CN 113535370 A CN113535370 A CN 113535370A
Authority
CN
China
Prior art keywords
network card
rdma network
rdma
virtualization
load
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010277822.XA
Other languages
Chinese (zh)
Inventor
孙夏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Zhixing Technology Co Ltd
Original Assignee
Shenzhen Zhixing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Zhixing Technology Co Ltd filed Critical Shenzhen Zhixing Technology Co Ltd
Priority to CN202010277822.XA priority Critical patent/CN113535370A/en
Publication of CN113535370A publication Critical patent/CN113535370A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45583Memory management, e.g. access or allocation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides a method and equipment for realizing multi-RDMA network card virtualization for load balancing. The method, the equipment, the related code storage medium and the like provided by the method are used for managing the virtualization process of the RDMA network card virtualization unit through the multi-network-card virtualization management unit, specifically, the low-load RDMA network card is selected to be virtualized through the RDMA network card virtualization unit through the (acquired) physical function module configuration of the RDMA network card and according to the load condition of each RDMA network card, so that the virtual RDMA network card is provided for a logic computer, and the multi-RDMA network card virtualization with balanced load is realized.

Description

Method and equipment for realizing multiple RDMA network card virtualization of load balancing
Technical Field
The invention relates to an RDMA network card virtualization technology in the field of computer communication, in particular to a method and equipment for realizing load balancing and multi-RDMA network card virtualization.
Background
Generally, virtualization generally refers to virtualizing a computer into a plurality of logical computers through virtualization technology; and the application programs of each logic computer can run in mutually independent spaces without influencing each other, so that the working efficiency of the computer is obviously improved. The virtualization can realize dynamic allocation, flexible scheduling and cross-domain sharing of IT resources by redefining and dividing the IT resources through software, so that the IT resource utilization rate is improved, the limited IT resources are used as infrastructure of computing capacity, and various and changeable requirements are met in the application process of various industries.
In the initial application scenario of virtualization technology, only the computing resources (e.g., CPU, GPU) and the storage resources (e.g., memory and hard disk) of a computer are virtualized by virtual machine technology (e.g., VMware) and container technology (e.g., docker), and a plurality of logic computers isolated from each other for each business operation and storage are virtualized, and often, communication between the logic computers is not involved, and communication between the logic computers virtualized by different machine devices is not involved. However, with the emergence of new scenes and the evolution of technology, the problem of communication between different logical computers (e.g., different logical computers of the same machine equipment, logical computers between different machine equipments) is promoted. It is obvious that configuring separate physical network devices for each logical computer enables communication between them, but there are no doubt limitations (after all, existing network device interfaces of machine devices are strictly set according to certain industry standards, and the number thereof is limited) and lack of flexibility. The macvlan is a network virtualization solution; a physical network card is virtualized into a plurality of virtual network cards, which is equivalent to split operation of the physical network card, and the virtual network cards are configured to each logic computer through a plurality of sub-interfaces, and one network card is changed into a plurality of sub-interfaces. The macvlan can also achieve flow isolation through different sub-interfaces; the macvlan can judge which virtual network card the packet needs to be handed to according to the destination MAC address of the received packet, and the virtual network card then hands the packet to the upper protocol stack for processing.
However, rdma (remote direct Memory access), i.e., a remote direct Memory access technology, has been applied in more and more new scenarios as a network communication with high throughput and low latency without intervention of operating systems of both communication parties. Compared with the conventional TCP/IP network communication supported by the common network card, the bottleneck problem of I/O (namely, the high overhead of processing (such as data moving operation and copying operation) related to network I/O under high-speed network conditions limits the bandwidth which can be sent between machines), in particular, the conventional TCP/IP network communication sends messages through a kernel, the mode of message transmission requires kernel intervention causes very low performance and very low flexibility, wherein the reason of low performance is mainly due to the high data moving and data copying overhead of the network communication which is transmitted through the kernel, the memory bandwidth is greatly different from the CPU bandwidth and the network bandwidth nowadays, and the reason of very low flexibility is mainly due to the fact that all network communication protocols are transmitted through the kernel, this approach is difficult to support new network protocols and new messaging protocols and send and receive interfaces, and RDMA communications supported by an RDMA network card (i.e., a special network card that supports RDMA communications) advantageously avoids the above-mentioned problems.
However, it should be noted that the RDMA technology does not need the intervention of the operating systems of both communication parties, and does not pass through the kernel protocol stack, so the aforementioned virtualization technology of the common network card cannot be directly applied to the virtualization of the RDMA network card, especially the virtualization of multiple RDMA network cards. Furthermore, even if virtualization can be implemented for multiple RDMA network cards of the same device, it is still necessary to consider how to fully, evenly, and efficiently utilize the individual RDMA network cards.
Disclosure of Invention
In view of this, the present invention provides a method and an apparatus for implementing load balancing multiple RDMA network card virtualization.
In one aspect, an embodiment of the present invention provides a method for implementing load balancing multiple RDMA network card virtualization.
The method for realizing the multi-RDMA network card virtualization of the load balancing comprises the following steps:
running network virtualization software on a device configured with multiple RDMA network cards, wherein a multiple network card virtualization management unit is included; the multi-network card virtualization management unit is configured to be coupled with each logic computer, the RDMA network card virtualization unit and further coupled with each RDMA network card through the RDMA network card virtualization unit;
the multi-network-card virtualization management unit is used for acquiring the configuration of the physical function modules of the RDMA network cards corresponding to the RDMA network cards, selecting the low-load RDMA network cards according to the configuration of the physical function modules of the RDMA network cards and the load conditions of the RDMA network cards, and virtualizing the RDMA network cards through the RDMA network card virtualization unit to provide virtual RDMA network cards for the logic computers; the configuration of the RDMA network card physical function module refers to configuration information of the RDMA network card physical function module of the RDMA network card virtualization unit; different RDMA network cards respectively correspond to different physical function modules of the RDMA network cards;
the RDMA network card virtualization unit is used for providing the RDMA network card physical function module and the RDMA network card virtual function module; the RDMA network card physical function module has the PCIe function of the full function of the RDMA network card and can be found, managed, processed and the like the RDMA network card serving as hardware; each RDMA network card physical function module can be associated with a certain number of RDMA network card virtual function modules; the RDMA network card virtual function module is only allowed to execute the resource configuration operation related to the RDMA network card virtual function module; the RDMA network card virtual function module and the RDMA network card physical function module associated with the RDMA network card virtual function module can provide a virtual RDMA network card for the logic computer through the corresponding RDMA network card, thereby realizing the configuration of the RDMA network card and providing the complete function of the RDMA network card;
when any logic computer configured with RDMA communication requirements is created, the multi-network-card virtualization management unit selects a low-load RDMA network card from the RDMA network cards according to the configuration of (acquired) RDMA network card physical function modules and the load condition of the RDMA network cards, allocates the available RDMA network card virtual function modules associated with the RDMA network card physical function modules corresponding to the RDMA network cards to the logic computer, and then achieves the purpose of providing virtual RDMA network cards for the logic computer.
On the other hand, the embodiment of the invention provides a load-balanced management method for multiple RDMA network card virtualization.
With reference to the first aspect, the method for managing multiple RDMA network card virtualization for load balancing includes:
running network virtualization software on a device configured with multiple RDMA network cards, wherein a multiple network card virtualization management unit is included; the multi-network card virtualization management unit is configured to be coupled with each logic computer and is coupled with the RDMA network card virtualization unit;
the foregoing multi-network card virtualization management unit is, for example, the multi-network card virtualization management unit according to the first aspect; the system is used for managing multiple RDMA network card virtualization and providing the RDMA virtual network card for a logic computer; specifically, the multi-network-card virtualization management unit can acquire the configuration of the physical function modules of the RDMA network cards corresponding to the RDMA network cards, select low-load RDMA network cards to virtualize according to the configuration of the physical function modules of the RDMA network cards and the load conditions of the RDMA network cards, and allocate available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards corresponding to the low-load RDMA network cards to a logic computer, so that the RDMA network card physical function modules and the allocated RDMA network card virtual function modules together provide virtual RDMA network cards for the logic computer through the low-load RDMA network cards;
the RDMA network card physical function module and the RDMA network card virtual function module related to the RDMA network card physical function module are both provided by the RDMA network card virtualization unit; the RDMA network card physical function module has the PCIe function of the full function of the RDMA network card, and can be found, managed, processed and the like the RDMA network card serving as hardware; each RDMA network card physical function module can be associated with a certain number of RDMA network card virtual function modules; the RDMA network card virtual function module is only allowed to execute the resource configuration operation related to the RDMA network card virtual function module; the two can provide virtual RDMA network card for logic computer through corresponding RDMA network card, realize RDMA network card configuration and provide complete RDMA network card function.
In yet another aspect, an embodiment of the present invention provides a machine device with multiple RDMA network cards.
With reference to the first and second aspects, the multi-screen card reader device includes:
no less than two RDMA network cards; wherein the content of the first and second substances,
the machine device can execute the method for virtualizing multiple RDMA network cards with balanced load in the first aspect or the method for managing multiple RDMA network cards with balanced load in the second aspect, and implement the virtualization for balanced load of the RDMA network cards.
In yet another aspect, an embodiment of the present invention provides a computer-readable storage medium.
With reference to the first and second aspects, the computer-readable storage medium stores code for multiple RDMA network card virtualization, and includes instructions for performing the method for load-balanced multiple RDMA network card virtualization according to the first aspect or the method for managing load-balanced multiple RDMA network card virtualization according to the second aspect.
The method, the device, the computer-readable storage medium storing related codes and the like for implementing load balancing virtualization of multiple RDMA network cards provided by the above embodiments manage the virtualization process of the RDMA network card virtualization unit through the multiple network card virtualization management unit, specifically, configure and select a low-load RDMA network card among the multiple RDMA network cards according to the load condition of each RDMA network card through the (acquired) RDMA network card physical function module to virtualize through the RDMA network card virtualization unit, and allocate an available RDMA virtual function module associated with the RDMA network card physical function module corresponding to the low-load RDMA network card and provided by the RDMA network card virtualization unit, also provided by the RDMA network card virtualization unit, to a logical computer so that the two RDMA network cards (the RDMA network card physical function module and the allocated RDMA network card virtualization unit associated therewith) together pass through the low-load RDMA network card, a virtual RDMA network card is provided for the logical computer.
The technical solution of the present invention is further described with reference to the accompanying drawings and specific embodiments.
Drawings
To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings related to a part of the embodiments of the present invention or the description in the prior art will be briefly introduced below.
Fig. 1 is a schematic diagram of a specific process for implementing load balancing virtualization of multiple RDMA network cards of a device according to some embodiments of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention is clearly and completely described below with reference to the drawings of the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of a portion of the invention and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In the prior art, macvlan is a network card virtualization scheme commonly used in the art. Specifically, the macvlan sets a plurality of mac addresses for one physical network card, and then sets a plurality of IP addresses, so that the mac addresses are configured to each logical computer through a plurality of sub-interfaces, and the macvlan can also communicate with an external network through a physical interface connection network cable, thereby realizing a change from "one physical network card access network" to "a plurality of virtual network card access networks". However, because the RDMA network card supporting high-speed communication avoids the intervention of operating systems of both communication parties during communication, and does not pass through a kernel protocol stack, a common network card virtualization technology such as macvlan cannot be directly applied to the virtualization of the RDMA network card basically, and particularly to the virtualization of multiple RDMA network cards. In addition, even if the multiple RDMA network cards of the same device can be virtualized, how to fully, evenly and efficiently utilize the RDMA network cards needs to be considered, so as to improve the utilization rate of hardware communication resources of the device and obtain better communication services.
Therefore, the invention provides a method and equipment for realizing multiple RDMA network card virtualization for load balancing. The following are some preferred embodiments of the invention. Wherein the content of the first and second substances,
some preferred embodiments of the present invention provide a method for multiple RDMA network card virtualization for load balancing. The method comprises the following steps:
running network virtualization software on a device configured with multiple RDMA network cards, wherein a multiple network card virtualization management unit is included; the multi-network card virtualization management unit is configured to be respectively coupled with each logic computer and provide a virtual network card for the logic computer; and is coupled with the RDMA network card virtualization unit, and is further coupled with each RDMA network card through the RDMA network card virtualization unit to finally realize the virtualization of the RDMA network card (in a physical sense);
the multi-network-card virtualization management unit can acquire configuration information of the RDMA network card physical function modules of the RDMA network card virtualization units corresponding to the RDMA network cards (namely configuration of the RDMA network card physical function modules, wherein different RDMA network cards configured on the equipment correspond to different RDMA network card physical function modules), select the low-load RDMA network card according to the configuration of the RDMA network card physical function modules and the load conditions of the RDMA network cards, and virtualize the RDMA network card through the RDMA network card virtualization units so as to provide a virtual RDMA network card for the logic computer;
the RDMA network card virtualization unit is used for providing the RDMA network card physical function module and the RDMA network card virtual function module; the RDMA network card virtualization unit is generally realized by combining an RDMA network card supporting hardware virtualization with related firmware and the like; generally, an RDMA network card (for example, an RDMA network card with SR-IOV function) supporting hardware virtualization may be regarded as an RDMA network card virtualization unit; that is, one device may have a plurality of RDMA network card virtualization units corresponding to the number of RDMA network cards configured on the device; the RDMA network card physical function module has the PCIe function of the full function of the RDMA network card and can be found, managed, processed and the like the RDMA network card serving as hardware; the multi-network-card virtualization management unit determines the corresponding physical function module of the RDMA network card through the configuration of the (acquired) physical function module of the RDMA network card, namely, the corresponding RDMA network card virtualization unit is selected, and further the corresponding RDMA network card is determined; in addition, each RDMA network card physical function module is an RDMA network card virtual function module which can be associated with a certain number of RDMA network cards (for example, an RDMA network card with SR-IOV function, the PF of which can be associated with tens of thousands of VFs at most); the RDMA network card virtual function module is only allowed to execute the resource configuration operation related to the RDMA network card virtual function module, for example, the RDMA network card virtual function module is mounted to a network card interface; the RDMA network card virtual function module and the RDMA network card physical function module associated with the RDMA network card virtual function module can provide a virtual RDMA network card for the logic computer through the corresponding RDMA network card, thereby realizing the configuration of the RDMA network card and providing the complete function of the RDMA network card;
wherein, for any logical computer which utilizes the device resource to communicate externally, if it has the RDMA communication need, it is configured when it is created: and selecting a low-load RDMA network card from the RDMA network cards according to the configuration of the (acquired) RDMA network card physical function module and the load condition of the RDMA network card, and allocating an available RDMA network card virtual function module associated with the RDMA network card physical function module corresponding to the low-load RDMA network card to the logic computer, so that the purpose of providing the logic computer with the virtual RDMA network card is realized, and the logic computer has the RDMA communication function.
Further, in some of the above-mentioned preferred embodiments, in a method for implementing load-balanced multiple RDMA network card virtualization, selecting a low-load RDMA network card for virtualization according to a load condition may be: and simultaneously acquiring configuration information of the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards (namely the configuration of the available RDMA network card virtual function modules), judging the load of the RDMA network cards according to the number of the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards corresponding to the RDMA network cards, and preferentially distributing the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards with the largest number of the available RDMA network card virtual function modules.
Further, in some of the above-mentioned preferred embodiments, in a method for implementing load-balanced multiple RDMA network card virtualization, selecting a low-load RDMA network card for virtualization according to a load condition may be: and detecting the load condition of each RDMA network card, and reporting the load condition to a multi-network-card virtualization management unit in the RDMA network card or enabling the multi-network-card virtualization management unit to obtain the load detection result of each RDMA network card so as to select a low-load RDMA network card and perform subsequent RDMA network card virtualization. And further, the RDMA network card load therein is preferably an average load per unit time; therefore, compared with the instantaneous load of the RDMA network card, the real load condition of the RDMA network card in the communication process can be reflected better.
Further, in some of the above preferred embodiments, there is provided a method for implementing load-balanced multiple RDMA network card virtualization, wherein when any one of the logical computers is removed, the allocation of its RDMA network card virtual function module is cancelled. For example, when a virtual machine is used as a logical computer, when an RDMA network card virtual function module is allocated, the RDMA network card virtual function module to be allocated is coupled to a virtual RDMA network card interface of the virtual machine; when the allocation is cancelled, the RDMA network card virtual function module is decoupled from the virtual RDMA network card interface. The specific implementation is generally as follows: during allocation, writing configuration information of the RDMA network card virtual function module to be allocated (namely configuration of the RDMA network card virtual function module) into a corresponding position of a virtual machine partition; when the allocation is cancelled, the configuration is removed, the occupation of the RDMA network card virtual function module is eliminated, and the purposes of releasing the RDMA network card virtual function module and cancelling the allocation can be realized.
Further, in some of the above-mentioned preferred embodiments, in a method for implementing multiple RDMA network card virtualization for load balancing, the logical computer may also be a container/container group (i.e., a logical computer composed of several containers, such as Pod in Kubernets); when the RDMA network card virtual function module is distributed, the RDMA network card virtual function module to be distributed only needs to be configured and written into a namespace (namespace) of a target container/container group; when the container/container group is removed, that is, when the allocation is to be cancelled, the allocation can be cancelled by shifting the configuration of the RDMA network card virtual function module out of the namespace of the container/container group to be removed, so that the occupation is removed. Still further, where the RDMA network card virtual function module configuration is moved out of the namespace of the container/container group to be removed, it is typically moved to a common namespace for efficient management and multiplexing of the RDMA network card virtual function modules in the containerization scenario.
Other preferred embodiments of the present invention provide a method for managing load-balanced multiple RDMA network card virtualization. The management method comprises the following steps: running network virtualization software on a device configured with multiple RDMA network cards, wherein a multiple network card virtualization management unit is included; the multi-network card virtualization management unit is configured to be coupled with each logic computer and is coupled with the RDMA network card virtualization unit;
wherein, the multiple network card virtualization management unit is a multiple network card virtualization management unit in the method for implementing load balancing multiple RDMA network card virtualization according to the preferred embodiment; the system is used for managing multiple RDMA network card virtualization, providing an RDMA virtual network card for a logic computer and the like; specifically, the multi-network-card virtualization management unit can obtain the configuration of the physical function modules of the RDMA network cards corresponding to the RDMA network cards, select low-load RDMA network cards to virtualize according to the configuration of the physical function modules of the RDMA network cards and the load conditions of the RDMA network cards, allocate available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards corresponding to the low-load RDMA network cards to a logic computer, and provide the virtual RDMA network cards for the logic computer through the low-load RDMA network cards by the RDMA network card physical function modules and the allocated RDMA network card virtual function modules; virtual RDMA network cards are provided for logical computers to enable virtualization of multiple RDMA network cards with load balancing across devices, thereby providing efficient RDMA communication services. It should be added that, the RDMA network card physical function module and the RDMA network card virtual function module associated therewith are provided by the RDMA network card virtualization unit. The RDMA network card virtualization unit is generally realized by combining an RDMA network card supporting hardware virtualization with related firmware and the like; generally, an RDMA network card supporting hardware virtualization can be regarded as an RDMA network card virtualization unit. The RDMA network card physical function module has the PCIe function of the full function of the RDMA network card, and can be found, managed, processed and the like the RDMA network card serving as hardware; in addition, each RDMA network card physical function module is an RDMA network card virtual function module which can be associated with a certain number of RDMA network cards (for example, an RDMA network card with SR-IOV function, the PF of which can be associated with tens of thousands of VFs at most); the RDMA network card virtual function module is only allowed to execute the resource configuration operation related to the RDMA network card virtual function module, for example, the RDMA network card virtual function module is mounted to a network card interface; the two modules (namely the RDMA network card virtual function module and the RDMA network card physical function module associated with the RDMA network card virtual function module) can provide a virtual RDMA network card for the logic computer through the corresponding RDMA network card, thereby realizing the configuration of the RDMA network card and providing the complete functions of the RDMA network card.
The following describes, with reference to a specific embodiment, a process for implementing load balancing virtualization of multiple RDMA network cards on a device, that is, a process for providing a virtual RDMA network card for a logical computer of the device configured with multiple RDMA network cards in a load balancing manner.
As shown in FIG. 1, there is shown a server device configured with two RDMA network cards supporting SR-IOV functionality, on which containers/container groups are deployed; as shown in fig. 1, on which kubernets run, 4 Pod are pulled (spawn) (i.e., the minimum scheduling unit of kubernets, which is actually made up of another set of tightly coupled containers, which can be understood as a logical computer);
run mult _ sriov _ cni and RDMA network card driver supporting SR-IOV function on it, etc.; before providing (virtual) RDMA network cards for each Pod, detecting all PFs (physical function modules of the RDMA network cards), starting SRIOV functions for the RDMA network cards which are not started, and acquiring associated VFs (virtual function modules of the RDMA network cards), namely the VFs under the same PF;
when the Pod is pulled up, the specific process of providing a virtual RDMA network card for the Pod can be as follows:
mult _ sriov _ cni acquires the information of each PF and the associated usable VF;
selecting the PF with the maximum available VF number according to the available VF number of each PF;
acquiring a namespace corresponding to the Pod;
selecting an available VF from associated VFs of the selected PF, and moving the available VF to a namespace corresponding to the Pod to realize the distribution of the VF;
and after the VF is distributed, configuring an IP (Internet protocol) for the VF and the like, and finally providing an RDMA (remote direct memory access) communication function for the corresponding Pod through the corresponding PF and the distributed associated VF.
When a certain Pod is removed and releases the resource, the specific processes of virtual RDMA network card unloading and resource release are as follows:
when a request for removing and releasing a certain Pod is received, configuration information of a VF corresponding to a target Pod is received at the same time;
and moving the VF to a common name space for multiplexing so as to complete the release of the VF, namely the unloading of the virtual RDMA network card.
On the basis of the method provided by the above preferred embodiment, further preferred embodiments of the present invention provide a machine device with multiple RDMA network cards. The device is configured with a plurality of RDMA network cards; the machine device can execute the load-balanced multiple RDMA network card virtualization method or the load-balanced multiple RDMA network card virtualization management method in any of the preferred embodiments, thereby implementing the load-balanced virtualization of the RDMA network cards.
On the basis of the method provided by the preferred embodiment, the invention further provides a computer-readable storage medium. The computer-readable storage medium stores code for multiple RDMA network card virtualization, including instructions for performing the method for load-balanced multiple RDMA network card virtualization or the method for managing load-balanced multiple RDMA network card virtualization in any of the preferred embodiments described above.
The above description is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto.

Claims (10)

1. A method for implementing multiple RDMA network card virtualization for load balancing, comprising:
running network virtualization software on a device configured with multiple RDMA network cards, wherein the network virtualization software comprises a multiple network card virtualization management unit;
the multi-network card virtualization management unit is configured to be respectively coupled with each logic computer, coupled with the RDMA network card virtualization unit and further coupled with each RDMA network card through the RDMA network card virtualization unit;
the multi-network-card virtualization management unit is used for acquiring the configuration of the physical function modules of the RDMA network cards corresponding to the RDMA network cards, selecting low-load RDMA network cards according to the configuration of the physical function modules of the RDMA network cards and the load conditions of the RDMA network cards, virtualizing the low-load RDMA network cards through the RDMA network card virtualization unit, and providing virtual RDMA network cards for the logic computer; the configuration of the RDMA network card physical function module is the configuration information of the RDMA network card physical function module of the RDMA network card virtualization unit; the RDMA network cards respectively correspond to different physical function modules of the RDMA network cards;
the RDMA network card virtualization unit is used for providing the RDMA network card physical function module and the RDMA network card virtual function module; the RDMA network card physical function module has the PCIe function of the full function of the RDMA network card; each RDMA network card physical function module is associated with a plurality of RDMA network card virtual function modules; the RDMA network card virtual function module is used for executing the resource configuration operation related to the RDMA network card virtual function module; the RDMA network card virtual function module and the RDMA network card physical function module associated with the RDMA network card virtual function module can provide a virtual RDMA network card for the logic computer through the corresponding RDMA network card;
when any logic computer configured with RDMA communication requirements is created, the multi-network-card virtualization management unit selects a low-load RDMA network card from the RDMA network cards through the configuration of the RDMA network card physical function modules and the load of the RDMA network cards, and allocates the available RDMA network card virtual function modules associated with the RDMA network card physical function modules corresponding to the low-load RDMA network cards to the logic computers.
2. The method for multiple RDMA network card virtualization for load balancing according to claim 1,
the method for selecting the low-load RDMA network card to virtualize according to the load condition comprises the following steps:
and simultaneously acquiring the configuration of the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards, judging the load of each RDMA network card according to the number of the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards corresponding to each RDMA network card, and preferentially distributing the available RDMA network card virtual function modules associated with the physical function modules of the RDMA network cards with the largest number of the available RDMA network card virtual function modules.
3. The method for multiple RDMA network card virtualization for load balancing according to claim 1,
the method for selecting the low-load RDMA network card to virtualize according to the load condition comprises the following steps:
and detecting the load condition of each RDMA network card, and reporting the load condition to a multi-network-card virtualization management unit in the RDMA network card or enabling the multi-network-card virtualization management unit to obtain the load detection result of each RDMA network card so as to select the low-load RDMA network card.
4. The method for multiple RDMA network card virtualization for load balancing according to claim 3,
the RDMA network card load to be detected is the average load in unit time.
5. The method for multiple RDMA network card virtualization for load balancing according to claim 1,
when any logic computer is removed, the RDMA network card virtual function module is not allocated.
6. The method for multiple RDMA network card virtualization for load balancing according to claim 1,
the logic computer is a container/container group; wherein the content of the first and second substances,
when the RDMA network card virtual function module is distributed, the configuration of the RDMA network card virtual function module to be distributed is written into the name space of a target container/container group;
and when the allocation is cancelled, the RDMA network card virtual function module is configured and moved out of the name space of the container/container group to be removed.
7. The method for multiple RDMA network card virtualization for load balancing according to claim 6,
when the RDMA network card virtual function module configuration is moved out of the namespace of the container/container group, the RDMA network card virtual function module configuration is moved to the public namespace management and multiplexing.
8. A management method for load-balanced multiple RDMA network card virtualization is characterized by comprising the following steps:
running network virtualization software on a device configured with multiple RDMA network cards, including the multiple network card virtualization management unit of any of the methods of claims 1-7;
the multi-network card virtualization management unit is configured to be coupled with each logic computer and coupled with the RDMA network card virtualization unit respectively;
the multi-network card virtualization management unit is used for managing multi-RDMA network card virtualization and providing the RDMA virtual network card for the logic computer;
the multi-network-card virtualization management unit can acquire the configuration of the RDMA network card physical function modules corresponding to each RDMA network card, select low-load RDMA network cards to be virtualized according to the configuration of the RDMA network card physical function modules and the load condition of each RDMA network card, and allocate available RDMA network card virtual function modules related to the RDMA network card physical function modules corresponding to the low-load RDMA network cards to the logic computer;
the RDMA network card physical function module and the RDMA network card virtual function module related to the RDMA network card physical function module are both provided by the RDMA network card virtualization unit; the RDMA network card physical function module has a PCIe function of the full function of the RDMA network card; each RDMA network card physical function module is associated with a plurality of RDMA network card virtual function modules; the RDMA network card virtual function module is used for executing the resource configuration operation related to the RDMA network card virtual function module; the RDMA network card virtual function module and the RDMA network card physical function module associated with the RDMA network card virtual function module can provide a virtual RDMA network card for the logic computer through the corresponding RDMA network card.
9. An apparatus, comprising:
no less than two RDMA network cards; wherein the content of the first and second substances,
the device is capable of performing the method of any of claims 1-8, implementing virtualization for load balancing of the RDMA network cards.
10. A computer-readable storage medium, characterized in that,
storing code for multiple RDMA network card virtualization;
including instructions for performing the operations of the method of any of claims 1-8.
CN202010277822.XA 2020-04-09 2020-04-09 Method and equipment for realizing multiple RDMA network card virtualization of load balancing Pending CN113535370A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010277822.XA CN113535370A (en) 2020-04-09 2020-04-09 Method and equipment for realizing multiple RDMA network card virtualization of load balancing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010277822.XA CN113535370A (en) 2020-04-09 2020-04-09 Method and equipment for realizing multiple RDMA network card virtualization of load balancing

Publications (1)

Publication Number Publication Date
CN113535370A true CN113535370A (en) 2021-10-22

Family

ID=78087735

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010277822.XA Pending CN113535370A (en) 2020-04-09 2020-04-09 Method and equipment for realizing multiple RDMA network card virtualization of load balancing

Country Status (1)

Country Link
CN (1) CN113535370A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363170A (en) * 2021-12-29 2022-04-15 中国电信股份有限公司 Container service network configuration method and related product

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104168135A (en) * 2014-08-06 2014-11-26 中国船舶重工集团公司第七0九研究所 Network card resource pooling management method and system
US20180052701A1 (en) * 2016-08-17 2018-02-22 Red Hat Israel, Ltd. Hot-plugging of virtual functions in a virtualized environment
CN110071900A (en) * 2018-01-23 2019-07-30 华为技术有限公司 The method and apparatus that data are sent

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104168135A (en) * 2014-08-06 2014-11-26 中国船舶重工集团公司第七0九研究所 Network card resource pooling management method and system
US20180052701A1 (en) * 2016-08-17 2018-02-22 Red Hat Israel, Ltd. Hot-plugging of virtual functions in a virtualized environment
CN110071900A (en) * 2018-01-23 2019-07-30 华为技术有限公司 The method and apparatus that data are sent

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363170A (en) * 2021-12-29 2022-04-15 中国电信股份有限公司 Container service network configuration method and related product

Similar Documents

Publication Publication Date Title
US11934341B2 (en) Virtual RDMA switching for containerized
KR102008551B1 (en) Offloading virtual machine flows to physical queues
US8589920B2 (en) Resource allocation
CN109302466B (en) Data processing method, related device and computer storage medium
WO2015196931A1 (en) Disk io-based virtual resource allocation method and device
CN110098946B (en) Method and device for deploying virtualized network element equipment
US9910687B2 (en) Data flow affinity for heterogenous virtual machines
US10897428B2 (en) Method, server system and computer program product for managing resources
CN113535319A (en) Method, equipment and storage medium for realizing multiple RDMA network card virtualization
US11928493B2 (en) Sharing of FPGA board by multiple virtual machines
US20190007339A1 (en) Method and device for managing stateful application on server
CN102316043A (en) Port virtualization method, switch and communication system
US8458702B1 (en) Method for implementing user space up-calls on java virtual machine before/after garbage collection
CN113296926B (en) Resource allocation method, computing device and storage medium
US20210157652A1 (en) Customized partitioning of compute instances
CN114050998A (en) Method, device, electronic equipment and medium for realizing remote direct memory access
CN112583655B (en) Data transmission method and device, electronic equipment and readable storage medium
CN113535370A (en) Method and equipment for realizing multiple RDMA network card virtualization of load balancing
WO2018057165A1 (en) Technologies for dynamically transitioning network traffic host buffer queues
CN113254160B (en) IO resource request method and device
CN111294220A (en) Network isolation configuration method and device based on nginx
US7669202B1 (en) Resource management
US20090182927A1 (en) Direct Memory Move of Multiple Buffers Between Logical Partitions
CN111245794B (en) Data transmission method and device
EP3811210B1 (en) Method and supporting node for supporting process scheduling in a cloud system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination