CN113259142B - Multicast performance optimization method, system, device and storage medium in virtual machine - Google Patents

Multicast performance optimization method, system, device and storage medium in virtual machine Download PDF

Info

Publication number
CN113259142B
CN113259142B CN202110764857.0A CN202110764857A CN113259142B CN 113259142 B CN113259142 B CN 113259142B CN 202110764857 A CN202110764857 A CN 202110764857A CN 113259142 B CN113259142 B CN 113259142B
Authority
CN
China
Prior art keywords
packet loss
stream data
loss rate
multicast
multicast stream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110764857.0A
Other languages
Chinese (zh)
Other versions
CN113259142A (en
Inventor
李文成
潘全通
顾剑
刘勇鹏
陈鲍孜
罗勇
孟祥波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Phytium Technology Co Ltd
Original Assignee
Phytium Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Phytium Technology Co Ltd filed Critical Phytium Technology Co Ltd
Priority to CN202110764857.0A priority Critical patent/CN113259142B/en
Publication of CN113259142A publication Critical patent/CN113259142A/en
Application granted granted Critical
Publication of CN113259142B publication Critical patent/CN113259142B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/02Details
    • H04L12/16Arrangements for providing special services to substations
    • H04L12/18Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/185Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with management of multicast group membership
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0823Errors, e.g. transmission errors
    • H04L43/0829Packet loss
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/16Threshold monitoring
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/4557Distribution of virtual machine instances; Migration and load balancing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45583Memory management, e.g. access or allocation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45591Monitoring or debugging support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Abstract

The application relates to the technical field of multicast, and discloses a method, a system, a device and a storage medium for optimizing multicast performance in a virtual machine, wherein the method comprises the following steps: setting a multicast target, wherein the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded; sending multicast stream data corresponding to the multicast stream data quantity to a virtual machine, wherein the virtual machine carries out data transmission to multicast group members corresponding to the number of the multicast stream data to be forwarded; a network card is virtualized and transmitted to the virtual machine in a virtual mode by adopting an SR-IOV technology. The invention can improve the network performance through network card virtualization, thereby ensuring that the multicast performance of the virtual machine can meet the required multicast requirement.

Description

Multicast performance optimization method, system, device and storage medium in virtual machine
Technical Field
The present application relates to the field of multicast technologies, and in particular, to a method, a system, an apparatus, and a storage medium for optimizing multicast performance in a virtual machine.
Background
The multicast technology is one of three modes of IP network data transmission, is a data transmission mode between unicast and broadcast, can meet the requirement of sending information from a source host to a plurality of target hosts in a network, and simultaneously solves the problem of low efficiency of the unicast and broadcast modes.
In the era of the prevalence of virtualization and cloud platforms, the performance of the network becomes an unavoidable topic. In some multicast environments requiring high precision and high timeliness, the requirements on network performance are more strict.
Factors influencing the application layer multicast performance in the virtual machine mainly include low performance of a virtual network card, unreliable UDP data transmission protocol, loss of computing performance of a virtual CPU, unbalanced load and the like. However, there is a blank point in the current scheme for solving the multicast performance in the virtual machine, and the required multicast requirements cannot be met.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, a system, an apparatus, and a storage medium for optimizing multicast performance in a virtual machine, which can improve network performance through network card virtualization, thereby ensuring that the multicast performance of the virtual machine can meet a required multicast requirement.
In order to achieve the purpose, the invention adopts the following technical scheme:
the first aspect of the present invention provides a multicast performance optimization method in a virtual machine, where the method includes:
setting a multicast target, wherein the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded;
sending multicast stream data corresponding to the multicast stream data volume to a virtual machine, wherein the virtual machine carries out data transmission to multicast group members corresponding to the multicast stream data number to be forwarded according to the multicast stream data;
a network card is virtualized and transmitted to the virtual machine in a virtual mode by adopting an SR-IOV technology.
According to a manner that can be realized by the first aspect of the present invention, the multicast target further includes a multicast stream data transmission rate threshold, and before the step of virtualizing a network card by using SR-IOV technology and transmitting the network card to the virtual machine is executed, the method further includes: calculating the data transmission rate of each multicast stream data every a preset time interval;
and only when the minimum value of the data transmission rate is lower than the threshold value of the multicast stream data transmission rate, the step of virtualizing a network card by adopting the SR-IOV technology and transmitting the network card to the virtual machine is executed.
According to a manner that can be achieved by the first aspect of the present invention, the multicast target further includes a first packet loss rate threshold, and the method further includes:
and monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than the first packet loss rate threshold, adding the huge pages.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the huge page is added, if the packet loss rate of one multicast stream data transmission is larger than the first packet loss rate threshold, adjusting the size of the huge page, and increasing the size of the huge page by multiple times when the size of the huge page is adjusted each time;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold or the adjusted huge page size reaches the maximum memory value.
According to an implementation manner of the first aspect of the present invention, the obtaining the maximum memory value according to the total bandwidth of the virtual machine specifically includes:
matching the total bandwidth of the virtual machine with a preset standard bandwidth list, wherein each standard bandwidth in the standard bandwidth list is associated with a standard macro page size;
and taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
and if the packet loss rate of the multicast stream data transmission still exists when the adjusted huge page size reaches the maximum memory value, the UDP cache is added.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the UDP cache is added, and if the packet loss rate of multicast stream data transmission still exists and is larger than the first packet loss rate threshold value, adjusting the size of the UDP cache;
and stopping adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold, wherein the UDP buffer size is multiplied when the UDP buffer size is adjusted each time.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
and adjusting the receiving and sending soft interrupts of the network card to balance the interrupt load of the network card.
According to a manner that can be achieved by the first aspect of the present invention, the multicast target further includes a second packet loss rate threshold, and the method further includes:
and monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than the second packet loss rate threshold value, adding a huge page and a UDP cache.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
after the huge page and the UDP cache are added, detecting the packet loss rate of each multicast stream data transmission, if the packet loss rate of one multicast stream data transmission is still larger than the second packet loss rate threshold value, adjusting the size of the huge page, and increasing the size of the huge page by multiple times when the size of the huge page is adjusted each time;
determining a UDP cache adjustment value according to the adjusted size of the huge page, and adjusting the size of the UDP cache to the UDP cache adjustment value;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
A second aspect of the present invention provides a multicast performance optimization system in a virtual machine, the system including:
the multicast target setting module is used for setting a multicast target, and the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded;
a sending module, configured to send multicast stream data corresponding to the multicast stream data amount to a virtual machine, so that the virtual machine performs data transmission to a multicast group member corresponding to the number of multicast stream data to be forwarded according to the multicast stream data;
and the network card virtual module is used for virtualizing a network card to transmit to the virtual machine by adopting the SR-IOV technology.
According to an implementation manner of the second aspect of the present invention, the multicast target further includes a multicast stream data transmission rate threshold, and the system further includes a calculating module, configured to calculate a data transmission rate of each multicast stream data every preset time period;
and the network card virtual module virtualizes a network card to transmit to the virtual machine by adopting SR-IOV technology only when the minimum value of the data transmission rate is lower than the multicast stream data transmission rate threshold value.
According to a manner that can be realized by the second aspect of the present invention, the multicast target further includes a first packet loss rate threshold, and the system further includes:
the first monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
and the huge page increasing module is used for increasing the huge pages when the packet loss rate of the multicast stream data transmission is greater than the first packet loss rate threshold.
According to a manner that can be realized in the second aspect of the present invention, the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after adding the macro page, and the system further includes:
the first big page adjusting module is used for adjusting the size of the big page when the packet loss rate of one multicast stream data transmission is larger than the first packet loss rate threshold value, and the size of the big page is increased by multiple times when the size of the big page is adjusted each time;
and the first giant page adjusting module stops adjusting the size of the giant page when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold or the adjusted giant page size reaches a maximum memory value.
According to a manner that can be implemented in the second aspect of the present invention, the first macro page adjustment module includes a maximum memory value obtaining unit, where the maximum memory value obtaining unit is configured to obtain the maximum memory value according to a total bandwidth of the virtual machine, and the maximum memory value obtaining unit specifically includes:
the matching subunit is used for matching the total bandwidth size of the virtual machine with a preset standard bandwidth list, and each standard bandwidth in the standard bandwidth list is associated with a standard giant page size;
and the determining subunit is used for taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
According to an enabling manner of the second aspect of the invention, the system further comprises:
and the UDP cache increasing module is used for increasing the UDP cache when the packet loss rate of the multicast stream data transmission still exists and is greater than the first packet loss rate threshold when the adjusted huge page size reaches the maximum memory value.
According to an implementable manner of the second aspect of the present invention, the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after adding the UDP buffer, and the system further includes:
a first UDP buffer adjustment module, configured to adjust a size of the UDP buffer if a packet loss rate of a multicast stream data transmission still exceeds the first packet loss rate threshold;
the first UDP buffer adjustment module stops adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold, where the UDP buffer size is increased by multiple times when adjusting the UDP buffer size each time.
According to an enabling manner of the second aspect of the invention, the system further comprises:
and the load balancing module is used for adjusting the receiving and sending soft interrupts of the network card so as to balance the interrupt load of the network card.
According to a manner that can be realized by the second aspect of the present invention, the multicast target further includes a second packet loss rate threshold, and the system further includes:
the second monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
and the huge page and UDP cache increasing module is used for increasing the huge page and UDP cache when the packet loss rate of the multicast stream data transmission is greater than the second packet loss rate threshold.
According to a manner that can be realized in the second aspect of the present invention, the second monitoring module is further configured to detect a packet loss rate of each multicast stream data transmission after adding the megapage and the UDP buffer, and the system further includes:
the second macro page adjusting module is used for adjusting the size of the macro page if the packet loss rate of one multicast stream data transmission is larger than the second packet loss rate threshold, and the size of the macro page is increased by multiple times when the size of the macro page is adjusted each time;
the second UDP cache adjusting module is used for determining a UDP cache adjusting value according to the adjusted size of the huge page and adjusting the size of the UDP cache to the UDP cache adjusting value;
and the second huge page adjusting module stops adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
A third aspect of the present invention provides an apparatus for optimizing multicast performance in a virtual machine, where the apparatus includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor, when executing the computer program, implements a method for optimizing multicast performance in a virtual machine as described in any one of the above embodiments.
A fourth aspect of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the method for optimizing multicast performance in a virtual machine according to any one of the above embodiments is implemented.
The invention can improve the network performance through network card virtualization, thereby ensuring that the multicast performance of the virtual machine can meet the required multicast requirement.
Drawings
Fig. 1 is a schematic flowchart of a multicast performance optimization method in a virtual machine according to a preferred embodiment of the present invention;
fig. 2 is a schematic structural diagram of a multicast performance optimization system in a virtual machine according to a preferred embodiment of the present invention.
Reference numerals:
the device comprises a setting module 1, a sending module 2 and a network card virtual module 3.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic flowchart illustrating a multicast performance optimization method in a virtual machine according to a preferred embodiment of the present invention.
As shown in fig. 1, the method includes:
s1, setting a multicast target, wherein the multicast target comprises the multicast stream data volume and the number of the multicast stream data to be forwarded;
s2 sending multicast stream data corresponding to the multicast stream data amount to a virtual machine, where the virtual machine performs data transmission to multicast group members corresponding to the number of multicast stream data to be forwarded according to the multicast stream data;
s3 virtualizes a network card to transmit to the virtual machine by SR-IOV technology.
The SR-IOV technology is a virtualization solution based on hardware, and can improve performance and scalability. The SR-IOV standard virtualizes a PCIE network controller into multiple PCIE devices, and each device can be directly allocated to one virtual machine, allowing PCIE devices to be efficiently shared between virtual machines, and it is implemented in hardware, and can obtain I/O performance comparable to native performance. Compared with the fully virtualized or semi-virtualized virtio technology, the SR-IOV has better performance.
A network card is virtualized by adopting an SR-IOV technology and is transmitted to a virtual machine in a transparent mode, and in the specific implementation, the network card is virtualized on a physical machine by adopting the SR-IOV technology, and the purpose can be achieved by writing a numerical value into an SR-IOV _ numvfs file under sysfs. The sysfs is a memory-based file system that functions to provide kernel information to user programs in the form of files.
The invention virtualizes a network card by directly adopting SR-IOV technology to pass through to the virtual machine, and can improve multicast performance.
Further, the multicast target further includes a multicast stream data transmission rate threshold, and before the step of virtualizing a network card to pass through to the virtual machine by using the SR-IOV technology is executed, the method further includes: calculating the data transmission rate of each multicast stream data every a preset time interval;
and only when the minimum value of the data transmission rate is lower than the threshold value of the multicast stream data transmission rate, the step of virtualizing a network card by adopting the SR-IOV technology and transmitting the network card to the virtual machine is executed.
The invention provides the limiting condition of the virtual network card, so that the virtualization of the network card is more in line with the actual situation, and the network card resource is saved.
The data transmission rate refers to the number of multicast stream data transmitted by the virtual machine to the multicast group members in a unit time. And calculating the data transmission rate of each multicast stream data, namely calculating the data transmission rate when the virtual machine transmits the multicast stream data to each multicast group member. For example, when the number of multicast stream data to be forwarded is 30, the number of corresponding multicast members is 30, the virtual machine transmits data to the 30 multicast members according to the multicast stream data, for each multicast member transmitting a multicast stream data, the data transmission rate of each multicast stream data needs to be calculated every preset time interval, and if the minimum value of the data transmission rate in the calculation result is lower than the multicast stream data transmission rate threshold, a virtual network card is transmitted to the virtual machine by using the SR-IOV technology.
Preferably, the multicast stream data transmission rate threshold is determined according to the requirement of bidirectional 500Mbps transmission of multicast stream data, that is, the multicast stream data transmission rate threshold is 500 megabits per second. Correspondingly, if the minimum value of the data transmission rates between the virtual machine and each multicast group member is lower than the 500 megabits per second, a network card is virtualized and transmitted to the virtual machine by adopting the SR-IOV technology.
The multicast target is set according to the required multicast requirement, the embodiment of the invention judges whether the transmission rate of the multicast stream data meets the network performance requirement according to the multicast target, and the network card virtualization is carried out to improve the network performance when the transmission rate does not meet the network performance requirement, thereby ensuring that the multicast performance of the virtual machine can meet the required multicast requirement.
In one embodiment, the multicast target further includes a first packet loss rate threshold, and the method further includes:
and monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than the first packet loss rate threshold, adding the huge pages.
Preferably, the first packet loss rate threshold is set to 10e-6
Wherein the original size value of the macro page is set according to a minimum size threshold. For example, if the minimum size threshold is set to 2 megabits, the size of the macro page is 2 megabits when the macro page is added. In particular, the corresponding macro page may be allocated by writing a corresponding size value to the nr _ hugepages file under sysfs.
For testing the effect of increasing huge pages, the test environment is set as follows: the source host sends multicast stream data to the virtual machine, and the virtual machine forwards the data to 30 multicast members after receiving the multicast stream data.
The packet loss rate of each multicast stream data transmission is tested, the average packet loss rate is calculated, the test result is shown in table 1, and it can be known from table 1 that the packet loss rate of multicast stream data transmission can be reduced by adding huge pages under the condition of different bandwidths for the virtual machine.
TABLE 1
Figure 412193DEST_PATH_IMAGE001
The invention adds the huge page when the packet loss rate of multicast stream data transmission is larger than the first packet loss rate threshold, solves the problem of jitter packet loss caused by miss problem generated by TLB addressing, can effectively reduce the packet loss rate of multicast stream data transmission, and enables the multicast performance of the virtual machine to meet certain low packet loss rate requirement.
Further, the method further comprises:
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the huge page is added, if the packet loss rate of one multicast stream data transmission is larger than a first packet loss rate threshold value, adjusting the size of the huge page, and enabling the size of the huge page to be multiplied each time the size of the huge page is adjusted;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold or the adjusted huge page size reaches the maximum memory value.
The embodiment of the invention adjusts the size of the huge page according to the packet loss rate, and the size adjustment of the huge page needs to be limited by the maximum memory of the virtual machine, thereby achieving the optimization of the huge page value under the condition of limited resources and avoiding the waste of resources of the virtual machine caused by infinitely adjusting the huge page.
The maximum memory value can be preset, and the relevant configuration file is stored in the virtual machine according to the maximum memory value, so that the maximum memory value can be obtained according to the configuration file when the huge page is adjusted.
In another embodiment, the obtaining the maximum memory value according to the total bandwidth of the virtual machine specifically includes:
matching the total bandwidth of the virtual machine with a preset standard bandwidth list, wherein each standard bandwidth in the standard bandwidth list is associated with a standard macro page size;
and taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
The embodiment of the invention determines the maximum memory value according to the total bandwidth of the virtual machine, so that the determination of the maximum memory value is suitable for the specific virtual machine, and the objectivity of the macro page adjustment is improved.
Further, the method further comprises:
and if the packet loss rate of the multicast stream data transmission still exists when the adjusted huge page size reaches the maximum memory value, the UDP cache is added.
And setting the original value of the UDP buffer according to the lowest buffer threshold value. Preferably, the lowest caching threshold is set to 128 bytes.
For testing the effect of increasing the UDP cache, setting the test environment to be the same as the environment of increasing the huge pages, namely: the source host sends multicast stream data to the virtual machine, and the virtual machine forwards the data to 30 multicast members after receiving the multicast stream data.
Setting the size of the virtual machine adding huge page as 2 million, adding UDP cache on the basis, testing the packet loss rate of each multicast stream data transmission, and calculating the average packet loss rate. As shown in table 2, it can be known from table 2 that the packet loss rate of multicast stream data transmission can be reduced by adding UDP buffer to the virtual machine under different bandwidths.
TABLE 2
Figure 254247DEST_PATH_IMAGE002
According to the embodiment of the invention, the UDP cache is added when the packet loss rate of the multicast stream data transmission is larger than the first packet loss rate threshold after the huge page is added, and the packet loss problem caused by less cache is reduced at the transmission layer, so that the packet loss rate of the multicast stream data transmission can be effectively reduced, and the multicast performance of the virtual machine can meet a certain requirement of low packet loss rate.
Further, the method further comprises:
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the UDP cache is added, and if the packet loss rate of multicast stream data transmission still exists and is larger than a first packet loss rate threshold value, adjusting the size of the UDP cache;
and stopping adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold, wherein the UDP buffer size is multiplied when the UDP buffer size is adjusted each time.
The embodiment of the invention continuously adjusts the size of the UDP cache according to the condition of the packet loss rate, thereby achieving the optimization of the UDP cache under the condition of limited resources and improving the overall utilization rate of the resources of the virtual machine.
Further, the method further comprises:
and adjusting the receiving and sending soft interrupts of the network card to balance the interrupt load of the network card.
The receiving and sending soft interrupts of the network card can be adjusted by adopting the existing load balancing method, so as to realize the load balancing of the network card interrupt.
The embodiment of the invention adopts soft interrupt load balancing in the virtual machine, disperses the data flow to the receiving queue of the multi-core for processing, and can further improve the network performance of the virtual machine.
In another embodiment, the multicast target further includes a second packet loss rate threshold, and the method further includes:
and monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than a second packet loss rate threshold value, adding the huge page and the UDP cache.
The first packet loss rate threshold is the same as the second packet loss rate threshold.
Wherein the original size value of the macro page is set according to a minimum size threshold. For example, if the minimum size threshold is set to 2 megabits, the size of the macro page is 2 megabits when the macro page is added. Wherein, the original value of the UDP buffer is set according to a minimum buffer threshold, and the minimum buffer threshold is preferably 128 bytes.
Further, the method further comprises:
after the huge page and the UDP cache are added, detecting the packet loss rate of each multicast stream data transmission, if the packet loss rate of one multicast stream data transmission is still larger than the second packet loss rate threshold value, adjusting the size of the huge page, and increasing the size of the huge page by multiple times when the size of the huge page is adjusted each time;
determining a UDP cache adjustment value according to the adjusted size of the huge page, and adjusting the size of the UDP cache to the UDP cache adjustment value;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
The huge page size adjustment list can be prestored in the virtual machine, and each huge page size in the huge page size adjustment list is associated with a UDP cache adjustment value, so that the UDP cache adjustment value can be determined according to the size of the adjusted huge page by querying the huge page size adjustment list.
According to the embodiment of the invention, the packet loss rate is reduced by simultaneously adjusting the giant pages and the UDP cache, so that the overall performance of the virtual machine can be optimized.
The embodiment of the second aspect of the invention provides a multicast performance optimization system in a virtual machine.
Fig. 2 is a schematic structural diagram of a preferred embodiment of the multicast performance optimization system in a virtual machine according to the present invention, where the system is capable of implementing all the processes of the multicast performance optimization method in a virtual machine according to any of the above embodiments.
As shown in fig. 2, the system includes:
the multicast forwarding system comprises a setting module 1, a forwarding module and a forwarding module, wherein the setting module is used for setting a multicast target, and the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded;
a sending module 2, configured to send multicast stream data corresponding to the multicast stream data amount to a virtual machine, so that the virtual machine performs data transmission to a multicast group member corresponding to the number of multicast stream data to be forwarded according to the multicast stream data;
and the network card virtual module 3 is used for virtualizing a network card to transmit to the virtual machine by adopting SR-IOV technology.
Further, the multicast target further comprises a multicast stream data transmission rate threshold, and the system further comprises a calculation module, wherein the calculation module is used for calculating the data transmission rate of each multicast stream data every preset time interval;
and the network card virtual module 3 virtualizes a network card to transmit to the virtual machine by adopting SR-IOV technology only when the minimum value of the data transmission rate is lower than the multicast stream data transmission rate threshold value.
According to a manner that can be realized by the second aspect of the present invention, the multicast target further includes a first packet loss rate threshold, and the system further includes:
the first monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
and the huge page increasing module is used for increasing the huge pages when the packet loss rate of the multicast stream data transmission is greater than the first packet loss rate threshold.
According to a manner that can be realized in the second aspect of the present invention, the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after adding the macro page, and the system further includes:
the first big page adjusting module is used for adjusting the size of the big page when the packet loss rate of one multicast stream data transmission is larger than the first packet loss rate threshold value, and the size of the big page is increased by multiple times when the size of the big page is adjusted each time;
and the first giant page adjusting module stops adjusting the size of the giant page when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold or the adjusted giant page size reaches a maximum memory value.
According to a manner that can be implemented in the second aspect of the present invention, the first macro page adjustment module includes a maximum memory value obtaining unit, where the maximum memory value obtaining unit is configured to obtain the maximum memory value according to a total bandwidth of the virtual machine, and the maximum memory value obtaining unit specifically includes:
the matching subunit is used for matching the total bandwidth size of the virtual machine with a preset standard bandwidth list, and each standard bandwidth in the standard bandwidth list is associated with a standard giant page size;
and the determining subunit is used for taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
According to an enabling manner of the second aspect of the invention, the system further comprises:
and the UDP cache increasing module is used for increasing the UDP cache when the packet loss rate of the multicast stream data transmission still exists and is greater than the first packet loss rate threshold when the adjusted huge page size reaches the maximum memory value.
According to an implementable manner of the second aspect of the present invention, the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after adding the UDP buffer, and the system further includes:
a first UDP buffer adjustment module, configured to adjust a size of the UDP buffer if a packet loss rate of a multicast stream data transmission still exceeds the first packet loss rate threshold;
the first UDP buffer adjustment module stops adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold, where the UDP buffer size is increased by multiple times when adjusting the UDP buffer size each time.
According to an enabling manner of the second aspect of the invention, the system further comprises:
and the load balancing module is used for adjusting the receiving and sending soft interrupts of the network card so as to balance the interrupt load of the network card.
According to a manner that can be realized by the second aspect of the present invention, the multicast target further includes a second packet loss rate threshold, and the system further includes:
the second monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
and the huge page and UDP cache increasing module is used for increasing the huge page and UDP cache when the packet loss rate of the multicast stream data transmission is greater than the second packet loss rate threshold.
According to a manner that can be realized in the second aspect of the present invention, the second monitoring module is further configured to detect a packet loss rate of each multicast stream data transmission after adding the megapage and the UDP buffer, and the system further includes:
the second macro page adjusting module is used for adjusting the size of the macro page if the packet loss rate of one multicast stream data transmission is larger than the second packet loss rate threshold, and the size of the macro page is increased by multiple times when the size of the macro page is adjusted each time;
the second UDP cache adjusting module is used for determining a UDP cache adjusting value according to the adjusted size of the huge page and adjusting the size of the UDP cache to the UDP cache adjusting value;
and the second huge page adjusting module stops adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
The functions and implementation manners of the modules in the above embodiment of the system of the present invention are the same as those in the above embodiment of the multicast performance optimization method in the virtual machine, and accordingly, the technical effects that can be achieved are also the same.
The present invention also provides an apparatus for optimizing multicast performance in a virtual machine, the apparatus including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor, when executing the computer program, implements a method for optimizing multicast performance in a virtual machine as described in any of the above embodiments.
The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the method for optimizing multicast performance in a virtual machine according to any of the above embodiments is implemented.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor is a control center of the multicast performance optimization apparatus in the virtual machine, and various interfaces and lines are used to connect various parts of the multicast performance optimization apparatus in the entire virtual machine.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the multicast performance optimization apparatus in the virtual machine by running or executing the computer programs and/or modules stored in the memory and calling data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
Wherein, the module/unit integrated by the multicast performance optimization device in the virtual machine can be stored in a computer readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
The foregoing is a preferred embodiment of the present application, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations are also regarded as the protection scope of the present application.

Claims (18)

1. A multicast performance optimization method in a virtual machine, the method comprising:
setting a multicast target, wherein the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded;
sending multicast stream data corresponding to the multicast stream data volume to a virtual machine, wherein the virtual machine carries out data transmission to multicast group members corresponding to the multicast stream data number to be forwarded according to the multicast stream data;
virtualizing a network card by adopting an SR-IOV technology and transmitting the network card to a virtual machine;
the multicast target further comprises a first packet loss rate threshold, and the method further comprises:
monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than the first packet loss rate threshold value, adding a huge page;
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the huge page is added, if the packet loss rate of one multicast stream data transmission is larger than the first packet loss rate threshold, adjusting the size of the huge page, and increasing the size of the huge page by multiple times when the size of the huge page is adjusted each time;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold or the adjusted huge page size reaches the maximum memory value.
2. The method according to claim 1, wherein the multicast target further includes a multicast stream data transmission rate threshold, and before the step of virtualizing a network card for transparent transmission to the virtual machine by using the SR-IOV technology is performed, the method further includes: calculating the data transmission rate of each multicast stream data every a preset time interval;
and only when the minimum value of the data transmission rate is lower than the threshold value of the multicast stream data transmission rate, the step of virtualizing a network card by adopting the SR-IOV technology and transmitting the network card to the virtual machine is executed.
3. The method according to claim 1, wherein the obtaining the maximum memory value according to the total bandwidth of the virtual machine specifically includes:
matching the total bandwidth of the virtual machine with a preset standard bandwidth list, wherein each standard bandwidth in the standard bandwidth list is associated with a standard macro page size;
and taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
4. The method of claim 3, wherein the method further comprises:
and if the packet loss rate of the multicast stream data transmission still exists when the adjusted huge page size reaches the maximum memory value, the UDP cache is added.
5. The method of claim 4, wherein the method further comprises:
monitoring the packet loss rate of multicast stream data transmission of the virtual machine after the UDP cache is added, and if the packet loss rate of multicast stream data transmission still exists and is larger than the first packet loss rate threshold value, adjusting the size of the UDP cache;
and stopping adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold, wherein the UDP buffer size is multiplied when the UDP buffer size is adjusted each time.
6. The method of claim 4, wherein the method further comprises:
and adjusting the receiving and sending soft interrupts of the network card to balance the interrupt load of the network card.
7. The method of claim 1, wherein the multicast target further comprises a second packet loss rate threshold, and wherein the method further comprises:
and monitoring the packet loss rate of each multicast stream data transmission, and if the packet loss rate of one multicast stream data transmission is greater than the second packet loss rate threshold value, adding a huge page and a UDP cache.
8. The method of claim 7, wherein the method further comprises:
after the huge page and the UDP cache are added, detecting the packet loss rate of each multicast stream data transmission, if the packet loss rate of one multicast stream data transmission is still larger than the second packet loss rate threshold value, adjusting the size of the huge page, and increasing the size of the huge page by multiple times when the size of the huge page is adjusted each time;
determining a UDP cache adjustment value according to the adjusted size of the huge page, and adjusting the size of the UDP cache to the UDP cache adjustment value;
and stopping adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
9. A system for optimizing multicast performance in a virtual machine, the system comprising:
the multicast target setting module is used for setting a multicast target, and the multicast target comprises multicast stream data quantity and the number of multicast stream data to be forwarded;
a sending module, configured to send multicast stream data corresponding to the multicast stream data amount to a virtual machine, so that the virtual machine performs data transmission to a multicast group member corresponding to the number of multicast stream data to be forwarded according to the multicast stream data;
the network card virtual module is used for virtualizing a network card transparent transmission to the virtual machine by adopting an SR-IOV technology;
the multicast target further includes a first packet loss rate threshold, and the system further includes:
the first monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
the huge page increasing module is used for increasing the huge pages when the packet loss rate of multicast stream data transmission is greater than the first packet loss rate threshold;
the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after the macro page is added, and the system further includes:
the first big page adjusting module is used for adjusting the size of the big page when the packet loss rate of one multicast stream data transmission is larger than the first packet loss rate threshold value, and the size of the big page is increased by multiple times when the size of the big page is adjusted each time;
and the first giant page adjusting module stops adjusting the size of the giant page when the packet loss rate of each multicast stream data transmission is not greater than a first packet loss rate threshold or the adjusted giant page size reaches a maximum memory value.
10. The system according to claim 9, wherein the multicast target further includes a multicast stream data transmission rate threshold, the system further includes a calculation module, and the calculation module is configured to calculate a data transmission rate of each multicast stream data every preset time period;
and the network card virtual module virtualizes a network card to transmit to the virtual machine by adopting SR-IOV technology only when the minimum value of the data transmission rate is lower than the multicast stream data transmission rate threshold value.
11. The system according to claim 9, wherein the first macro page adjusting module includes a maximum memory value obtaining unit, the maximum memory value obtaining unit is configured to obtain the maximum memory value according to a total bandwidth of the virtual machine, and the maximum memory value obtaining unit specifically includes:
the matching subunit is used for matching the total bandwidth size of the virtual machine with a preset standard bandwidth list, and each standard bandwidth in the standard bandwidth list is associated with a standard giant page size;
and the determining subunit is used for taking the size of the standard macro page associated with the matched standard bandwidth as the maximum memory value.
12. The system of claim 9, wherein the system further comprises:
and the UDP cache increasing module is used for increasing the UDP cache when the packet loss rate of the multicast stream data transmission still exists and is greater than the first packet loss rate threshold when the adjusted huge page size reaches the maximum memory value.
13. The system according to claim 12, wherein the first monitoring module is further configured to monitor a packet loss rate of multicast stream data transmission of the virtual machine after adding UDP buffer, and the system further includes:
a first UDP buffer adjustment module, configured to adjust a size of the UDP buffer if a packet loss rate of a multicast stream data transmission still exceeds the first packet loss rate threshold;
the first UDP buffer adjustment module stops adjusting the size of the UDP buffer when the packet loss rate of each multicast stream data transmission is not greater than the first packet loss rate threshold, where the UDP buffer size is increased by multiple times when adjusting the UDP buffer size each time.
14. The system of claim 12, wherein the system further comprises:
and the load balancing module is used for adjusting the receiving and sending soft interrupts of the network card so as to balance the interrupt load of the network card.
15. The system of claim 9, wherein the multicast target further comprises a second packet loss rate threshold, and wherein the system further comprises:
the second monitoring module is used for monitoring the packet loss rate of each multicast stream data transmission;
and the huge page and UDP cache increasing module is used for increasing the huge page and UDP cache when the packet loss rate of the multicast stream data transmission is greater than the second packet loss rate threshold.
16. The system of claim 15, wherein the second monitoring module is further configured to detect a packet loss rate of each multicast stream data transmission after adding the megapages and the UDP buffer, and the system further comprises:
the second macro page adjusting module is used for adjusting the size of the macro page if the packet loss rate of one multicast stream data transmission is larger than the second packet loss rate threshold, and the size of the macro page is increased by multiple times when the size of the macro page is adjusted each time;
the second UDP cache adjusting module is used for determining a UDP cache adjusting value according to the adjusted size of the huge page and adjusting the size of the UDP cache to the UDP cache adjusting value;
and the second huge page adjusting module stops adjusting the size of the huge page when the packet loss rate of each multicast stream data transmission is not greater than the second packet loss rate threshold.
17. An apparatus for multicast performance optimization in a virtual machine, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing a method for multicast performance optimization in a virtual machine according to any one of claims 1 to 8 when executing the computer program.
18. A computer-readable storage medium, in which a computer program is stored, which, when executed, implements a multicast performance optimization method in a virtual machine according to any one of claims 1 to 8.
CN202110764857.0A 2021-07-07 2021-07-07 Multicast performance optimization method, system, device and storage medium in virtual machine Active CN113259142B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110764857.0A CN113259142B (en) 2021-07-07 2021-07-07 Multicast performance optimization method, system, device and storage medium in virtual machine

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110764857.0A CN113259142B (en) 2021-07-07 2021-07-07 Multicast performance optimization method, system, device and storage medium in virtual machine

Publications (2)

Publication Number Publication Date
CN113259142A CN113259142A (en) 2021-08-13
CN113259142B true CN113259142B (en) 2021-10-12

Family

ID=77190946

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110764857.0A Active CN113259142B (en) 2021-07-07 2021-07-07 Multicast performance optimization method, system, device and storage medium in virtual machine

Country Status (1)

Country Link
CN (1) CN113259142B (en)

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101459618B (en) * 2009-01-06 2011-01-19 北京航空航天大学 Data packet forwarding method and device for virtual machine network
US9606823B2 (en) * 2013-12-09 2017-03-28 Vmware, Inc. Adaptive interrupt and system call coalescing
US10320921B2 (en) * 2014-12-17 2019-06-11 Vmware, Inc. Specializing virtual network device processing to bypass forwarding elements for high packet rate applications
CN105119724A (en) * 2015-07-20 2015-12-02 山东超越数控电子有限公司 Network virtualization implementation method based on Loongson platform
CN105868000B (en) * 2016-06-14 2019-02-19 上海交通大学 A kind of expansible data processing method of parallelization for network I/O virtualization
CN107222349B (en) * 2017-06-23 2021-09-10 北京三快在线科技有限公司 Method and device for configuring network card rate of virtual machine, computer equipment and storage medium
CN110113184B (en) * 2019-04-17 2021-10-26 杭州中科先进技术研究院有限公司 KVM virtual machine network optimization method and device under SR-IOV environment
CN110798412B (en) * 2019-10-18 2023-05-23 北京浪潮数据技术有限公司 Multicast service processing method and device, cloud platform, equipment and readable storage medium
CN113037582A (en) * 2019-12-25 2021-06-25 中国电信股份有限公司 Network Function Virtualization (NFV) test method, device and system
CN112468365A (en) * 2020-11-26 2021-03-09 上海阅维科技股份有限公司 Data quality detection method, system and medium for network mirror flow

Also Published As

Publication number Publication date
CN113259142A (en) 2021-08-13

Similar Documents

Publication Publication Date Title
US20220197685A1 (en) Technologies for application-specific network acceleration with unified coherency domain
US11652752B2 (en) Congestion control method and network device
EP2630579B1 (en) Unified i/o adapter
US20140032796A1 (en) Input/output processing
US9864717B2 (en) Input/output processing
US20200067826A1 (en) Method for link aggregation and related products
CN113411262B (en) Method and device for setting large-scale receiving and unloading functions
CN107094119B (en) A kind of control method for equalizing load and system based on cloud computing and SDN network
WO2023165518A1 (en) Interrupt message processing method and apparatus, electronic device, and readable storage medium
Musleh et al. Bridging the virtualization performance gap for HPC using SR-IOV for InfiniBand
CN114553780A (en) Load balancing method and device and network card
WO2023221847A1 (en) Data access method based on direct communication of virtual machine device, and device and system
Agarwal et al. Understanding host interconnect congestion
US20160196222A1 (en) Systems and methods for network i/o based interrupt steering
CN113259142B (en) Multicast performance optimization method, system, device and storage medium in virtual machine
WO2012141695A1 (en) Input/output processing
CN106790162B (en) Virtual network optimization method and system
WO2023155815A1 (en) Packet processing method, apparatus, terminal device, and storage medium
US20220179805A1 (en) Adaptive pipeline selection for accelerating memory copy operations
Tang et al. Towards high-performance packet processing on commodity multi-cores: current issues and future directions
JP7366240B2 (en) Message processing methods, devices and computer storage media
US9606823B2 (en) Adaptive interrupt and system call coalescing
CN110336759A (en) Protocol message transfer method and device based on RDMA
KR102145183B1 (en) Device and Method for Data Transmission and QoS Guarantee of Virtual Machines in Multicore-based Network Interface Card
WO2024008197A1 (en) Bandwidth adjustment method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant