CN115348208B - Flow control method and device, electronic equipment and storage medium - Google Patents
Flow control method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115348208B CN115348208B CN202110462209.XA CN202110462209A CN115348208B CN 115348208 B CN115348208 B CN 115348208B CN 202110462209 A CN202110462209 A CN 202110462209A CN 115348208 B CN115348208 B CN 115348208B
- Authority
- CN
- China
- Prior art keywords
- gateway
- directory
- flow
- real
- limit value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000004590 computer program Methods 0.000 claims description 15
- 230000004044 response Effects 0.000 claims description 5
- 241001475178 Dira Species 0.000 description 25
- LXMSZDCAJNLERA-ZHYRCANASA-N spironolactone Chemical compound C([C@@H]1[C@]2(C)CC[C@@H]3[C@@]4(C)CCC(=O)C=C4C[C@H]([C@@H]13)SC(=O)C)C[C@@]21CCC(=O)O1 LXMSZDCAJNLERA-ZHYRCANASA-N 0.000 description 25
- 238000010586 diagram Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- KLDZYURQCUYZBL-UHFFFAOYSA-N 2-[3-[(2-hydroxyphenyl)methylideneamino]propyliminomethyl]phenol Chemical compound OC1=CC=CC=C1C=NCCCN=CC1=CC=CC=C1O KLDZYURQCUYZBL-UHFFFAOYSA-N 0.000 description 3
- 201000001098 delayed sleep phase syndrome Diseases 0.000 description 3
- 208000033921 delayed sleep phase type circadian rhythm sleep disease Diseases 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a flow control method, a flow control device, an electronic device and a computer storage medium, wherein the flow control method is applied to a first flow control component of a distributed file storage system, and comprises the following steps: after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway; determining the flow limit value of each gateway to each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory; and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.
Description
Technical Field
The present invention relates to the field of computer application technologies, and in particular, to a flow control method, a flow control device, an electronic device, and a computer storage medium.
Background
International data corporation (International Data Corporation, IDC) predicts that global data volume is coming out, growing from 33ZB in 2018 to 175ZB in 2025. By 2021, the amount of data stored globally on the public Cloud would exceed that of a traditional data center, which would be Cloud broken to hold enterprise data since the last 60 th century, and enterprises would enter the All in clouds (All clouds) era. The cloud infrastructure (namely cloud storage) is divided into three types of file storage, object storage and block storage according to the storage mode, arrangement and data presentation format of data; according to the management mode of metadata, the metadata can be classified into a symmetrical type and an asymmetrical type. The roles of each node in the symmetrical distributed file storage system are equal, metadata is managed and maintained together, and information synchronization, mutual exclusion locking and other operations are performed among the nodes through a high-speed network. Fig. 1a is a schematic structural diagram of a symmetric distributed file storage system in the related art, as shown in fig. 1a, where the symmetric distributed file storage system is formed by a plurality of peer storage nodes, each storage node includes a plurality of storage buckets, and each storage bucket is generally abutted to a local file system for storing file data and metadata information of a client. Each storage cluster belongs to the same namespace, which provides a common file interface protocol for clients, such as the network file system (Network File System, NFS) protocol, the common network file system (Common Internet File System, CIFS) protocol, and so on.
In the related art, for a symmetric distributed file storage system, metadata of the system is scattered on all storage nodes in the system, and generally, the flow of a certain gateway can be limited only; that is, the current limiting effect cannot be achieved in the case that a single tenant uses multiple gateways, so that in the actual use process, it is required to ensure that the domain name system (Domain Name System, DNS) returns the same gateway internet protocol (Internet Protocol, IP) to the same tenant.
It can be seen that, in order to precisely limit tenant traffic, each tenant can only use one gateway; however, limited by the network bandwidth of the single gateway and the capabilities of the central processor (Central Processing Unit, CPU), resulting in low single tenant resource usage; particularly, under the condition that the tenant is in a large bandwidth, the flow control method cannot meet the requirements on the cloud; thus, there is a need to provide a method that enables flow control in a single tenant multi-gateway scenario.
Disclosure of Invention
The invention provides a flow control method, a flow control device, electronic equipment and a computer storage medium; the problem that the symmetrical distributed file storage system cannot control flow under the condition of single tenant multi-gateway can be solved.
The technical scheme of the invention is realized as follows:
the invention provides a flow control method which is applied to a first flow control component of a distributed file storage system, and comprises the following steps:
after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway;
determining the flow limit value of each gateway to each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory;
and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.
In some embodiments, the determining the flow limit value of each gateway for each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory includes:
determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway;
and when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway.
In some embodiments, the distributed file storage system further comprises a second flow control component, the method further comprising:
establishing a connection with the respective gateway using the second flow control component when the first flow control component is out of service; and migrating services provided by the first flow control component to the second flow control component.
In some embodiments, the method further comprises:
and when the plurality of flow control components of the distributed file storage system are out of service and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, sending the preset flow limit value of each directory to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.
In some embodiments, the method further comprises:
when the virtual IP corresponding to the first gateway is determined to be transferred to the second gateway, the real-time flow of each catalogue reported by the rest gateways is received again; the remaining gateways represent other gateways than the first gateway.
In some embodiments, the receiving the real-time traffic of each directory reported by the respective gateway includes:
and periodically receiving the real-time traffic of each catalogue reported by each gateway according to a set interval.
The invention provides a flow control device which is applied to a first flow control component of a distributed file storage system, and comprises a receiving module, a determining module and a control module, wherein,
the receiving module is used for receiving the real-time flow of each catalogue reported by each gateway after the first flow control assembly establishes connection with each gateway;
the determining module is used for determining the flow limit value of each gateway to each catalogue according to the real-time flow of each catalogue reported by each gateway and the preset flow limit value of each catalogue;
and the control module is used for sending the flow limit value of each gateway to each directory to each gateway so that each gateway controls the flow of each directory based on the flow limit value of each directory.
The invention provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the flow control method provided by one or more of the technical schemes when executing the program.
The present invention provides a computer storage medium storing a computer program; the computer program, when executed, is capable of implementing the flow control method provided by one or more of the foregoing aspects.
The invention provides a flow control method, a flow control device, electronic equipment and a computer storage medium, wherein the flow control method is applied to a first flow control component of a distributed file storage system, and comprises the following steps: after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway; determining the flow limit value of each gateway to each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory; the flow limit value of each gateway to each directory is sent to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory; therefore, the flow control component is introduced into the distributed file storage system, and calculates the flow limit value of each gateway to each directory according to the received real-time flow of each directory reported by each gateway and the preset flow limit value; further, according to the flow limit value, flow control for each directory is realized; because each directory corresponds to one tenant, the flow control of the distributed file storage system under the condition of single tenant and multiple gateways can be realized; further, because the introduced flow control assembly is extremely lightweight, the design of the distributed file storage system software is greatly simplified while flow control is achieved.
Drawings
FIG. 1a is a schematic diagram of a symmetric distributed file storage system according to the related art;
FIG. 1b is a flow chart of a flow control method according to an embodiment of the present invention;
FIG. 2a is a schematic diagram illustrating a flow control method in a distributed file storage system according to an embodiment of the present invention;
fig. 2b is a schematic diagram of a format of a real-time traffic reported by a gateway according to an embodiment of the present invention;
fig. 2c is a schematic diagram two of a real-time traffic format reported by a gateway according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a flow control device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings.
The present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the examples provided herein are for the purpose of illustration only and are not intended to limit the invention. The following examples are provided for carrying out the present invention, but not for all examples, and the embodiments described in the present invention can be carried out in any combination without any conflict.
It should be noted that, in the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a method or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such method or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other related elements in a method or apparatus comprising the element (e.g., a step in a method or an element in an apparatus, e.g., an element may be part of a processor, part of a program or software, etc.).
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
For example, the flow control method provided by the present invention includes a series of steps, but the flow control method provided by the present invention is not limited to the steps described above, and similarly, the flow control device provided by the present invention includes a series of modules, but the flow control device provided by the present invention is not limited to the modules explicitly described, and may include modules that are required to be provided for acquiring related information or performing processing based on the information.
The present invention may be implemented on the basis of an electronic device, which may be a thin client, thick client, handheld or laptop device, microprocessor-based system, set top box, programmable consumer electronics, network personal computer, small computer system, or the like.
An electronic device may be described in the general context of computer-system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform particular tasks or implement particular abstract data types. The computer system may be implemented in a distributed cloud computing environment in which tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computing system storage media including memory storage devices.
Under a public cloud scene, the file storage of the tenant application corresponds to one directory in the underlying distributed file storage system, and a plurality of tenants share the whole flow of the same storage system. If not controlled, the flow between tenants can be mutually influenced; it can be seen that flow control has become an integral function of file storage.
In some embodiments of the invention, the flow control method may be implemented using a processor in the flow control device, which may be at least one of an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a CPU, a controller, a microcontroller, a microprocessor.
In the embodiment of the invention, the flow control method can be applied to a symmetrical distributed file storage system.
Fig. 1b is a flow chart of a flow control method according to an embodiment of the present invention, as shown in fig. 1b, the flow may include:
step 100: and after the first flow control component establishes connection with each gateway, receiving the real-time flow of each catalogue reported by each gateway.
Illustratively, the first flow control component may be a high availability service based on a Raft algorithm distributed across the various gateways; here, the Raft algorithm is mainly applied to a distributed cluster system, and can ensure high availability and data consistency; the first traffic control component is designed to be very lightweight, which can be responsible for the management of tenant traffic on the various gateways.
In one embodiment, each gateway may represent a storage gateway deployed on a tenant local data center and public cloud; here, the storage gateway can establish a mapping relation between file data of the client stored in the storage bucket and a directory of the symmetric distributed file storage system, so as to realize backup, distribution and the like of the file data; that is, the storage gateway corresponds to a portal for file data storage, through which tenants can access the corresponding directory of the distributed file storage system using NFS protocol or CIFS protocol.
For example, for an implementation in which the first flow control component establishes a connection with each gateway, after the first flow control component is started, a connection may be established with each gateway according to a port number of each gateway; and after the first flow control component is started, each gateway can be connected with the first flow control component according to the port number of the first flow control component.
In the embodiment of the invention, after the first flow control component is determined to establish connection with each gateway, the first flow control component receives the real-time flow of each catalogue reported by each gateway; for example, in the case where the gateway 1 and the gateway 2 exist in the symmetric distributed file storage system, assuming that the real-time traffic of the directory a on the gateway 1 is 5M and the real-time traffic of the directory a on the gateway 2 is 10M, the first traffic control component receives the real-time traffic 5M of the directory a reported by the gateway 1 and the real-time traffic 10M of the directory a reported by the gateway 2.
In the embodiment of the invention, the first flow control component and the symmetrical distributed file storage system are decoupled from each other, so that the tenant flow management can be facilitated; the maintenance cost of the symmetrical distributed file storage system can be reduced, and the stability of the system is improved.
In some embodiments, receiving the real-time traffic of each directory reported by the respective gateway may include: and periodically receiving the real-time traffic of each catalogue reported by each gateway according to the set interval.
In one embodiment, the setting interval may be set according to practical situations, for example, may be 1s,2s, etc., which is not limited in the embodiment of the present invention.
Illustratively, in the case where the time interval is 2S, the first flow control component receives real-time flow of each directory reported by the respective gateway every 2S.
In some embodiments, the distributed file storage system may further comprise a second flow control component, and the method may further comprise: when the first flow control component stops serving, a second flow control component is used for establishing connection with each gateway; and migrating services provided by the first flow control component to the second flow control component.
For example, after the first flow control component establishes a connection with each gateway, if the first flow control component stops servicing due to various abnormal conditions, the service provided by the first flow control component may be migrated to the second flow control component, i.e., immediately taken over by the second flow control component to provide the service; that is, a connection between the second flow control component and each gateway is established; the second flow control component then receives the real-time flow of each directory reported by the respective gateway.
In one embodiment, multiple flow control components may be introduced into the distributed file storage system, where only one flow control component need be selected at a time to service; when the flow control assembly stops serving, any other flow control assembly can be selected for serving; here, the individual gateways may not be aware of the switching of the flow control components.
In some embodiments, the method may further comprise: when the virtual IP corresponding to the first gateway is determined to be transferred to the second gateway, the real-time flow of each catalogue reported by the rest gateways is received again; the remaining gateways represent other gateways than the first gateway.
In one embodiment, each of the plurality of flow control components provides only one virtual IP for each gateway, such as downtime, after any gateway fails; the virtual IP corresponding to that gateway will be transferred to the other gateway.
Here, the first gateway and the second gateway represent two different gateways in the distributed file storage system; when the first gateway is determined to have a fault, transferring the virtual IP corresponding to the first gateway to other gateways except the first gateway, for example, a second gateway; in turn, the flow control component re-receives real-time traffic for each directory reported by other gateways than the first gateway.
Illustratively, in the case where the gateway 1, the gateway 2 and the gateway 3 exist in the symmetric distributed file storage system, it is assumed that the real-time traffic of the directory B on the gateway 1 is 2M, the real-time traffic of the directory B on the gateway 2 is 3M, and the real-time traffic of the directory B on the gateway 3 is 10M, and if the gateway 1 fails, the virtual IP corresponding to the gateway 1 may be transferred to the gateway 2 or may be transferred to the gateway 3.
Illustratively, in the case that the virtual IP corresponding to the gateway 1 is transferred to the gateway 2, the flow control component receives the real-time flow 5M of the directory B reported by the gateway 2 and the real-time flow 10M of the directory B reported by the gateway 3; in the case where the virtual IP corresponding to the gateway 1 is transferred to the gateway 3, the flow control component receives the real-time flow 3M of the directory B reported by the gateway 2 and the real-time flow 12M of the directory B reported by the gateway 3.
It can be seen that, in the embodiment of the present invention, when the above abnormal situation occurs in the symmetric distributed file storage system, that is, when a certain flow control component or gateway fails, control over the directory flow may also be implemented.
Step 101: and determining the flow limit value of each gateway to each directory according to the real-time flow of each directory and the preset flow limit value of each directory, which are reported by each gateway.
Here, each directory has a corresponding preset flow limit value; setting the preset flow limit value according to an actual application scene; the preset flow limit values of different catalogues can be the same or different; the embodiments of the present invention are not limited in this regard.
In one embodiment, a preset flow limit value for each directory may be preset in the first flow control component; when the first flow control component receives the real-time flow of each catalogue reported by each gateway, the flow limit value of each gateway to each catalogue can be determined according to the real-time flow of each catalogue reported by each gateway and the preset flow limit value of each catalogue.
In some embodiments, for the implementation of determining the flow limit value of each gateway for each directory according to the real-time flow of each directory and the preset flow limit value of each directory reported by each gateway, the implementation may include: determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway; when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway.
In one embodiment, taking a directory C as an example, describing a process of determining a flow limit value of each gateway to each directory by using a first flow control component, firstly determining a real-time flow sum of the directory C according to the real-time flow of the directory C reported by each gateway; then, judging whether the real-time flow sum of the catalogue C is larger than a preset flow limit value of the catalogue C. If not, the flow of the catalog C does not need to be subjected to any operation, and the preset flow limit value of each catalog is returned to each gateway. If yes, the flow of the catalog C is required to be limited; at this time, the first flow control component determines the flow limit value of each gateway to the directory C according to the product of the real-time flow proportion of the directory C reported by each gateway and the preset flow limit value of the directory C. Here, the flow limit values of the other directories may be determined by the above steps, which is not further described herein. Illustratively, the traffic limit value for each gateway for each directory can be calculated by equation (1):
here, dira represents one of the directories; node (i) represents an ith gateway, the value of i is 1 to n, and n represents the number of gateways in the symmetric distributed file storage system; req (dira, node (i)) represents the real-time traffic of the directory dira on gateway i;representing the real-time traffic sum of directory dira on n gateways; limit (dira) representationThe preset flow limit value of directory dira.
Illustratively, in the case where n is 2, i.e., where gateway 1 and gateway 2 are present in the symmetric distributed file storage system, the real-time traffic sum of directory dira can be determined assuming that the real-time traffic req (dira, node (1)) of directory dira on gateway 1 is 6M and the real-time traffic req (dira, node (2)) of directory dira on gateway 2 is 4M10M. Assuming that a preset flow limit value limit (dira) of the directory dira is 8M, determining that the real-time flow sum 10M of the directory dira is greater than the preset flow limit value 8M of the directory dira; at this time, the real-time flow rate ratio of the directory dira reported by the gateway 1 is determined to be 0.6, and the product of the real-time flow rate ratio of the directory dira reported by the gateway 1 of 0.6 and the preset flow limit value 8M of the directory dira is set to be 4.8M as the flow limit value resp (dira, node (1)) of the gateway 1 to the directory dira. Similarly, the real-time flow rate ratio of the directory dira reported by the gateway 2 is determined to be 0.4, and the product of the real-time flow rate ratio of the directory dira reported by the gateway 2 of 0.4 and the preset flow limit value of the directory dira 8M is 3.2M, which is taken as the flow limit value resp (dira, node (2)) of the gateway 2 to the directory dira.
In the embodiment of the invention, when the real-time flow sum of a certain directory in each gateway exceeds the preset flow limit value of the directory, the flow distribution is carried out on each gateway according to the proportion by taking the preset flow limit value as a reference, so that the accurate flow control can be realized.
Step 102: and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.
In one embodiment, after the first flow control component determines the flow limit value of each gateway for each directory, the preset flow limit value of each directory and the flow limit value of each gateway for each directory are returned to each gateway; at this time, the respective gateways may control the flow of each directory based on the flow limit value of each directory.
In some embodiments, the method may further comprise: when the plurality of flow control components of the distributed file storage system are out of service and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, the preset flow limit value of each directory is sent to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.
Illustratively, during communication between each of the plurality of flow control components and the respective gateway, the sender sends a message in a fixed format to the receiver according to a certain rule (e.g., periodic transmission, idle transmission, etc.), and the receiver replies to a message in a fixed format after receiving the message, and considers the current connection to be broken if a set time threshold is exceeded. Here, the sender may be a flow control component, or may be each gateway; in the case where the sender is a flow control component, the receiver is each gateway.
In one embodiment, the setting interval may be set according to practical situations, for example, may be 3 times, 5 times, or the like of the heartbeat period, which is not limited in the embodiment of the present invention.
In an exemplary embodiment, when the service of the plurality of flow control components of the distributed file storage system is stopped and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, after sending the preset flow limit value of each directory to each gateway, each gateway performs flow control on each directory based on the preset flow limit value of each directory, so as to ensure that the tenant on each gateway normally completes Input/Output (I/O) of the read/write directory.
In an embodiment of the present invention, the functions of the first control component may include: maintaining real-time traffic of each directory on each gateway and persisting a preset traffic limit value of each directory; maintaining heartbeats between each gateway and the first control component; and calculating the flow limit value of each gateway for each directory, and transmitting the flow limit value to each gateway to realize flow control.
The invention provides a flow control method, a flow control device, an electronic device and a computer storage medium, wherein the flow control method is applied to a first flow control component of a distributed file storage system, and comprises the following steps: after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway; determining the flow limit value of each gateway to each directory according to the real-time flow of each directory and the preset flow limit value of each directory, which are reported by each gateway; the flow limit value of each gateway to each directory is sent to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory; in this way, the flow control component is introduced into the distributed file storage system, and the flow control component calculates the flow limit value of each gateway to each directory according to the received real-time flow of each directory reported by each gateway and the preset flow limit value; further, according to the flow limit value, flow control for each directory is realized; because each directory corresponds to one tenant, the flow control of the distributed file storage system under the condition of single tenant and multiple gateways can be realized; further, because the introduced flow control assembly is extremely lightweight, the design of the distributed file storage system software is greatly simplified while flow control is achieved.
In order to further embody the object of the present invention, a flow control method in a distributed file storage system is described based on the above-described embodiments of the present invention; FIG. 2a is a schematic structural diagram of a flow control method in a distributed file storage system according to an embodiment of the present invention, as shown in FIG. 2a, where the structure includes: two tenants (tenant 1 and tenant 2), DNS, N gateways (gateway 1 to gateway N), three buckets (bucket 1, bucket 2, and bucket 3), and a first flow control component Qosctl; the first flow control component Qosctl establishes a connection with each of the N gateways.
Tenant 1 and tenant 2 respectively obtain the IP of N gateways from DNS, find the corresponding gateway 1 to gateway N according to the IP of N gateways; the gateways 1 to N acquire real-time traffic of each directory from three buckets. Here, the flow of the flow control method may include: after the first flow control component Qoscl is started, respectively establishing connection with N gateways according to the agreed ports; all gateways periodically report the real-time traffic of each directory to the first traffic control component Qosctl, the format of which is shown in fig. 2b and 2 c; fig. 2b is a schematic diagram of a format of a real-time traffic reported by a gateway according to an embodiment of the present invention; as can be seen, the gateway 1 reports the real-time traffic of the three directories, namely, directory a, directory b and directory c, to the first traffic control component Qosctl, wherein the real-time traffic of directory a, directory b and directory c is 1M, 2M and 3M respectively; fig. 2c is a schematic diagram two of a real-time traffic format reported by a gateway according to an embodiment of the present invention; it can be seen that the gateway N reports the real-time traffic of the three directories, namely, directory a, directory b and directory c, to the first traffic control component Qosctl, wherein the real-time traffic of directory a, directory b and directory c is 0M, 1M and 7M, respectively.
The first flow control component Qosctl calculates flow limit values to be set by each gateway for the three directories according to real-time flows of the directory a, the directory b and the directory c reported by each gateway and corresponding flow limit values. The calculation rule is as follows:
if the real-time flow sum of each of the three directories does not exceed the preset flow limit value of the corresponding directory, no operation is performed, and the preset flow limit value of each directory recorded by the first flow control component Qosctl is only returned to each gateway; if the sum of the real-time flow of a certain directory in the three directories exceeds the preset flow limit value of the corresponding directory, determining the flow limit value of each gateway to the directory according to the product of the real-time flow proportion of the directory reported by each gateway and the preset flow limit value of the corresponding directory; and after the flow limit value is determined, returning the preset flow limit value and the flow limit value of the catalog recorded by the first flow control component Qosctl to each gateway. And after each gateway receives the flow limit value of the directory, the flow value of the corresponding directory is updated immediately.
It can be seen that the embodiment of the present invention provides, through the first flow control component Qosctl, an implementation manner of directory flow control highly decoupled from the symmetric distributed file storage system itself; the method realizes the control of file storage flow of the symmetrical distributed file storage system under the condition of single tenant multi-gateway.
FIG. 3 is a schematic diagram of a flow control device according to an embodiment of the present invention, where the flow control device is applied to a first flow control component of a distributed file storage system, and as shown in FIG. 3, the flow control device includes: a receiving module 300, a determining module 301 and a control module 302, wherein:
a receiving module 300, configured to receive real-time traffic of each directory reported by each gateway after the first traffic control component establishes a connection with each gateway;
a determining module 301, configured to determine a flow limit value of each gateway for each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory;
and the control module 302 is configured to send the flow limit value of each gateway for each directory to the respective gateway, so that the respective gateway controls the flow of each directory based on the flow limit value of each directory.
In some embodiments, the determining module 301 is configured to determine, according to the real-time traffic of each directory reported by the respective gateway and the preset traffic limit value of each directory, the traffic limit value of the respective gateway for each directory, including:
determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway;
and when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway.
In some embodiments, the control module 302 is further configured to:
and when the plurality of flow control components of the distributed file storage system are out of service and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, sending the preset flow limit value of each directory to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.
In some embodiments, the receiving module 300 is further configured to:
when the virtual IP corresponding to the first gateway is determined to be transferred to the second gateway, the real-time flow of each catalogue reported by the rest gateways is received again; the remaining gateways represent other gateways than the first gateway.
In some embodiments, the receiving module 300 is further configured to receive real-time traffic of each directory reported by the respective gateway, including:
and periodically receiving the real-time traffic of each catalogue reported by each gateway according to a set interval.
In practical applications, the receiving module 300, the determining module 301 and the controlling module 302 may be implemented by a processor located in an electronic device, where the processor may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, a controller, a microcontroller and a microprocessor.
In addition, each functional module in the present embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.
The integrated units, if implemented in the form of software functional modules, may be stored in a computer-readable storage medium, if not sold or used as separate products, and based on such understanding, the technical solution of the present embodiment may be embodied essentially or partly in the form of a software product, which is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or processor (processor) to perform all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk or an optical disk, or the like, which can store program codes.
Specifically, the computer program instructions corresponding to one flow control method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disc, or a usb disc, and when the computer program instructions corresponding to one flow control method in the storage medium are read or executed by an electronic device, any one of the flow control methods in the foregoing embodiments is implemented.
Based on the same technical concept as the foregoing embodiments, referring to fig. 4, an electronic device 400 provided by an embodiment of the present invention may include: a memory 401 and a processor 402; wherein,
a memory 401 for storing computer programs and data;
a processor 402 for executing a computer program stored in a memory to implement any of the flow control methods of the foregoing embodiments.
In practical applications, the memory 401 may be a volatile memory (RAM); or a non-volatile memory (non-volatile memory), such as ROM, flash memory (flash memory), hard Disk (HDD), or Solid State Drive (SSD); or a combination of the above types of memory and provides instructions and data to the processor 402.
The processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, a controller, a microcontroller, and a microprocessor. It will be appreciated that the electronics for implementing the processor functions described above may be other for different flow control devices, and embodiments of the present invention are not specifically limited.
In some embodiments, the functions or modules included in the apparatus provided by the embodiments of the present invention may be used to perform the methods described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.
The methods disclosed in the method embodiments provided by the invention can be arbitrarily combined under the condition of no conflict to obtain a new method embodiment.
The features disclosed in the embodiments of the products provided by the invention can be combined arbitrarily under the condition of no conflict to obtain new embodiments of the products.
The features disclosed in the embodiments of the method or the device provided by the invention can be arbitrarily combined under the condition of no conflict to obtain a new embodiment of the method or the device.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (8)
1. A flow control method for use in a first flow control component of a distributed file storage system, the method comprising:
after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway;
determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway;
when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the product of the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway; each directory corresponds to one tenant; the preset flow limit value is set according to an actual application scene; the real-time flow ratio of the catalogue represents the ratio of the real-time flow of the catalogue to the sum of the real-time flows of the catalogue reported by each gateway;
and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.
2. The method of claim 1, wherein the distributed file storage system further comprises a second flow control component, the method further comprising:
establishing a connection with the respective gateway using the second flow control component when the first flow control component is out of service; and migrating services provided by the first flow control component to the second flow control component.
3. The method according to claim 1, wherein the method further comprises:
and when the plurality of flow control components of the distributed file storage system are out of service and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, sending the preset flow limit value of each directory to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.
4. The method according to claim 1, wherein the method further comprises:
when the virtual Internet Protocol (IP) corresponding to the first gateway is determined to be transferred to the second gateway, the real-time flow of each directory reported by the rest gateways is received again; the remaining gateways represent other gateways than the first gateway.
5. The method of claim 1, wherein receiving real-time traffic for each directory reported by the respective gateway comprises:
and periodically receiving the real-time traffic of each catalogue reported by each gateway according to a set interval.
6. A flow control apparatus for use in a first flow control component of a distributed file storage system, the apparatus comprising:
the receiving module is used for receiving the real-time flow of each catalogue reported by each gateway after the first flow control assembly establishes connection with each gateway;
the determining module is used for determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway; when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the product of the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway; each directory corresponds to one tenant; the preset flow limit value is set according to an actual application scene; the real-time flow ratio of the catalogue represents the ratio of the real-time flow of the catalogue to the sum of the real-time flows of the catalogue reported by each gateway;
and the control module is used for sending the flow limit value of each gateway to each directory to each gateway so that each gateway controls the flow of each directory based on the flow limit value of each directory.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 5 when the program is executed.
8. A computer storage medium having stored thereon a computer program, which when executed by a processor implements the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110462209.XA CN115348208B (en) | 2021-04-27 | 2021-04-27 | Flow control method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110462209.XA CN115348208B (en) | 2021-04-27 | 2021-04-27 | Flow control method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115348208A CN115348208A (en) | 2022-11-15 |
| CN115348208B true CN115348208B (en) | 2024-04-09 |
Family
ID=83947223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110462209.XA Active CN115348208B (en) | 2021-04-27 | 2021-04-27 | Flow control method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115348208B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008086720A1 (en) * | 2006-12-13 | 2008-07-24 | Huawei Technologies Co., Ltd. | A method, equipment and system for path selection and flow control |
| EP2874376A1 (en) * | 2013-11-15 | 2015-05-20 | Huawei Technologies Co., Ltd. | Method and system for information synchronization between cloud storage gateways, and cloud storage gateway |
| WO2015077756A1 (en) * | 2013-11-25 | 2015-05-28 | Amazon Technologies, Inc. | Customer-directed networking limits in distributed systems |
| CN105827523A (en) * | 2016-06-03 | 2016-08-03 | 无锡华云数据技术服务有限公司 | Virtual gateway capable of dynamically adjusting bandwidths of multiple tenants in cloud storage environment |
| CN106059915A (en) * | 2016-07-20 | 2016-10-26 | 赛特斯信息科技股份有限公司 | System and method for implementing limitation of north-south traffic of tenants based on SDN controller |
| WO2017050166A1 (en) * | 2015-09-24 | 2017-03-30 | 阿里巴巴集团控股有限公司 | Method, apparatus and system for controlling service transmission rate |
| CN106921584A (en) * | 2017-03-31 | 2017-07-04 | 武汉绿色网络信息服务有限责任公司 | A kind of distributed network flow control method |
| WO2019232681A1 (en) * | 2018-06-04 | 2019-12-12 | Beijing Thomson Commerce Co., Ltd. | Method for providing services and corresponding gateway, storage medium and computer program product |
| CN110830384A (en) * | 2019-09-30 | 2020-02-21 | 浙江口碑网络技术有限公司 | Method, device and system for limiting service flow |
| KR20200072128A (en) * | 2018-12-12 | 2020-06-22 | 네이버 주식회사 | Distributed file system and file managing method for live service |
| CN112333112A (en) * | 2020-10-27 | 2021-02-05 | 华云数据控股集团有限公司 | Method and system for realizing shared bandwidth |
| CN112350918A (en) * | 2020-12-10 | 2021-02-09 | 武汉绿色网络信息服务有限责任公司 | Service traffic scheduling method, device, equipment and storage medium |
| CN112437018A (en) * | 2020-11-19 | 2021-03-02 | 百度在线网络技术（北京）有限公司 | Flow control method, device, equipment and storage medium for distributed cluster |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070081543A1 (en) * | 2005-10-11 | 2007-04-12 | Manrique Brenes | Network utilization control apparatus and method of using |
| US9015343B2 (en) * | 2010-11-23 | 2015-04-21 | Centurylink Intellectual Property Llc | User control over content delivery |
| US10721098B2 (en) * | 2015-08-28 | 2020-07-21 | Vmware, Inc. | Optimizing connectivity between data centers in a hybrid cloud computing system |
| KR20170139872A (en) * | 2016-06-10 | 2017-12-20 | 삼성에스디에스 주식회사 | Multi-tenant based system and method for providing services |
| US10931640B2 (en) * | 2018-06-22 | 2021-02-23 | International Business Machines Corporation | Tunneling network traffic using object storage |
| US11240160B2 (en) * | 2018-12-28 | 2022-02-01 | Alibaba Group Holding Limited | Method, apparatus, and computer-readable storage medium for network control |
-
2021
- 2021-04-27 CN CN202110462209.XA patent/CN115348208B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008086720A1 (en) * | 2006-12-13 | 2008-07-24 | Huawei Technologies Co., Ltd. | A method, equipment and system for path selection and flow control |
| EP2874376A1 (en) * | 2013-11-15 | 2015-05-20 | Huawei Technologies Co., Ltd. | Method and system for information synchronization between cloud storage gateways, and cloud storage gateway |
| WO2015077756A1 (en) * | 2013-11-25 | 2015-05-28 | Amazon Technologies, Inc. | Customer-directed networking limits in distributed systems |
| WO2017050166A1 (en) * | 2015-09-24 | 2017-03-30 | 阿里巴巴集团控股有限公司 | Method, apparatus and system for controlling service transmission rate |
| CN106559349A (en) * | 2015-09-24 | 2017-04-05 | 阿里巴巴集团控股有限公司 | The control method and device of service transmission rate, system |
| CN105827523A (en) * | 2016-06-03 | 2016-08-03 | 无锡华云数据技术服务有限公司 | Virtual gateway capable of dynamically adjusting bandwidths of multiple tenants in cloud storage environment |
| CN106059915A (en) * | 2016-07-20 | 2016-10-26 | 赛特斯信息科技股份有限公司 | System and method for implementing limitation of north-south traffic of tenants based on SDN controller |
| CN106921584A (en) * | 2017-03-31 | 2017-07-04 | 武汉绿色网络信息服务有限责任公司 | A kind of distributed network flow control method |
| WO2019232681A1 (en) * | 2018-06-04 | 2019-12-12 | Beijing Thomson Commerce Co., Ltd. | Method for providing services and corresponding gateway, storage medium and computer program product |
| KR20200072128A (en) * | 2018-12-12 | 2020-06-22 | 네이버 주식회사 | Distributed file system and file managing method for live service |
| CN110830384A (en) * | 2019-09-30 | 2020-02-21 | 浙江口碑网络技术有限公司 | Method, device and system for limiting service flow |
| CN112333112A (en) * | 2020-10-27 | 2021-02-05 | 华云数据控股集团有限公司 | Method and system for realizing shared bandwidth |
| CN112437018A (en) * | 2020-11-19 | 2021-03-02 | 百度在线网络技术（北京）有限公司 | Flow control method, device, equipment and storage medium for distributed cluster |
| CN112350918A (en) * | 2020-12-10 | 2021-02-09 | 武汉绿色网络信息服务有限责任公司 | Service traffic scheduling method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于网关的应用软件流量监控系统设计;邹连英;李飞;王美珍;;计算机安全(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115348208A (en) | 2022-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9912538B2 (en) | Synchronization of configuration file of virtual application distribution chassis | |
| US9992274B2 (en) | Parallel I/O write processing for use in clustered file systems having cache storage | |
| US9830240B2 (en) | Smart storage recovery in a distributed storage system | |
| US9330156B2 (en) | System and method for software defined network aware data replication | |
| US10657108B2 (en) | Parallel I/O read processing for use in clustered file systems having cache storage | |
| WO2018076765A1 (en) | Content distribution method and device for cloud computing system, computing node and system | |
| US20190014161A1 (en) | Downloading of server-based content through peer-to-peer networks | |
| US9830091B2 (en) | Policy-based data tiering using a cloud architecture | |
| CN110266822B (en) | Shared load balancing implementation method based on nginx | |
| US11593496B2 (en) | Decentralized data protection system for multi-cloud computing environment | |
| US11461053B2 (en) | Data storage system with separate interfaces for bulk data ingestion and data access | |
| Chang et al. | Write-aware replica placement for cloud computing | |
| JP2006221423A (en) | Data distribution method and device and storage medium | |
| CN116418876A (en) | Migration method and system of computing power network service and cloud management platform | |
| Safrianti | Peer Connection Classifier Method for Load Balancing Technique | |
| CN113766013A (en) | Session creation method, device, equipment and storage medium | |
| CN115348208B (en) | Flow control method and device, electronic equipment and storage medium | |
| US20240015135A1 (en) | Domain management and synchronization system | |
| US11595471B1 (en) | Method and system for electing a master in a cloud based distributed system using a serverless framework | |
| JP5894981B2 (en) | Accessing a network of nodes distributed across a communication architecture using a topology server with multiple criteria selection | |
| US8145781B2 (en) | Data distribution system | |
| US10938728B2 (en) | High performance for efficient auto-scaling of stateful service | |
| US9960957B2 (en) | Methods for prioritizing failover of logical interfaces (LIFs) during a node outage and devices thereof | |
| US11481359B2 (en) | Parallel distributed ledger construction | |
| CN110636091A (en) | Data balancing method, device, equipment and storage medium for cloud storage cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |