CN108234188B - Service platform resource scheduling processing method and device - Google Patents
Service platform resource scheduling processing method and device Download PDFInfo
- Publication number
- CN108234188B CN108234188B CN201611198113.2A CN201611198113A CN108234188B CN 108234188 B CN108234188 B CN 108234188B CN 201611198113 A CN201611198113 A CN 201611198113A CN 108234188 B CN108234188 B CN 108234188B
- Authority
- CN
- China
- Prior art keywords
- resource
- resources
- isolated
- fault
- recovered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0659—Management of faults, events, alarms or notifications using network fault recovery by isolating or reconfiguring faulty entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a service platform resource scheduling processing method and device. Wherein the method comprises the following steps: acquiring working index information of each resource in a service platform through a fault acquisition interface, and acquiring a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period; acquiring a data flow table through a data switch, wherein the data flow table comprises resources in a normal working state in the last period; and determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule. The device is used for executing the method. The service platform resource scheduling processing method and the service platform resource scheduling processing device avoid using fault resources in the resource scheduling process, thereby improving the efficiency of resource scheduling.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a service platform resource scheduling processing method and device.
Background
In the internet + era, wireless communication service functions continue to be continuously enhanced and improved, and industrial short message services are developed vigorously, for example: the industry short message service provided for large customers such as banks, securities, e-commerce and the like based on the industry gateway mainly comprises a verification code short message, a member notification short message, a member marketing short message and the like. With the increasing scale and importance of the industry short message service, the client also puts higher requirements on the arrival speed, the arrival rate and the stability and the safety of the industry short message service.
Currently, each large network operator is seeking various methods to reduce the probability of occurrence of a fault or to most effectively cope with a sudden fault so as to reduce the service influence range and degree to the maximum extent. The existing resource scheduling method mainly focuses on the conditions of resources, such as resource CPU, memory, network bandwidth, utilization rate and the like, and achieves the purpose of relieving the resource use pressure by taking the indexes reaching a certain threshold as a key basis and performing resource scheduling according to a preset strategy. The actual fault factors affecting the service stability and normal operation are many, and besides the resource condition, the fault factors also relate to the application software operation condition, the link state, the service index and the like. In addition, if the resources in use are abnormal, the resource nodes are in failure or sudden failure also affects the scheduling of the resources. If the resource is over-scheduled and frequently scheduled, a new fault may be caused when the current service cannot be carried due to too few resources, or the system stability is poor due to frequent resource scheduling.
Therefore, how to provide a method to improve the efficiency of resource scheduling in the process of resource scheduling is an important issue to be solved in the industry.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a service platform resource scheduling processing method and device.
In one aspect, the present invention provides a method for scheduling and processing resources of a service platform, including:
the method comprises the steps that a controller obtains working index information of each resource in a service platform, and a resource fault set is obtained according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period;
the controller acquires a data flow table, wherein the data flow table comprises resources in a normal working state in the last period;
and the controller determines the resources needing to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
In another aspect, the present invention provides a service platform resource scheduling processing apparatus, including:
the processing unit is used for acquiring working index information of each resource in the service platform and acquiring a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period;
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a data flow table which comprises resources in a normal working state in the last period;
and the processing unit is used for determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
According to the service platform resource scheduling processing method and device provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic diagram of a resource scheduling system architecture based on SDN technology according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a service platform resource scheduling processing method according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of a resource scheduling processing method of a service platform according to another embodiment of the present invention;
fig. 4 is a flowchart illustrating a resource scheduling processing method of a service platform according to another embodiment of the present invention;
fig. 5 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to another embodiment of the present invention;
fig. 7 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to another embodiment of the present invention;
fig. 8 is a schematic physical structure diagram of an electronic device according to an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic diagram of a resource scheduling system architecture based on an SDN technology according to an embodiment of the present invention, which is an application environment according to an embodiment of the present invention. The Software Defined Network (SDN) is a novel Network innovation architecture, and is an implementation manner of Network virtualization, and a core technology OpenFlow separates a control plane and a data plane of a Network device, so that flexible control of Network flow is realized, and the Network becomes more intelligent as a pipeline.
As shown in fig. 1, the service platform 105 is composed of a plurality of application servers, and provides resources required for processing services. The fault collection interface 104 is used for acquiring information of the service platform 105 and uploading the acquired information to the controller 102. The data switch 103 adopts an OpenFlow protocol interface of the SDN technology, and may upload the data flow table to the controller 102, and forward the data packet according to an instruction issued by the controller 102. The controller 102 may receive information uploaded by the failure acquisition interface 104, and may also send a control instruction to the data switch 103 to update flow table information of the data switch 103. The SDN Manager101 is a manual console, and provides a human-computer interface for the SDN controller 102, and manages various parameters of the controller 102, OpenFlow protocol parameters, parameters of the fault acquisition interface 104, and the like through the console 102.
Fig. 2 is a schematic flow chart of a service platform resource scheduling processing method according to an embodiment of the present invention, and as shown in fig. 2, the service platform resource scheduling processing method provided by the present invention includes:
s201, a controller acquires working index information of each resource in a service platform, and acquires a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period;
specifically, the fault acquisition interface may acquire work index information of each resource in the service platform, and upload the acquired work index information to the controller, where the acquisition of the work index information by the fault acquisition interface may be periodic, that is, the work index information is acquired every predetermined time, for example, 30 seconds, and the predetermined time may be set according to an actual situation, which is not limited in the embodiment of the present invention. The working index information is preset from the perspective of business processing, and includes but is not limited to: the running condition of the processor process or thread, the connectivity between the processor and the database, the success rate of processing the processor message, the overstock condition of the processor message queue and the proportion of the processor abnormal error code. And the controller receives the working information indexes, compares the working index information of each resource with preset health index information, and judges the resource corresponding to the working index information as a fault resource if the working index information does not meet the condition of the preset health index information. All the judged fault resources in the current period form a fault resource set. The preset health index information corresponds to the working information index, the condition meeting the resource is set, and the resource is judged to be the resource in the normal working state, namely the normal resource, by the controller only when the resource needs to meet all the conditions. For example, the processor process or thread runs normally, the connectivity between the processor and the database is normal, the message processing success rate of the processor is not lower than 60%, and the like. The working index information and the preset health index information are correspondingly set according to a service platform of actual work, and the embodiment of the invention is not limited.
S202, the controller acquires a data flow table, wherein the data flow table comprises resources in a normal working state in the last period;
specifically, the data switch uploads a stored data flow table to the controller, the data flow table includes resources in a normal working state in a previous period, which are hereinafter referred to as normal resources, and the controller receives the data flow table uploaded by the data switch.
S203, the controller determines the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
Specifically, after the controller obtains the resource fault set and the data flow table, the controller obtains the resource to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and the preset scheduling rule. For example, the controller may determine that a normal resource that belongs to the resource failure set and appears in the data flow table is a resource that needs to be isolated; and judging the normal resources which do not belong to the resource fault set and do not appear in the data flow table as the resources needing to be recovered. For the resource needing to be isolated and/or recovered in the current period, in the subsequent processing, the resource needing to be isolated is subjected to an isolation operation, that is, the resource needing to be isolated and existing in the data flow table is deleted, and the resource needing to be recovered is subjected to a recovery operation, that is, the resource needing to be recovered and not existing in the data flow table is added.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved.
Fig. 3 is a schematic flow chart of a service platform resource scheduling processing method according to another embodiment of the present invention, and as shown in fig. 3, on the basis of the foregoing embodiment, further, the service platform resource scheduling processing method according to the embodiment of the present invention further includes:
and S204, the controller issues a control instruction to the data switch according to the resource needing to be isolated and/or recovered, so that the data switch updates the data flow table, and forwards data to each resource in the service platform according to the updated data flow table.
Specifically, after the resources that need to be isolated and/or recovered are obtained, the controller issues a control instruction to the data switch, updates the data flow table of the data switch, deletes the resources that need to be isolated from the data flow table, adds the resources that need to be recovered to the data flow table, and the data switch forwards data to each resource in the service platform according to the updated data flow table.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. And through updating the data flow table in the data switch, the resources used in the data flow table are all normal resources.
On the basis of the foregoing embodiments, further, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources belong to frequently scheduled resources, not recovering the recovered isolated resources;
determining recovered isolated resources appearing in the current period according to the resource fault set and the data flow table, wherein the recovered isolated resources refer to resources which have faults in the previous period and are in a normal working state in the current period; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed.
Specifically, the controller compares the number of the isolated resources at the current time with the scheduling critical value, and if the number of the isolated resources at the current time is greater than or equal to the scheduling critical value, the controller recovers the recovered isolated resources to ensure the processing of the traffic volume. In the current period, the number of the isolated resources at the current moment is possibly changed, and when the controller compares the number of the isolated resources at the current moment with the scheduling critical value for the first time, the number of the isolated resources at the current moment is equal to the number of the fault resources in the previous period; in the current period, if a resource is recovered, the number of the corresponding isolated resources at the current moment is correspondingly reduced, and if a resource is isolated, the number of the corresponding isolated resources at the current moment is correspondingly increased.
The controller compares the number of the isolated resources at the current time with the scheduling critical value, and if the number of the isolated resources at the current time is less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, the controller recovers the recovered isolated resources.
The controller compares the number of sequestered resources at the current time with the scheduling threshold, and if the number of sequestered resources at the current time is less than the scheduling threshold and the recovered sequestered resources belong to frequently scheduled resources, the controller does not recover the recovered sequestered resources.
The controller obtains all fault resources in the current period according to the resource fault set, and can determine normal resources in the current period by combining all the resources of the service platform; the controller obtains normal resources in the previous period according to the data flow table, and can determine all fault resources in the previous period by combining all resources of the service platform; the controller may determine the recovered isolated resource occurring in the current period according to the normal resource in the current period and all the fault resources in the previous period, where the recovered isolated resource is a resource that has a fault in the previous period but is in a normal working state in the current period.
The scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed. For example, according to the traffic situation, the service platform has 5 resources in total, and if 3 resources are needed to ensure that the current traffic is processed normally, that is, the current maximum isolatable resource number is 2, and the scheduling critical value Δ is 2. The determination of the scheduling critical value is related to the current traffic of the service platform, and the larger the current traffic is, the more the number of required resources is, the fewer the number of isolatable resources should be, and thus the smaller the scheduling critical value should be. The scheduling critical value may be determined according to the following formula
Wherein N is the current traffic of the service platform, M is the maximum processing traffic of the service platform, and the total number of resources of the service platform S, k is greater than 0 and less than or equal to 1, and is the utilization rate of the service platform, and generally, in order to ensure the normal operation of the service platform, k is less than 1, for example, k is 90%.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. And by giving the recovered isolated resource scheduling rule, whether the recovered isolated resource is recovered or not is judged conveniently.
On the basis of the above embodiments, further, the method further includes:
acquiring the number of times of recovering the recovered isolated resource in unit time, and if the number of times is greater than a threshold value, determining that the recovered isolated resource belongs to a frequently scheduled resource; if the number of times is not greater than the threshold, determining that the recovered isolated resource does not belong to a frequently scheduled resource.
Specifically, the frequently scheduled resource refers to a resource whose number of times of isolation or recovery reaches a threshold in a unit time, and the unit time and the threshold are set according to an actual situation, which is not limited in the embodiment of the present invention. The controller obtains the number of times of recovery of the recovered isolated resource in unit time, if the number of times of recovery is greater than the threshold, the recovered isolated resource belongs to the frequently scheduled resource, and if the number of times of recovery is not greater than the threshold, the recovered isolated resource belongs to the frequently scheduled resource. When the resource is frequently restored or isolated, the resource is judged to be a fault resource for multiple times in a short time, although the resource is restored for multiple times, if the resource is used, a great risk of fault exists, instability of resource scheduling is increased, and therefore the resource which is judged to be frequently scheduled cannot be restored.
For example, a time point when each resource is isolated or recovered is marked, time windows with different granularities can be selected according to the nature of the service, when a certain resource is isolated or recovered for 3 times or more in the same time window, the resource can be judged to be a frequently scheduled resource, and a time window with a smaller granularity is recommended to be selected for the service with high requirement on the network stability of the service platform.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. By judging the frequently scheduled resources, the efficiency of resource scheduling is further improved.
On the basis of the foregoing embodiments, further, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, calculating the times of isolating all fault resources in the service platform within a preset time period, carrying out priority sequencing on all the fault resources according to the times, recovering the fault resources with low priority, and isolating the fault resources with high priority; wherein, the lower the frequency is, the lower the priority corresponding to the fault resource is;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the fault resources are not isolated fault resources, isolating the fault resources; wherein the non-isolated fault resource refers to a resource which is not isolated in the previous cycle and has a fault in the current cycle;
determining the fault resource occurring in the current period according to the resource fault set; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed.
Specifically, the controller compares the number of the isolated resources at the current time with a scheduling critical value, and if the number of the isolated resources at the current time is greater than or equal to the scheduling critical value, the fault resources with low priority are recovered, so that the number of the isolated resources is smaller than the scheduling critical value, and the number of the available resources in the current period can meet the normal processing of the current traffic. The fault resource with the low priority is obtained by the controller by calculating the times of isolating all fault resources in the service platform within a preset time period and performing priority sequencing on all fault resources according to the times, wherein the lower the times, the lower the priority corresponding to the fault resource is. It is appreciated that the non-recovered high priority failed resource may be isolated after the number of isolated resources is less than the scheduling threshold.
And the controller compares the number of the isolated resources at the current moment with the scheduling critical value, and if the number of the isolated resources at the current moment is smaller than the scheduling critical value, the controller judges whether the fault resources are the non-isolated fault resources. And the controller acquires the resources in the normal working state in the last period according to the data flow table, and if the fault resources appear in the data flow table, the fault resources are the resources which are not isolated, and the fault resources which are not isolated are isolated. The non-isolated fault resource refers to a resource which is not isolated in the previous period and has a fault in the current period.
And the controller acquires the fault resource occurring in the current period according to the resource fault set. The scheduling threshold has already been explained in the above embodiments, and is not described herein again.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. By giving out a fault resource scheduling rule, whether to recover or isolate the fault resource is convenient to judge.
Fig. 4 is a schematic flow chart of a service platform resource scheduling processing method according to another embodiment of the present invention, and the service platform resource scheduling processing method provided by the present invention is exemplified with reference to fig. 4. Assuming that the service platform has A, B, C, D, E, F, G, H eight resources in total, the controller obtains the fault set of the current cycle as: A. c, H, respectively; then the normal resources for the current period are: B. d, E, F, G, respectively; if the normal resources in the flow table acquired by the controller are: B. c, E, F, G, respectively; then the last cycle failed resource is: A. d, H are provided.
As shown in fig. 4, if the controller determines in step 1 whether the resource is a failed resource, it first processes a normal resource in the current cycle, that is, B, D, E, F, G. In step 2, it is determined whether the normal resource is the recovered isolated resource, only D is the resource failed in the previous cycle, and D is the normal resource in the current cycle, i.e., D is the recovered isolated resource, no processing will be performed on B, E, F, G. Continuing step 3 to judge whether the number of the isolated resources at the current moment reaches a scheduling critical value, if the number of the resources required by the current service processing is 4, and under the condition that the service platform utilization rate is 100%, the scheduling critical value is 8-4 to 4; since the number of the isolated resources at the current moment is equal to the number of the resources with the fault in the previous period, both are 3, and are smaller than the scheduling critical value, the controller performs step 4 to judge whether D is a frequently scheduled resource, and if D is determined to be the frequently scheduled resource, D is not recovered; and if D is determined not to be the frequently scheduled resource, starting a general scheduling mechanism for D, wherein the general scheduling mechanism refers to changing the state of the resource in the last period, namely recovering the isolated resource or isolating the normal resource. And when the step 3 is continued for the D to judge whether the scheduling critical value is reached, if the number of the resources required by the current service processing is 5, the scheduling critical value is 3, and the number of the resources is equal to the number of the resources which are isolated currently, the controller directly enables a general scheduling mechanism for the D, namely recovers the D.
After the controller completes processing the normal resource B, D, E, F, G, the controller sequentially processes the failed resource A, C, H of the current cycle. If the number of the resources required by the current service processing is 4, the normal resource D is not recovered as a result of the processing. For A, the controller performs step 5, determines whether the number of the isolated resources at the current time reaches a scheduling critical value, and then determines whether the A is the non-isolated fault resource because the number of the currently isolated resources is 3 and is less than the scheduling critical value, and the controller does not recover the A because the A is the fault resource in the previous period and the A is not the non-isolated fault resource; for C, the number of the currently isolated resources is 3, which is smaller than the scheduling critical value, and then C is judged whether the fault resources are not isolated in the step 6, because C is not isolated in the previous period and is a fault resource in the current period, C is the non-isolated fault resources, and C is isolated by adopting a general scheduling mechanism; for H, as C is isolated, the number of the isolated resources at the current moment is changed to be 4 and equal to the scheduling critical value, the controller starts a protection scheduling mechanism, namely, the number of times that all fault resources in the service platform are isolated within a preset time period is calculated, all fault resources are subjected to priority ranking according to the number of times, the fault resources with low priority are recovered, and the fault resources with high priority are isolated; the controller carries out priority sequencing on A, C, D, H, and if the priority of A is the lowest, namely the number of times of isolation in a preset time interval is the lowest, the controller preferentially restores the A; after recovering from a, the number of currently isolated resources becomes 3, which is less than the scheduling critical value 4, so that the current service processing requirement can be met, and the controller maintains C, D, H in an isolated state.
Fig. 5 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to an embodiment of the present invention, and as shown in fig. 5, the service platform resource scheduling processing apparatus provided by the present invention includes an obtaining unit 501, a receiving unit 502, and a processing unit 503, where:
the acquiring unit 501 is configured to acquire work index information of each resource in a service platform, and acquire a resource fault set according to the work index information and preset health index information for representing whether a resource has a fault; wherein the resource failure set comprises resources which have failed in the current period; the receiving unit 502 is configured to obtain a data flow table, where the data flow table includes resources in a normal working state in a previous cycle; the processing unit 503 is configured to determine the resource that needs to be isolated and/or recovered in the current cycle according to the resource fault set, the data flow table, and a preset scheduling rule.
Specifically, the fault collection interface may obtain the working index information of each resource in the service platform, and upload the obtained working index information to the obtaining unit 501, where the obtaining of the working index information by the fault collection interface may be periodic, that is, the working index information is obtained every predetermined time, for example, 30s, and the predetermined time may be set according to an actual situation, which is not limited in the embodiment of the present invention. The working index information is preset from the perspective of business processing, and includes but is not limited to: the running condition of the processor process or thread, the connectivity between the processor and the database, the success rate of processing the processor message, the overstock condition of the processor message queue and the proportion of the processor abnormal error code. The obtaining unit 501 receives the work information index, compares the work index information of each resource with preset health index information, and if the work index information does not satisfy the condition of the preset health index information, the obtaining unit 501 determines the resource corresponding to the work index information as a faulty resource. All the judged fault resources in the current period form a fault resource set. The preset health degree index information corresponds to the working information index, the condition that the resource is in the normal working state is set, and the resource is judged to be the resource in the normal working state, namely the normal resource, by the controller only when the resource needs to meet all the conditions. For example, the processor process or thread runs normally, the connectivity between the processor and the database is normal, the message processing success rate of the processor is not lower than 60%, and the like. The working index information and the preset health index information are correspondingly set according to a service platform of actual work, and the embodiment of the invention is not limited.
The data switch uploads the stored data flow table to the receiving unit 502, the data flow table includes resources in a normal working state in the last period, hereinafter referred to as normal resources, and the receiving unit 502 receives the data flow table uploaded by the data switch.
After acquiring the resource fault set and the data flow table, the processing unit 503 acquires the resources to be isolated and/or recovered in the current cycle according to the resource fault set, the data flow table, and the preset scheduling rule. For example, the controller may determine that a normal resource that belongs to the resource failure set and appears in the data flow table is a resource that needs to be isolated; and judging the normal resources which do not belong to the resource fault set and do not appear in the data flow table as the resources needing to be recovered. For the resource needing to be isolated and/or recovered in the current period, in the subsequent processing, the resource needing to be isolated is subjected to an isolation operation, that is, the resource needing to be isolated and existing in the data flow table is deleted, and the resource needing to be recovered is subjected to a recovery operation, that is, the resource needing to be recovered and not existing in the data flow table is added.
According to the service platform resource scheduling processing device provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved.
Fig. 6 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to another embodiment of the present invention, and as shown in fig. 6, the service platform resource scheduling processing apparatus provided in the present invention further includes:
the sending unit 504 is configured to issue a control instruction to the data switch according to the resource that needs to be isolated and/or recovered, so that the data switch updates the data flow table, and forwards data to each resource in the service platform according to the updated data flow table.
Specifically, after the resources that need to be isolated and/or recovered are obtained, the controller issues a control instruction to the data switch, updates the data flow table of the data switch, deletes the resources that need to be isolated from the data flow table, adds the resources that need to be recovered to the data flow table, and the data switch forwards data to each resource in the service platform according to the updated data flow table.
According to the service platform resource scheduling processing device provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. And through updating the data flow table in the data switch, the resources used in the data flow table are all normal resources.
On the basis of the foregoing embodiments, further, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources belong to frequently scheduled resources, not recovering the recovered isolated resources;
determining the recovered isolated resource which appears in the current period according to the resource fault set and the data flow table, wherein the recovered isolated resource refers to a resource which has a fault in the previous period and is in a normal working state in the current period; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed.
Specifically, the controller compares the number of the isolated resources at the current time with the scheduling critical value, and if the number of the isolated resources at the current time is greater than or equal to the scheduling critical value, the controller recovers the recovered isolated resources to ensure the processing of the traffic volume. In the current period, the number of the isolated resources at the current moment is possibly changed, and when the controller compares the number of the isolated resources at the current moment with the scheduling critical value for the first time, the number of the isolated resources at the current moment is equal to the number of the fault resources in the previous period; in the current period, if a resource is recovered, the number of the corresponding isolated resources at the current moment is correspondingly reduced, and if a resource is isolated, the number of the corresponding isolated resources at the current moment is correspondingly increased.
The controller compares the number of the isolated resources at the current time with the scheduling critical value, and if the number of the isolated resources at the current time is less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, the controller recovers the recovered isolated resources.
The controller compares the number of sequestered resources at the current time with the scheduling threshold, and if the number of sequestered resources at the current time is less than the scheduling threshold and the recovered sequestered resources belong to frequently scheduled resources, the controller does not recover the recovered sequestered resources.
The controller obtains all fault resources in the current period according to the resource fault set, and can determine normal resources in the current period by combining all the resources of the service platform; the controller obtains normal resources in the previous period according to the data flow table, and can determine all fault resources in the previous period by combining all resources of the service platform; the controller may determine the recovered isolated resource occurring in the current period according to the normal resource in the current period and all the fault resources in the previous period, where the recovered isolated resource is a resource that has a fault in the previous period but is in a normal working state in the current period.
The scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed. For example, according to the traffic situation, the service platform has 5 resources in total, and if 3 resources are needed to ensure that the current traffic is processed normally, that is, the current maximum isolatable resource number is 2, and the scheduling critical value Δ is 2. The determination of the scheduling critical value is related to the current traffic of the service platform, and the larger the current traffic is, the more the number of required resources is, the fewer the number of isolatable resources should be, and thus the smaller the scheduling critical value should be. The scheduling critical value may be determined according to the following formula
Wherein N is the current traffic of the service platform, M is the maximum processing traffic of the service platform, and the total number of resources of the service platform S, k is greater than 0 and less than or equal to 1, and is the utilization rate of the service platform, and generally, in order to ensure the normal operation of the service platform, k is less than 1, for example, k is 90%.
According to the service platform resource scheduling processing device provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. And by giving the recovered isolated resource scheduling rule, whether the recovered isolated resource is recovered or not is judged conveniently.
Fig. 7 is a schematic structural diagram of a service platform resource scheduling processing apparatus according to another embodiment of the present invention, and as shown in fig. 7, the service platform resource scheduling processing apparatus provided in the present invention further includes:
the determining unit 505 is configured to obtain a number of times that the recovered isolated resource is recovered in a unit time, and if the number of times is greater than a threshold, determine that the recovered isolated resource belongs to a resource that is frequently scheduled; if the number of times is not greater than the threshold, determining that the recovered isolated resource does not belong to a frequently scheduled resource.
Specifically, the frequently scheduled resource refers to a resource whose number of times of isolation or recovery reaches a threshold in a unit time, and the unit time and the threshold are set according to an actual situation, which is not limited in the embodiment of the present invention. The determining unit 505 obtains the number of times that the recovered isolated resource is recovered in a unit time, if the number of times of recovery is greater than the threshold, it indicates that the recovered isolated resource belongs to the frequently scheduled resource, and if the number of times of recovery is not greater than the threshold, it indicates that the recovered isolated resource belongs to the frequently scheduled resource. When the resource is frequently restored or isolated, the resource is judged to be a fault resource for multiple times in a short time, although the resource is restored for multiple times, if the resource is used, a great risk of fault exists, instability of resource scheduling is increased, and therefore the resource which is judged to be frequently scheduled cannot be restored.
According to the service platform resource scheduling processing device provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. By judging the frequently scheduled resources, the efficiency of resource scheduling is further improved.
On the basis of the foregoing embodiments, further, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, calculating the times of isolating each resource in the service platform within a preset time period, sequencing the priority of all fault resources according to the times, recovering the fault resources with low priority, and isolating the fault resources with high priority; wherein, the lower the frequency is, the lower the priority corresponding to the fault resource is;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the fault resources are not isolated fault resources, isolating the fault resources; wherein the non-isolated fault resource refers to a resource which is not isolated in the previous cycle and has a fault in the current cycle;
determining fault resources occurring in the current period according to the resource fault set; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed.
Specifically, the controller compares the number of the isolated resources at the current time with a scheduling critical value, and if the number of the isolated resources at the current time is greater than or equal to the scheduling critical value, the fault resources with low priority are recovered, so that the number of the isolated resources is smaller than the scheduling critical value, and the number of the available resources in the current period can meet the normal processing of the current traffic. The fault resource with the low priority is obtained by the controller by calculating the times of isolating all fault resources in the service platform within a preset time period and performing priority sequencing on all fault resources according to the times, wherein the lower the times, the lower the priority corresponding to the fault resource is. It is appreciated that the non-recovered high priority failed resource may be isolated after the number of isolated resources is less than the scheduling threshold.
And the controller compares the number of the isolated resources at the current moment with the scheduling critical value, and if the number of the isolated resources at the current moment is smaller than the scheduling critical value, the controller judges whether the fault resources are the non-isolated fault resources. And the controller acquires the resources in the normal working state in the last period according to the data flow table, if the fault resources appear in the data flow table, the fault resources are the non-isolated resources, and the non-isolated fault resources are isolated. The non-isolated fault resource refers to a resource which is not isolated in the previous period and has a fault in the current period.
The controller obtains fault resources occurring in the current period according to the resource fault set; the scheduling threshold has already been explained in the above embodiments, and is not described herein again.
According to the service platform resource scheduling processing method provided by the invention, the resource fault set can be obtained by comparing the working index information with the health index information, the data flow table is obtained through the controller, and the resource needing to be isolated and/or recovered is determined according to the resource fault set, the data flow table and the preset scheduling rule, so that the use of fault resources in the resource scheduling process is avoided, and the resource scheduling efficiency is improved. By giving out a fault resource scheduling rule, whether to recover or isolate the fault resource is convenient to judge.
The embodiment of the apparatus provided in the present invention may be specifically configured to execute the processing flows of the above method embodiments, and the functions of the apparatus are not described herein again, and refer to the detailed description of the above method embodiments.
Fig. 8 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 8, the electronic device provided by the present invention includes:
a processor (processor)801, a memory (memory)802, and a communication bus 803;
wherein the content of the first and second substances,
the processor 801 and the memory 802 complete communication with each other through the communication bus 803;
the processor 801 is configured to call program instructions in the memory 802 to perform the methods provided by the above-described method embodiments, including for example: acquiring working index information of each resource in a service platform, and acquiring a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period; acquiring a data flow table, wherein the data flow table comprises resources in a normal working state in the last period; and determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, enable the computer to perform the methods provided by the above-mentioned method embodiments, for example, including: acquiring working index information of each resource in a service platform, and acquiring a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period; acquiring a data flow table, wherein the data flow table comprises resources in a normal working state in the last period; and determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
The embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores a computer instruction, where the computer instruction causes a computer to execute the method provided in the foregoing method embodiments, for example, obtain work index information of each resource in a service platform, and obtain a resource fault set according to the work index information and preset health index information used for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period; acquiring a data flow table, wherein the data flow table comprises resources in a normal working state in the last period; and determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (6)
1. A service platform resource scheduling processing method is characterized by comprising the following steps:
the method comprises the steps that a controller obtains working index information of each resource in a service platform, and a resource fault set is obtained according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period;
the controller acquires a data flow table, wherein the data flow table comprises resources in a normal working state in the last period;
the controller determines the resources needing to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule;
wherein the scheduling rule comprises:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources belong to frequently scheduled resources, not recovering the recovered isolated resources;
determining the recovered isolated resource which appears in the current period according to the resource fault set and the data flow table, wherein the recovered isolated resource refers to a resource which has a fault in the previous period and is in a normal working state in the current period; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed;
or, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, calculating the times of isolating all fault resources in the service platform within a preset time period, carrying out priority sequencing on all the fault resources according to the times, recovering the fault resources with low priority, and isolating the fault resources with high priority; wherein, the lower the frequency is, the lower the priority corresponding to the fault resource is;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the fault resources are not isolated fault resources, isolating the fault resources; wherein the non-isolated fault resource refers to a resource which is not isolated in the previous cycle and has a fault in the current cycle;
and determining the fault resources occurring in the current period according to the resource fault set.
2. The method of claim 1, further comprising:
and the controller issues a control instruction to a data switch according to the resources needing to be isolated and/or recovered so that the data switch updates the data flow table and forwards data to each resource in the service platform according to the updated data flow table.
3. The method of claim 1, further comprising:
acquiring the number of times of recovering the recovered isolated resource in unit time, and if the number of times is greater than a threshold value, determining that the recovered isolated resource belongs to a frequently scheduled resource; if the number of times is not greater than the threshold, determining that the recovered isolated resource does not belong to a frequently scheduled resource.
4. A service platform resource scheduling processing device is characterized by comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring working index information of each resource in a service platform and acquiring a resource fault set according to the working index information and preset health index information for representing whether the resource has a fault; wherein the resource failure set comprises resources which have failed in the current period;
the receiving unit is used for acquiring a data flow table, and the data flow table comprises resources which are in a normal working state in the last period;
the processing unit is used for determining the resources which need to be isolated and/or recovered in the current period according to the resource fault set, the data flow table and a preset scheduling rule;
wherein the scheduling rule comprises:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources do not belong to the frequently scheduled resources, recovering the recovered isolated resources;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the recovered isolated resources belong to frequently scheduled resources, not recovering the recovered isolated resources;
determining the recovered isolated resource which appears in the current period according to the resource fault set and the data flow table, wherein the recovered isolated resource refers to a resource which has a fault in the previous period and is in a normal working state in the current period; the scheduling critical value is the maximum isolatable resource number in the service platform under the condition of ensuring that the current service volume is normally processed;
or, the scheduling rule includes:
if the number of the isolated resources at the current moment is judged to reach the scheduling critical value, calculating the times of isolating all fault resources in the service platform within a preset time period, carrying out priority sequencing on all the fault resources according to the times, recovering the fault resources with low priority, and isolating the fault resources with high priority; wherein, the lower the frequency is, the lower the priority corresponding to the fault resource is;
if the number of the isolated resources at the current moment is judged to be less than the scheduling critical value and the fault resources are not isolated fault resources, isolating the fault resources; wherein the non-isolated fault resource refers to a resource which is not isolated in the previous cycle and has a fault in the current cycle;
and determining the fault resources occurring in the current period according to the resource fault set.
5. The apparatus of claim 4, further comprising:
and the sending unit is used for sending a control instruction to the data switch according to the resources needing to be isolated and/or recovered so that the data switch updates the data flow table and forwards data to each resource in the service platform according to the updated data flow table.
6. The apparatus of claim 4, further comprising:
a judging unit, configured to obtain a number of times that the recovered isolated resource is recovered in a unit time, and if the number of times is greater than a threshold, determine that the recovered isolated resource belongs to a frequently scheduled resource; if the number of times is not greater than the threshold, determining that the recovered isolated resource does not belong to a frequently scheduled resource.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611198113.2A CN108234188B (en) | 2016-12-22 | 2016-12-22 | Service platform resource scheduling processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611198113.2A CN108234188B (en) | 2016-12-22 | 2016-12-22 | Service platform resource scheduling processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108234188A CN108234188A (en) | 2018-06-29 |
| CN108234188B true CN108234188B (en) | 2021-06-04 |
Family
ID=62656112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611198113.2A Active CN108234188B (en) | 2016-12-22 | 2016-12-22 | Service platform resource scheduling processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108234188B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103491115A (en) * | 2012-06-12 | 2014-01-01 | 华为软件技术有限公司 | Resource dispatching method, resource dispatching device and resource dispatching system |
| CN104090823A (en) * | 2014-06-09 | 2014-10-08 | 中国建设银行股份有限公司 | Flow control method and device for computer system |
| CN104486255A (en) * | 2014-12-30 | 2015-04-01 | 杭州华三通信技术有限公司 | Service resource dispatching method and device |
-
2016
- 2016-12-22 CN CN201611198113.2A patent/CN108234188B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103491115A (en) * | 2012-06-12 | 2014-01-01 | 华为软件技术有限公司 | Resource dispatching method, resource dispatching device and resource dispatching system |
| CN104090823A (en) * | 2014-06-09 | 2014-10-08 | 中国建设银行股份有限公司 | Flow control method and device for computer system |
| CN104486255A (en) * | 2014-12-30 | 2015-04-01 | 杭州华三通信技术有限公司 | Service resource dispatching method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108234188A (en) | 2018-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108667654B (en) | Automatic server cluster capacity expansion method and related equipment | |
| CN110457176B (en) | Monitoring method and device for distributed system, storage medium and electronic equipment | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| KR20210156243A (en) | Training methods of deep-running frameworks, devices and storage media | |
| CN111666187B (en) | Method and apparatus for detecting abnormal response time | |
| CN113742174B (en) | Cloud mobile phone application monitoring method and device, electronic equipment and storage medium | |
| JP2013150083A (en) | Network abnormality detection device and network abnormality detection method | |
| CN108234188B (en) | Service platform resource scheduling processing method and device | |
| WO2022121656A1 (en) | System performance optimization method and apparatus, and electronic device and readable medium | |
| CN116431313A (en) | Scheduling method, device, equipment and medium for polling task | |
| CN115774602A (en) | Container resource allocation method, device, equipment and storage medium | |
| US20230336409A1 (en) | Combination rules creation device, method and program | |
| CN114936106A (en) | Method, device and medium for processing host fault | |
| CN111694705A (en) | Monitoring method, device, equipment and computer readable storage medium | |
| CN114706893A (en) | Fault detection method, device, equipment and storage medium | |
| CN113656239A (en) | Monitoring method and device for middleware and computer program product | |
| CN114035906A (en) | Virtual machine migration method and device, electronic equipment and storage medium | |
| CN112631843A (en) | Equipment testing method and device, electronic equipment, readable medium and product | |
| CN116016265B (en) | Message all-link monitoring method, device, system, equipment and storage medium | |
| CN115600671B (en) | Data processing method, device, equipment and storage medium of deep learning framework | |
| CN116723111B (en) | Service request processing method, system and electronic equipment | |
| CN112988842B (en) | Method and device for associating user ID | |
| CN111124630B (en) | System and method for operating Spark Streaming program | |
| CN108184141B (en) | Processing method of monitoring video task and server | |
| CN115934479A (en) | Interface service control method, device, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |