CN112445857A - Resource quota management method and device based on database - Google Patents

Resource quota management method and device based on database Download PDF

Info

Publication number
CN112445857A
CN112445857A CN201910801852.3A CN201910801852A CN112445857A CN 112445857 A CN112445857 A CN 112445857A CN 201910801852 A CN201910801852 A CN 201910801852A CN 112445857 A CN112445857 A CN 112445857A
Authority
CN
China
Prior art keywords
quota
information
resource
database
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910801852.3A
Other languages
Chinese (zh)
Inventor
温帮
许钦亚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201910801852.3A priority Critical patent/CN112445857A/en
Publication of CN112445857A publication Critical patent/CN112445857A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying

Abstract

The invention discloses a resource quota management method and device based on a database, and relates to the technical field of computers. One embodiment of the method comprises: predicting resource usage information of a user to a database cluster in a future time period based on historical resource usage information of the user to the database cluster; acquiring preset resource quota information corresponding to the user, and judging whether the resource quota information needs to be adjusted or not according to the resource use information; and under the condition that the resource quota information needs to be adjusted, adjusting the resource quota information according to a set adjustment strategy. According to the method, the resource use condition of the database cluster by the user in the future time period is predicted, and the resource quota is adjusted based on the prediction result, so that the dynamic adjustment of the resource quota is realized, and the manual intervention is reduced.

Description

Resource quota management method and device based on database
Technical Field
The invention relates to the field of computers, in particular to a resource quota management method and device based on a database.
Background
The HBase is a distributed and column-oriented database built on top of the distributed file system Hadoop, and quota limits are added after the hbase1.1 version to limit the amount of traffic, storage, etc. users access to the HBase. The quota limit of the flow and the storage amount refers to: and setting a flow quota and a storage quota, and returning overrun abnormal information to the client after the flow or the storage corresponding to the user request exceeds the corresponding quota limit.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
the quota value is fixed and can only be manually modified; when the quota is over-limit, the exception is directly returned to the client, and the processing mode is single; the value of the quota is irrelevant to the cluster resource, and the maximum utilization of the cluster resource cannot be realized.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for managing resource quotas based on a database, which implement dynamic adjustment of resource quotas and reduce manual intervention by predicting resource usage of a database cluster by a user in a future time period and adjusting the resource quotas based on a prediction result.
To achieve the above object, according to an aspect of the embodiments of the present invention, a resource quota management method based on a database is provided.
The resource quota management method based on the database comprises the following steps: predicting resource usage information of a user to a database cluster in a future time period based on historical resource usage information of the user to the database cluster; acquiring preset resource quota information corresponding to the user, and judging whether the resource quota information needs to be adjusted or not according to the resource use information; and under the condition that the resource quota information needs to be adjusted, adjusting the resource quota information according to a set adjustment strategy.
Optionally, the historical resource usage information includes historical traffic information and/or historical storage information, and the method further includes: counting the historical flow information and/or the historical storage amount information generated when the user operates the data stored in the database cluster; and summarizing the historical traffic information and/or the historical storage information by taking the data table to which the data belongs or the naming space to which the data table belongs as granularity.
Optionally, the resource usage information includes traffic information and/or storage information; the predicting resource usage information of the user for the database cluster over a future time period comprises: determining at least one parameter value of a plurality of time intervals from the summary result according to a set time interval, wherein the parameter value comprises: a flow peak value, a storage peak value, a flow average value and a storage average value; and calculating the resource use information of the user on the database cluster in a future time period according to the incremental change trend of the parameter values.
Optionally, the determining whether the resource quota information needs to be adjusted includes: judging whether the resource usage information is larger than a quota in the resource quota information; the adjusting the resource quota information includes: if the resource usage information is less than or equal to the quota and the quota utilization rate of the database cluster is less than or equal to a first threshold, carrying out capacity reduction processing on the resource quota information; and if the resource usage information is larger than the quota and the sum of the idle resources existing in the database cluster and the quota meets the resource usage information, performing capacity expansion processing on the resource quota information.
Optionally, the method further comprises: under the condition that the resource use information is larger than the quota of the resource use information, judging whether the resource use information is larger than the quota of the database node in the database cluster; and if the resource use information is larger than the quota of the database node and the database node has idle resources, opening the permission of allowing the temporary operation of the service overrun for the user.
Optionally, the method further comprises: under the condition that the resource use information is larger than the quota of the resource use information, judging whether the service flow larger than the quota is burst flow; and if the service flow larger than the quota is burst flow, isolating the corresponding service table into an independent packet.
Optionally, the method further comprises: archiving cold data stored by the database cluster; the cold data is data which are not accessed within set time or the number of access times is less than or equal to a second threshold value; and adjusting the partition distribution of the database nodes according to the flow information of the current partition on the database nodes in the database cluster so as to realize flow balance.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a resource quota management apparatus based on a database.
The resource quota management device based on the database of the embodiment of the invention comprises: the prediction module is used for predicting the resource use information of the user to the database cluster in a future time period based on the historical resource use information of the user to the database cluster; the judging module is used for acquiring preset resource quota information corresponding to the user and judging whether the resource quota information needs to be adjusted or not according to the resource use information; and the adjusting module is used for adjusting the resource quota information according to a set adjusting strategy under the condition that the resource quota information needs to be adjusted.
Optionally, the historical resource usage information includes historical traffic information and/or historical storage information, the apparatus further includes: the statistical summarizing module is used for counting the historical flow information and/or the historical storage amount information generated when the user operates the data stored in the database cluster; and summarizing the historical traffic information and/or the historical storage information by taking the data table to which the data belongs or the naming space to which the data table belongs as granularity.
Optionally, the resource usage information includes traffic information and/or storage information; the prediction module is further configured to: determining at least one parameter value of a plurality of time intervals from the summary result according to a set time interval, wherein the parameter value comprises: a flow peak value, a storage peak value, a flow average value and a storage average value; and calculating the resource use information of the user on the database cluster in a future time period according to the incremental change trend of the parameter values.
Optionally, the determining module is further configured to: judging whether the resource usage information is larger than a quota in the resource quota information; the adjusting module is further configured to: if the resource usage information is less than or equal to the quota and the quota utilization rate of the database cluster is less than or equal to a first threshold, carrying out capacity reduction processing on the resource quota information; and if the resource usage information is larger than the quota and the sum of the idle resources existing in the database cluster and the quota meets the resource usage information, performing capacity expansion processing on the resource quota information.
Optionally, the apparatus further comprises: the quota soft limiting module is used for judging whether the resource use information is larger than the quota of the database node in the database cluster or not under the condition that the resource use information is larger than the quota of the resource use information; and if the resource use information is larger than the quota of the database node and the database node has idle resources, opening the permission of allowing the temporary operation of the service overrun for the user.
Optionally, the apparatus further comprises: the abnormal traffic isolation module is used for judging whether the service traffic larger than the quota is burst traffic or not under the condition that the resource use information is larger than the quota of the resource use information; and if the service flow larger than the quota is burst flow, isolating the corresponding service table into an independent packet.
Optionally, the apparatus further comprises: the storage quota prevention module is used for filing the cold data stored by the database cluster; the cold data is data which are not accessed within set time or the number of access times is less than or equal to a second threshold value; and the flow quota prevention module is used for adjusting the partition distribution of the database nodes according to the flow information of the current partition on the database nodes in the database cluster so as to realize flow balance.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided an electronic apparatus.
An electronic device of an embodiment of the present invention includes: one or more processors; a storage device, configured to store one or more programs, where when the one or more programs are executed by the one or more processors, the one or more processors implement a method for database-based resource quota management according to an embodiment of the present invention.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer-readable medium.
A computer-readable medium of an embodiment of the present invention stores thereon a computer program, and when the computer program is executed by a processor, the computer program implements a database-based resource quota management method of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: by predicting the resource use condition of the database cluster by the user in the future time period and adjusting the resource quota based on the prediction result, the dynamic adjustment of the resource quota is realized, and the manual intervention is reduced; capacity expansion and capacity reduction are correspondingly carried out on the resource quota by combining the idle resource condition and the quota use condition of the database cluster, so that the resource utilization rate is improved; when the quota exceeds the limit, the cluster resources are utilized to the maximum extent by a mode of allowing the temporary operation of the service on the premise of ensuring the stability of the system; when the quota exceeds the limit, the abnormal flow is automatically isolated, and other online services are prevented from being influenced; cold data is filed, storage capacity of an on-line cluster is reduced, flow is automatically balanced, and risk of overrun of database nodes is reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a resource quota management method according to an embodiment of the present invention;
fig. 2 is a schematic main flow diagram of a resource quota adjusting part in a resource quota managing method according to an embodiment of the present invention;
fig. 3 is a schematic main flowchart of a resource quota adjusting part in a resource quota managing method according to another embodiment of the present invention;
fig. 4 is a schematic diagram illustrating an implementation principle of a quota overrun processing portion in a resource quota management method according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating an implementation principle of a quota overrun prevention part in a resource quota management method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of the main modules of a resource quota management apparatus, according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
FIG. 8 is a schematic diagram of a computer apparatus suitable for use in an electronic device to implement an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a resource quota management method according to an embodiment of the present invention. As shown in fig. 1, a resource quota management method according to an embodiment of the present invention mainly includes the following steps:
step S101: predicting resource usage information of a user to a database cluster in a future time period based on historical resource usage information of the user to the database cluster. The historical resource usage information may include historical traffic information and/or historical storage information, and the resource usage information may include traffic information and/or storage information.
A user sends a request to a database cluster through a client to operate data stored in the database cluster; counting historical flow information and/or historical storage information generated by data of a user operation database cluster in a past period of time, and summarizing according to the granularity of a data Table (Table) or a Namespace (Namespace); and predicting the flow and/or storage occupied by the data of the user operation database cluster in the future time period according to the summarized incremental change trend of the flow and/or the storage.
Step S102: and acquiring preset resource quota information corresponding to the user, and judging whether the resource quota information needs to be adjusted or not according to the resource use information. The resource quota information may be used to limit the number of requests sent to the database cluster by the user per unit time, limit the amount of data requested, limit the amount of storage, and the like.
Resource quota information corresponding to each user with the access right of the database cluster is recorded in a quota table of the database cluster. The resource quota information includes a traffic quota and/or a storage quota. Judging whether the predicted resource use information of the future time period is larger than the current quota, if so, adjusting the current quota by combining with the idle resources of the database cluster, or keeping the current quota unchanged; if the quota is less than or equal to the current quota, the current quota can be adjusted according to the quota using condition of the database cluster, or the current quota is kept unchanged.
Step S103: and under the condition that the resource quota information needs to be adjusted, adjusting the resource quota information according to a set adjustment strategy. The adjustment strategy is provided with various adjustment triggering conditions and corresponding quota adjustment modes, and can be set in a user-defined mode according to requirements. For example, the adjustment trigger condition 1 is: the resource use information is less than or equal to the quota, and the corresponding quota adjusting mode is as follows: and reducing the quota according to the set capacity reduction proportion to perform capacity reduction processing. For another example, the adjustment triggering condition 2 is: the resource use information is larger than the quota, and the corresponding quota adjusting mode is as follows: and increasing quota according to the set expansion proportion to perform expansion processing.
If the resource usage information of the future time period predicted in step S102 is less than or equal to the quota and meets the adjustment triggering condition 1 of the adjustment policy, the quota is reduced according to the set capacity reduction ratio to perform capacity reduction processing. If the resource usage information of the future time period predicted in step S102 is greater than the quota and meets the adjustment triggering condition 2 of the adjustment policy, the quota is increased according to the set expansion ratio to perform expansion processing.
The resource quota management method provided by the embodiment of the invention comprises three parts of resource quota adjustment, quota overrun processing and quota overrun prevention. The resource quota adjusting part is used for predicting resource use information of the database cluster by the user in the future time period and adjusting the resource quota according to the prediction result. And the quota overrun processing part provides a plurality of processing modes for quota overrun under the condition that the resource use information is greater than the quota, and has good usability. And the quota overrun prevention part is used for archiving cold data in the database cluster and automatically balancing the flow of the database cluster.
Fig. 2 is a schematic main flow diagram of a resource quota adjusting part in a resource quota managing method according to an embodiment of the present invention. As shown in fig. 2, the resource quota management method according to the embodiment of the present invention is applied to an HBase cluster, and the processing procedure of the resource quota adjusting portion mainly includes the following steps:
step S201: and counting flow information and/or storage amount information generated when the user operates the data of the HBase cluster within a period of time by using a quota management thread, and summarizing according to set granularity. In an embodiment, traffic information and storage information of each HBase node are counted periodically (for example, 7 days) through a quota management thread (SmartQuotaManagerChore) on a HRegonServer of the HBase cluster. The hregeniserver is a component of the HBase, and is responsible for actual reading and writing of Table data, and partition (Region) management. Region is the minimum unit of HBase cluster distribution data. The traffic information may refer to the number of requests for the HBase per unit time of the user, or may refer to the amount of data requested. The storage information may be up to the total amount of storage of the user in the database cluster.
Because the HBase is established based on the Hadoop of the distributed file system, when the data of the Table or Namespace in the HBase is counted, the data of different HBase nodes need to be summarized. In the embodiment, according to the granularity of Table or Namespace, the flow information and the storage information of each HBase node in the HBase cluster are summarized. Namespace is the logical grouping of HBase to a set of tables. In the embodiment, traffic information and storage information generated by users accessing the HBase cluster at 2019.05.01-2019.06.30 are summarized with Table as granularity, and Table 1 shows the summarized results of 2019.05.01 and 2019.05.02 for two days, where req is an abbreviation of request.
Table 1 summarizes the results of a user accessing HBase clusters
Figure BDA0002182546750000081
Step S202: and determining parameter values of a plurality of time intervals from the summary result according to the set time interval. Wherein the parameter value is one or more of: flow peak, storage peak, flow mean, storage mean. In an embodiment, the flow peak value, the storage peak value, the flow average value and the storage average value of each day when the user accesses the data table are determined from the summary result.
Step S203: and predicting the resource use information of the HBase cluster in the future time period by the user according to the incremental change trend of the parameter values. In the embodiment, the incremental change trends of the flow peak value, the storage peak value, the flow mean value and the storage mean value are respectively determined according to the flow peak value, the storage peak value, the flow mean value and the storage mean value of each day in the period from 2019.05.01 to 2019.06.30; and executing a calculation method according to the incremental change trends of the flow peak value, the storage peak value, the flow mean value and the storage mean value, and predicting the flow and the storage of the HBase cluster within 7 days in the future. For example, if the incremental trend of the flow average satisfies the linear equation y ═ kx + b, where y is the flow average, x is time, and k and b are both coefficients, the flow of the HBase cluster at the future time can be predicted according to the linear equation. And subsequently, whether the prediction result is larger than the quota of the resource quota information can be jointly determined by combining the predicted flow peak value at the future time.
Step S204: acquiring resource quota information corresponding to a user, judging whether the predicted resource usage information is larger than a quota of the resource quota information, and if the predicted resource usage information is smaller than or equal to the quota, executing step S205; if greater than the quota, step S206 is performed. In an embodiment, the resource quota information includes a traffic quota and a storage quota, for example, the traffic quota is 20 times per day for a user, and the data volume requested per day is 10M; the storage quota of the user is 500M. Resource quota information corresponding to the user can be obtained through a getquota command, and then the predicted flow and storage amount are compared with the corresponding flow quota and storage amount quota respectively; and determining which adjustment trigger condition of the adjustment strategy the comparison result meets, and processing according to a corresponding quota adjustment mode.
The adjustment strategy is provided with a plurality of adjustment triggering conditions and corresponding quota adjustment modes. For example, the adjustment trigger condition 1 is: the resource use information is less than or equal to the quota, and the corresponding quota adjusting mode is as follows: reducing the quota according to the capacity reduction ratio of 20%; the adjustment trigger condition 2 is: the resource use information is larger than the quota, and the corresponding quota adjusting mode is as follows: and increasing quota according to the expansion ratio of 20%. It should be noted that the specific values of the capacity reduction ratio and the capacity expansion ratio are set by the user, and may be 20%, 10%, etc.
Step S205: and reducing the quota according to the capacity reduction ratio set in the adjustment strategy to perform capacity reduction processing, and ending the process. Since this condition satisfies the adjustment trigger condition 1, the quota of the resource quota information is adjusted according to the quota adjustment mode corresponding to the adjustment trigger condition 1.
Step S206: and increasing quota according to the expansion ratio set in the adjustment strategy to perform expansion processing, and ending the process. Since this condition satisfies the adjustment trigger condition 2, the quota of the resource quota information is adjusted according to the quota adjustment mode corresponding to the adjustment trigger condition 2.
In an alternative embodiment, step S202 and step S203 are used to predict resource usage information of the HBase cluster by the user in a future time period. In an embodiment, the prediction may also be performed using a machine learning algorithm. The machine learning algorithm is, for example, a decision tree, a logistic regression algorithm, a support vector machine, a random forest, etc. The specific realization principle is as follows: firstly, dividing the summary result of the step S201 into a training set and a test set; then, constructing prediction characteristics for the training set and the test set respectively; then, training a model on the prediction features and the target function of the training set by using a machine learning algorithm; and finally, calculating a prediction result through the trained model and the prediction characteristics of the test set. Where the predicted characteristics are traffic and storage.
Fig. 3 is a schematic main flowchart of a resource quota adjusting part in a resource quota managing method according to another embodiment of the present invention. As shown in fig. 3, a processing procedure of a resource quota adjusting part in the resource quota managing method according to the embodiment of the present invention mainly includes the following steps:
step S301: and counting flow information and/or storage amount information generated when the user operates the data of the HBase cluster within a period of time by using a quota management thread, and summarizing according to set granularity. This step is the same as the implementation principle of step S201.
Step S302: and determining parameter values of a plurality of time intervals from the summary result according to the set time interval. Wherein the parameter value is one or more of: flow peak, storage peak, flow mean, storage mean. In an embodiment, a daily peak flow value, a daily peak storage value, a daily average flow value, and a daily average storage value are determined from the summary result.
Step S303: and predicting the resource use information of the HBase cluster in the future time period by the user according to the incremental change trend of the parameter values. In the embodiment, a calculation method is executed according to the incremental change trends of the flow peak value, the storage peak value, the flow average value and the storage average value, and the flow and the storage of the HBase cluster in the future 7 days are predicted.
Step S304: acquiring resource quota information corresponding to a user, judging whether the predicted resource use information is larger than a quota of the resource quota information, and if the predicted resource use information is smaller than or equal to the quota, executing step S305; if it is larger than the quota, step S307 is performed. Acquiring resource quota information corresponding to the user, and then respectively comparing the predicted flow and storage with the corresponding flow quota and storage quota; and determining which adjustment trigger condition of the adjustment strategy the comparison result meets, and processing according to a corresponding quota adjustment mode.
The adjustment policy of this embodiment still has a plurality of adjustment triggering conditions and corresponding quota adjustment modes. For example, the adjustment trigger condition 1 is: the resource use information is less than or equal to the quota, the quota utilization rate is less than or equal to a first threshold, and the corresponding quota adjustment mode is as follows: reducing the quota according to the capacity reduction ratio of 20%; the adjustment trigger condition 2 is: the resource use information is less than or equal to the quota, and the quota utilization rate is greater than a first threshold, and the corresponding quota adjustment mode is as follows: keeping the current quota unchanged; the adjustment trigger condition 3 is: the resource use information is larger than the quota, and the sum of the idle resource and the quota is larger than or equal to the resource use information, and the corresponding quota adjusting mode is as follows: increasing quota according to 20% of capacity expansion ratio; the adjustment trigger condition 4 is: the resource use information is larger than the quota, and the sum of the idle resource and the quota is smaller than the resource use information, and the corresponding quota adjusting mode is as follows: and keeping the current quota unchanged, and sending out early warning information.
Step S305: judging whether the quota utilization rate of the HBase cluster is less than or equal to a first threshold, and if the quota utilization rate of the HBase cluster is less than or equal to the first threshold, executing the step S306; otherwise, the flow is ended. Quota utilization is resource quota information/predicted resource usage information. For example, the utilization rate of the traffic quota is equal to the traffic quota/predicted traffic value; the utilization of the storage quota is the storage quota/predicted storage amount. And when the resource use information is less than or equal to the quota and the quota utilization rate is greater than a first threshold value, the adjustment triggering condition 2 is met, so that the processing is carried out according to a quota adjustment mode corresponding to the adjustment triggering condition 2.
Step S306: and reducing the quota according to the capacity reduction ratio set in the adjustment strategy to perform capacity reduction processing, and ending the process. And when the resource usage information is less than or equal to the quota and the quota utilization rate is less than or equal to the first threshold, the adjustment triggering condition 1 is met, so that the quota of the resource quota information is adjusted according to a quota adjustment mode corresponding to the adjustment triggering condition 1.
Step S307: judging whether the sum of the idle resources and the quota of the HBase cluster is larger than or equal to the resource use information, if so, executing the step S308; otherwise, step S309 is performed. In the embodiment, the idle flow and the idle storage of the HBase cluster are determined, and the idle flow and the idle storage are respectively added with a flow quota and a storage quota to obtain usable flow and usable storage; and respectively comparing the available flow and the available storage with the predicted flow and the predicted storage.
Step S308: and increasing quota according to the expansion ratio set in the adjustment strategy to perform expansion processing, and ending the process. And when the resource use information is larger than the quota, and the sum of the idle resource and the quota is larger than or equal to the resource use information, the adjustment triggering condition 3 is met, so that the quota of the resource quota information is adjusted according to a quota adjustment mode corresponding to the adjustment triggering condition 3.
Step S309: and sending early warning information to the client, and ending the process. And when the resource use information is larger than the quota and the sum of the idle resources and the quota is smaller than the resource use information, the adjustment triggering condition 4 is met, so that the processing is carried out according to a mode corresponding to the adjustment triggering condition 4. In the embodiment, the early warning information can be sent to the client of the corresponding user in a mode of mail, short message and the like.
Fig. 4 is a schematic diagram illustrating an implementation principle of a quota overrun processing portion in a resource quota management method according to an embodiment of the present invention. As shown in fig. 4, the quota overrun processing portion of the embodiment of the present invention includes three processes, namely quota soft limit, abnormal traffic isolation, and client automatic retry. The following is a detailed description.
Quota soft limit: judging whether the resource use information is larger than the quota of a single database node in the database cluster; if the resource use information is larger than the quota of a single database node and the database node still has idle resources, opening the permission of allowing the temporary operation of the service in an overrun mode for the user and allowing the temporary operation of the service in an overrun mode. The processing can utilize the resources of the database cluster to the maximum extent on the premise of ensuring the stability of the system.
And (3) abnormal flow isolation: judging whether the service flow larger than the resource quota information is burst flow; and if the service flow larger than the resource quota information is burst flow, isolating the corresponding service table into an independent group. In the embodiment, if the predicted flow rate of the user accessing a certain service table is greater than the set multiple of the historical flow rate peak value, for example, 10 times or 20 times, the service flow rate is considered as a burst flow rate, and isolation is automatically triggered to isolate the service table into independent groups, so as to avoid influencing other services on the line. In a preferred embodiment, the service table may also be made inaccessible by setting the service table to disable.
The client automatically retries: the client retries times are configured, and the client automatically retries after the quota is exceeded, so that the method is more friendly to the service. The retry means that the client re-initiates the request after waiting for a period of time, and if the resource quota is no longer exceeded at this time, the request is successful, otherwise, the retry is continued.
Fig. 5 is a schematic diagram illustrating an implementation principle of a quota overrun prevention part in a resource quota management method according to an embodiment of the present invention. As shown in fig. 5, the quota overrun prevention portion of the embodiment of the present invention includes storage quota prevention and traffic quota prevention.
Storage quota prevention: and archiving cold data stored by the database cluster to the cold data cluster. And the cold data is data which is not accessed within a set time or the access times are less than or equal to a second threshold value. For example, data that has not been accessed for approximately three months, or has been accessed less than 5 times, is defined as cold data. This operation can reduce the storage of the on-line cluster.
And (3) flow quota prevention: and adjusting the partition distribution of the database nodes according to the flow information of the current partition on the database nodes in the database cluster so as to realize flow balance. In the embodiment, automatic balancing operation is started through a balance (Region load balancing policy of HBase) command, and in an actual service, flow balancing is realized by moving a Region, which specifically includes: and readjusting the distribution of the regions according to the flow of the regions on each database node, and reducing the risk of quota overrun of the single database node.
According to the resource quota management method, the resource use condition of the database cluster by the user in the future time period is predicted, and the resource quota is adjusted based on the prediction result, so that the dynamic adjustment of the resource quota is realized, and the manual intervention is reduced; capacity expansion and capacity reduction are correspondingly carried out on the resource quota by combining the idle resource condition and the quota use condition of the database cluster, so that the resource utilization rate is improved; when the quota exceeds the limit, the cluster resources are utilized to the maximum extent by a mode of allowing the temporary operation of the service on the premise of ensuring the stability of the system; when the quota exceeds the limit, the abnormal flow is automatically isolated, and other online services are prevented from being influenced; cold data is filed, storage capacity of an on-line cluster is reduced, flow is automatically balanced, and risk of overrun of database nodes is reduced.
Fig. 6 is a schematic diagram of main modules of a resource quota management apparatus according to an embodiment of the present invention. As shown in fig. 6, a resource quota management apparatus 600 according to an embodiment of the present invention mainly includes:
the prediction module 601 is configured to predict resource usage information of a database cluster in a future time period for a user based on historical resource usage information of the database cluster by the user. The historical resource usage information may include historical traffic information and/or historical storage information, and the resource usage information may include traffic information and/or storage information.
A user sends a request to a database cluster through a client to operate data stored in the database cluster; counting historical flow information and/or historical storage information generated by data of a user operation database cluster in a past period of time, and summarizing according to the granularity of a data Table (Table) or a Namespace (Namespace); and predicting the flow and/or storage occupied by the data of the user operation database cluster in the future time period according to the summarized incremental change trend of the flow and/or the storage.
The determining module 602 is configured to obtain preset resource quota information corresponding to the user, and determine whether the resource quota information needs to be adjusted according to the resource usage information. The resource quota information may be used to limit the number of requests sent to the database cluster by the user per unit time, limit the amount of data requested, limit the amount of storage, and the like.
Resource quota information corresponding to each user with the access right of the database cluster is recorded in a quota table of the database cluster. The resource quota information includes a traffic quota and/or a storage quota. Judging whether the predicted resource use information of the future time period is larger than the current quota, if so, adjusting the current quota by combining with the idle resources of the database cluster, or keeping the current quota unchanged; if the quota is less than or equal to the current quota, the current quota can be adjusted according to the quota using condition of the database cluster, or the current quota is kept unchanged.
An adjusting module 603, configured to adjust the resource quota information according to a set adjustment policy when the resource quota information needs to be adjusted. The adjustment strategy is provided with various adjustment triggering conditions and corresponding quota adjustment modes, and can be set in a user-defined mode according to requirements. For example, the adjustment trigger condition 1 is: the resource use information is less than or equal to the quota, and the corresponding quota adjusting mode is as follows: and reducing the quota according to the set capacity reduction proportion to perform capacity reduction processing. For another example, the adjustment triggering condition 2 is: the resource use information is larger than the quota, and the corresponding quota adjusting mode is as follows: and increasing quota according to the set expansion proportion to perform expansion processing.
If the resource usage information of the future time period predicted in the determination module 602 is less than or equal to the quota and meets the adjustment triggering condition 1 of the adjustment policy, the quota is reduced according to the set capacity reduction ratio to perform capacity reduction processing. If the resource usage information of the future time period predicted in the determination module 602 is greater than the quota and meets the adjustment triggering condition 2 of the adjustment policy, the quota is increased according to the set expansion ratio to perform expansion processing.
In addition, the resource quota managing apparatus 600 according to the embodiment of the present invention may further include: a statistics summary module, a quota soft limit module, an abnormal traffic isolation module, a storage quota prevention module, and a traffic quota prevention module (not shown in fig. 6). The functions of the respective modules are explained below.
The statistical summarizing module is used for counting the historical flow information and/or the historical storage amount information generated when the user operates the data stored in the database cluster; and summarizing the historical traffic information and/or the historical storage information by taking the data table to which the data belongs or the naming space to which the data table belongs as granularity.
The quota soft limiting module is used for judging whether the resource use information is larger than the quota of the database node in the database cluster or not under the condition that the resource use information is larger than the quota of the resource use information; and if the resource use information is larger than the quota of the database node and the database node has idle resources, opening the permission of allowing the temporary operation of the service overrun for the user.
The abnormal traffic isolation module is used for judging whether the service traffic larger than the quota is burst traffic or not under the condition that the resource use information is larger than the quota of the resource use information; and if the service flow larger than the quota is burst flow, isolating the corresponding service table into an independent packet.
The storage quota prevention module is used for filing the cold data stored by the database cluster; the cold data is data which are not accessed within a set time or the number of times of access is less than or equal to a second threshold value.
And the flow quota prevention module is used for adjusting the partition distribution of the database nodes according to the flow information of the current partition on the database nodes in the database cluster so as to realize flow balance.
From the above description, it can be seen that by predicting the resource usage of the database cluster by the user in the future time period and adjusting the resource quota based on the prediction result, the dynamic adjustment of the resource quota is realized, and the manual intervention is reduced; capacity expansion and capacity reduction are correspondingly carried out on the resource quota by combining the idle resource condition and the quota use condition of the database cluster, so that the resource utilization rate is improved; when the quota exceeds the limit, the cluster resources are utilized to the maximum extent by a mode of allowing the temporary operation of the service on the premise of ensuring the stability of the system; when the quota exceeds the limit, the abnormal flow is automatically isolated, and other online services are prevented from being influenced; cold data is filed, storage capacity of an on-line cluster is reduced, flow is automatically balanced, and risk of overrun of database nodes is reduced.
Fig. 7 illustrates an exemplary system architecture 700 to which the database-based resource quota management method or the database-based resource quota management apparatus of the embodiments of the invention can be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server that provides various services, such as a background management server that processes requests transmitted from the user using the terminal devices 701, 702, and 703. The backend management server may analyze the received request, and feed back a processing result (e.g., predicted resource usage information, adjusted resource quota information) to the terminal device.
It should be noted that the resource quota management method based on the database provided in the embodiment of the present application is generally executed by the server 705, and accordingly, the resource quota management apparatus based on the database is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The invention also provides an electronic device and a computer readable medium according to the embodiment of the invention.
The electronic device of the present invention includes: one or more processors; a storage device, configured to store one or more programs, where when the one or more programs are executed by the one or more processors, the one or more processors implement a method for database-based resource quota management according to an embodiment of the present invention.
The computer readable medium of the present invention stores thereon a computer program, which when executed by a processor implements a database-based resource quota management method of an embodiment of the present invention.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use in implementing an electronic device of an embodiment of the present invention. The electronic device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the computer system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, the processes described above with respect to the main step diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program containing program code for performing the method illustrated in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a prediction module, a determination module, and an adjustment module. Where the names of these modules do not in some cases constitute a limitation on the modules themselves, for example, a prediction module may also be described as a "module that predicts resource usage information of a database cluster for a user over a future time period based on the user's historical resource usage information for the database cluster".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: predicting resource usage information of a user to a database cluster in a future time period based on historical resource usage information of the user to the database cluster; acquiring preset resource quota information corresponding to the user, and judging whether the resource quota information needs to be adjusted or not according to the resource use information; and under the condition that the resource quota information needs to be adjusted, adjusting the resource quota information according to a set adjustment strategy.
From the above description, it can be seen that by predicting the resource usage of the database cluster by the user in the future time period and adjusting the resource quota based on the prediction result, the dynamic adjustment of the resource quota is realized, and the manual intervention is reduced.
The product can execute the method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A resource quota management method based on a database is characterized by comprising the following steps:
predicting resource usage information of a user to a database cluster in a future time period based on historical resource usage information of the user to the database cluster;
acquiring preset resource quota information corresponding to the user, and judging whether the resource quota information needs to be adjusted or not according to the resource use information;
and under the condition that the resource quota information needs to be adjusted, adjusting the resource quota information according to a set adjustment strategy.
2. The method of claim 1, wherein the historical resource usage information comprises historical traffic information and/or historical storage information, the method further comprising:
counting the historical flow information and/or the historical storage amount information generated when the user operates the data stored in the database cluster;
and summarizing the historical traffic information and/or the historical storage information by taking the data table to which the data belongs or the naming space to which the data table belongs as granularity.
3. The method of claim 2, wherein the resource usage information comprises traffic information and/or storage information;
the predicting resource usage information of the user for the database cluster over a future time period comprises:
determining at least one parameter value of a plurality of time intervals from the summary result according to a set time interval, wherein the parameter value comprises: a flow peak value, a storage peak value, a flow average value and a storage average value;
and calculating the resource use information of the user on the database cluster in a future time period according to the incremental change trend of the parameter values.
4. The method of claim 1, wherein the determining whether the resource quota information needs to be adjusted comprises: judging whether the resource usage information is larger than a quota in the resource quota information;
the adjusting the resource quota information includes:
if the resource usage information is less than or equal to the quota and the quota utilization rate of the database cluster is less than or equal to a first threshold, carrying out capacity reduction processing on the resource quota information;
and if the resource usage information is larger than the quota and the sum of the idle resources existing in the database cluster and the quota meets the resource usage information, performing capacity expansion processing on the resource quota information.
5. The method of claim 1, further comprising:
under the condition that the resource usage information is larger than the quota of the resource quota information, judging whether the resource usage information is larger than the quota of the database node in the database cluster;
and if the resource use information is larger than the quota of the database node and the database node has idle resources, opening the permission of allowing the temporary operation of the service overrun for the user.
6. The method of claim 1, further comprising:
under the condition that the resource use information is larger than the quota of the resource quota information, judging whether the service flow larger than the quota is burst flow;
and if the service flow larger than the quota is burst flow, isolating the corresponding service table into an independent packet.
7. The method of claim 1, further comprising:
archiving cold data stored by the database cluster; the cold data is data which are not accessed within set time or the number of access times is less than or equal to a second threshold value;
and adjusting the partition distribution of the database nodes according to the flow information of the current partition on the database nodes in the database cluster so as to realize flow balance.
8. A database-based resource quota management apparatus, comprising:
the prediction module is used for predicting the resource use information of the user to the database cluster in a future time period based on the historical resource use information of the user to the database cluster;
the judging module is used for acquiring preset resource quota information corresponding to the user and judging whether the resource quota information needs to be adjusted or not according to the resource use information;
and the adjusting module is used for adjusting the resource quota information according to a set adjusting strategy under the condition that the resource quota information needs to be adjusted.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN201910801852.3A 2019-08-28 2019-08-28 Resource quota management method and device based on database Pending CN112445857A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910801852.3A CN112445857A (en) 2019-08-28 2019-08-28 Resource quota management method and device based on database

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910801852.3A CN112445857A (en) 2019-08-28 2019-08-28 Resource quota management method and device based on database

Publications (1)

Publication Number Publication Date
CN112445857A true CN112445857A (en) 2021-03-05

Family

ID=74740668

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910801852.3A Pending CN112445857A (en) 2019-08-28 2019-08-28 Resource quota management method and device based on database

Country Status (1)

Country Link
CN (1) CN112445857A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112860440A (en) * 2021-03-12 2021-05-28 云知声智能科技股份有限公司 Method and device for allocating cluster computing resources, electronic equipment and storage medium
CN113434270A (en) * 2021-06-15 2021-09-24 北京百度网讯科技有限公司 Data resource scheduling method and device, electronic equipment and storage medium
CN113886078A (en) * 2021-09-28 2022-01-04 江苏安超云软件有限公司 Method and application for realizing quota unified management based on dynamic threshold mechanism
CN114143327A (en) * 2021-12-09 2022-03-04 深圳前海微众银行股份有限公司 Cluster resource quota allocation method and device and electronic equipment
CN114390058A (en) * 2022-02-24 2022-04-22 百果园技术(新加坡)有限公司 Service management system, method, device and storage medium
CN114860160A (en) * 2022-04-15 2022-08-05 北京科杰科技有限公司 Capacity expansion resource prediction method and system for Hadoop data platform
CN116610270A (en) * 2023-07-21 2023-08-18 湖南马栏山视频先进技术研究院有限公司 Video processing calculation and separation method and video calculation and separation system

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112860440A (en) * 2021-03-12 2021-05-28 云知声智能科技股份有限公司 Method and device for allocating cluster computing resources, electronic equipment and storage medium
CN113434270A (en) * 2021-06-15 2021-09-24 北京百度网讯科技有限公司 Data resource scheduling method and device, electronic equipment and storage medium
CN113434270B (en) * 2021-06-15 2023-06-23 北京百度网讯科技有限公司 Data resource scheduling method and device, electronic equipment and storage medium
CN113886078A (en) * 2021-09-28 2022-01-04 江苏安超云软件有限公司 Method and application for realizing quota unified management based on dynamic threshold mechanism
CN113886078B (en) * 2021-09-28 2022-06-17 江苏安超云软件有限公司 Method and device for realizing quota unified management based on dynamic threshold mechanism, electronic equipment and computer readable storage medium
CN114143327A (en) * 2021-12-09 2022-03-04 深圳前海微众银行股份有限公司 Cluster resource quota allocation method and device and electronic equipment
CN114143327B (en) * 2021-12-09 2024-04-09 深圳前海微众银行股份有限公司 Cluster resource quota allocation method and device and electronic equipment
CN114390058A (en) * 2022-02-24 2022-04-22 百果园技术(新加坡)有限公司 Service management system, method, device and storage medium
CN114390058B (en) * 2022-02-24 2024-03-08 百果园技术(新加坡)有限公司 Service management system, method, device and storage medium
CN114860160A (en) * 2022-04-15 2022-08-05 北京科杰科技有限公司 Capacity expansion resource prediction method and system for Hadoop data platform
CN116610270A (en) * 2023-07-21 2023-08-18 湖南马栏山视频先进技术研究院有限公司 Video processing calculation and separation method and video calculation and separation system
CN116610270B (en) * 2023-07-21 2023-10-03 湖南马栏山视频先进技术研究院有限公司 Video processing calculation and separation method and video calculation and separation system

Similar Documents

Publication Publication Date Title
CN109684358B (en) Data query method and device
CN112445857A (en) Resource quota management method and device based on database
CN110545246B (en) Token bucket-based current limiting method, device and computer readable medium
AU2015229200B2 (en) Coordinated admission control for network-accessible block storage
CN107241281B (en) Data processing method and device
US20170126583A1 (en) Method and electronic device for bandwidth allocation based on online media services
CN109981702B (en) File storage method and system
US10250673B1 (en) Storage workload management using redirected messages
US20130117438A1 (en) Methods for adapting application services based on current server usage and devices thereof
WO2024016596A1 (en) Container cluster scheduling method and apparatus, device, and storage medium
CN108924203B (en) Data copy self-adaptive distribution method, distributed computing system and related equipment
CN106936867A (en) The response method and device of a kind of service request
CN113905091B (en) Method and device for processing access request
CN112799824A (en) Current limiting method, device, equipment and storage medium
TW202112113A (en) Resource sharing method, device, computer device and storage media
CN111190719A (en) Method, device, medium and electronic equipment for optimizing cluster resource allocation
CN113268329A (en) Request scheduling method, device and storage medium
CN113765969A (en) Flow control method and device
CN112286930A (en) Method, device, storage medium and electronic equipment for resource sharing of redis business side
CN113765964A (en) Method and device for distributing services of distributed system
CN115858147A (en) Cost modeling method and device
CN115378879A (en) Data control method and related device
CN110838989A (en) Method and device for network current limiting based on token
CN114374657A (en) Data processing method and device
CN113918282A (en) Database access method and device, electronic equipment and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination