CN115438020A - Database resource scheduling method, device, equipment and medium - Google Patents
Database resource scheduling method, device, equipment and medium Download PDFInfo
- Publication number
- CN115438020A CN115438020A CN202210920192.2A CN202210920192A CN115438020A CN 115438020 A CN115438020 A CN 115438020A CN 202210920192 A CN202210920192 A CN 202210920192A CN 115438020 A CN115438020 A CN 115438020A
- Authority
- CN
- China
- Prior art keywords
- read operation
- operation request
- slave node
- processing
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a method, a device, equipment and a medium for scheduling database resources, which solve the problems of resource waste and low query efficiency of a database in the prior art. In the embodiment of the application, when a read operation request is received, whether a slave node capable of processing the read operation request exists is judged according to the maximum number of the slave nodes capable of processing the read operation request in the stored database and the number of the read operation requests currently processed by each slave node; if yes, determining a processing slave node in the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if the read operation request does not exist, the read operation request is forwarded to the main node; the phenomenon that certain slave nodes are higher in connection number and pressure and are idle is avoided, all nodes of the database are dispatched in a balanced mode, the resource utilization rate of the database is improved, the processing efficiency of the read operation request can be improved, and the data query efficiency of the database is improved.
Description
Technical Field
The present application relates to the field of database information technologies, and in particular, to a method, an apparatus, a device, and a medium for scheduling database resources.
Background
With the rapid development of the information age and the dramatic increase of the data volume of enterprises, the pressure borne by the database is also increasing, and a great amount of concurrent operations may cause the database to be paralyzed in the peak period of business. For example, when the amount of service data is large and the amount of concurrent access is large, the read-write capability of the database cannot meet the requirement.
However, in a general system, the data reading frequency is higher than the writing frequency, and therefore, problems in the prior art are mainly that scheduling of read operation requests for querying is unbalanced, all the read operation requests are sent to fixed slave nodes, so that the number of the read operation requests that the slave nodes need to process is too large, a phenomenon of too high pressure occurs, and other slave nodes are idle, so that resource waste of a database is caused, and the resource utilization rate is low.
Therefore, a method for scheduling database resources in a balanced manner is needed, so that the resource utilization rate of the database is improved.
Disclosure of Invention
The application provides a database resource scheduling method, a database resource scheduling device and a database resource scheduling medium, which are used for solving the problems of resource waste and low query efficiency of a database in the prior art.
The embodiment of the application provides the following specific technical scheme:
in a first aspect, an embodiment of the present application provides a method for scheduling database resources, where the method includes:
if the read operation request is received, judging whether a slave node capable of processing the read operation request exists according to the maximum number of the read operation requests which can be processed by each slave node in the stored database and the number of the read operation requests which are currently processed by each slave node;
if the read operation request does not exist, forwarding the read operation request to a main node;
if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In one possible implementation, the determining to process a slave node among slave nodes capable of processing the read operation request includes:
and determining the slave node with the minimum number according to the saved number of the read operation requests currently processed by each slave node, and forwarding the received read operation requests to the slave node with the minimum number.
In a possible implementation manner, before the determining whether there is a slave node capable of processing the read operation request, the method further includes:
acquiring stored historical read operation requests, and judging whether a target historical read operation request consistent with data inquired by the read operation request exists or not according to identification information of historical inquiry data carried in each historical read operation request and identification information of the data to be inquired;
if not, the subsequent step of judging whether a slave node capable of processing the read operation request exists is carried out.
In one possible embodiment, the method further comprises:
and if the target historical read operation request which is consistent with the data inquired by the read operation request exists, determining the target slave node receiving the target historical read operation request according to the stored historical slave nodes receiving the historical read operation requests, and forwarding the read operation request to the target slave node.
In a possible implementation manner, the determining whether there is a slave node capable of processing the read operation request according to the maximum number of the slave nodes capable of processing the read operation requests in the saved database and the number of the read operation requests currently being processed by each slave node includes:
determining slave nodes with normal states in the database according to the states of all the slave nodes in the stored database;
and for the slave nodes in the normal state in the database, subsequently judging whether slave node operation capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests currently processed by each slave node.
In one possible embodiment, the method further comprises:
and performing quantity increasing and updating on the quantity of the read operation requests currently processed by the nodes receiving the read operation requests.
In one possible embodiment, the method further comprises:
and if any node is detected to finish processing any read operation request, performing quantity reduction updating on the quantity of the read operation requests currently processed by the node.
In a second aspect, an embodiment of the present application further provides an apparatus for scheduling a database resource, where the apparatus includes:
the judging module is used for judging whether a slave node capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node and the number of the read operation requests which are currently processed by each slave node in the stored database if the read operation request is received;
the forwarding module is used for forwarding the read operation request to a main node if the slave node capable of processing the read operation request does not exist; if the processing slave node exists, determining the processing slave node in the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In a possible implementation manner, the forwarding module is further configured to determine, according to the stored number of read operation requests currently being processed by each slave node, the slave node with the smallest number, and forward the received read operation request to the slave node with the smallest number.
In a possible implementation manner, the determining module is further configured to obtain stored historical read operation requests, and determine whether a target historical read operation request consistent with data queried by the read operation request exists according to identification information of historical query data and identification information of the data to be queried, where the identification information is carried in each historical read operation request; if not, the subsequent step of judging whether a slave node capable of processing the read operation request exists is carried out.
In a possible implementation manner, the forwarding module is further configured to determine, according to the stored history slave node that receives each history read operation request, a target slave node that receives the target history read operation request, and forward the read operation request to the target slave node, if there is a target history read operation request that is consistent with the data queried by the read operation request.
In a possible implementation manner, the determining module is specifically configured to determine a slave node in a normal state in the database according to the saved states of the slave nodes in the database; and for the slave nodes with normal states in the database, subsequently judging whether slave node operations capable of processing the read operation requests exist according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests which are currently processed by each slave node.
In a possible embodiment, the apparatus further comprises:
and the updating module is used for increasing and updating the number of the read operation requests currently processed by the nodes receiving the read operation requests.
In a possible implementation manner, the update module is further configured to perform quantity reduction update on the number of read operation requests currently being processed by any node if it is detected that the node finishes processing any read operation request.
In a third aspect, the present application provides an electronic device, which at least includes a processor and a memory, and the processor is configured to implement the steps of the database resource scheduling method according to any one of the above when executing a computer program stored in the memory.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of any one of the above database resource scheduling methods.
In the embodiment of the application, when the read operation request is received, whether the slave node capable of processing the read operation request exists is judged according to the maximum number of the read operation requests which can be processed by each slave node in the stored database and the number of the read operation requests which are currently processed by each slave node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if the read operation request does not exist, the read operation request is forwarded to the main node; the phenomenon that certain slave nodes are high in connection number and pressure and are idle is avoided, all nodes of the database are dispatched in a balanced mode, the resource utilization rate of the database is improved, and the processing efficiency of the read operation request can be improved.
Drawings
Fig. 1 is a schematic diagram of a process of scheduling database resources according to an embodiment of the present application;
fig. 2 is a second schematic diagram illustrating a process of scheduling database resources according to an embodiment of the present application;
fig. 3 is a third schematic diagram illustrating a process of scheduling database resources according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a database resource scheduling apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
The terms "first" and "second" in the description and claims of the present application and the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the term "comprises" and any variations thereof, which are intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. The "plurality" in the present application may mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
The application provides a method, a device, equipment and a medium for scheduling database resources, wherein in the method, if a read operation request is received, whether a slave node capable of processing the read operation request exists is judged according to the maximum number of the read operation requests which can be processed by each slave node in a stored database and the number of the read operation requests which are currently processed by each slave node; if the read operation request does not exist, forwarding the read operation request to a main node; if the processing slave node exists, determining the processing slave node in the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In order to improve the resource utilization rate of the database and improve the processing efficiency of the database on the read operation request, the application provides a method, a device, equipment and a medium for scheduling the database resources.
Example 1:
fig. 1 is a schematic diagram of a process for scheduling database resources according to an embodiment of the present application, where the process includes:
s101: and if the read operation requests are received, judging whether the slave nodes capable of processing the read operation requests exist according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests which are currently processed by each slave node.
The database resource scheduling method provided by the embodiment of the application is applied to any equipment in the database. A database cluster generally includes a master node and a plurality of slave nodes, wherein the master node mainly processes write operation requests and the slave nodes process read operation requests. The device for scheduling database resources may be located on the slave node or on the master node.
Fig. 2 is a second schematic view of a process of scheduling a database resource according to an embodiment of the present application, as shown in fig. 2, a user (Actor) accesses a database through a terminal, inputs a Structured Query Language (SQL) statement carrying a service type to the terminal, and the terminal sends the SQL statement to a device performing scheduling of the database resource, that is, a self-developed gateway middleware in fig. 2. After identifying the SQL sentence carrying the business sentence input by the user, the equipment can analyze the SQL sentence by adopting a standard high-efficiency SQL grammar analyzer through a database grammar analysis module of the equipment to obtain the business type carried in the SQL sentence. The method not only reduces the possibility of errors in writing operation of the user, but also effectively improves the analysis efficiency of the SQL statement. The specific business type can be a writing type or a reading type, and if the business type is the writing type, the SQL statement is a writing operation request; if the type is read, the SQL statement is a read operation request.
Therefore, if a read operation request is received, that is, an SQL statement with a read type is received.
The device can also schedule and forward the SQL statement through a read-write separation forwarding module of the device, and specifically, if the SQL statement is a write operation request, the write operation request, namely the SQL statement with the service type as the write type, is forwarded to the host node; if the operation request is a read operation request, the read operation request, namely the SQL statement with the service type as the read type, is forwarded to the slave node.
Because the number of read operation requests is much larger than the number of write operation requests, and the number of read operation requests that each slave node can process is limited, in order to uniformly schedule each slave node of the database and improve the resource utilization rate of the database, the maximum number of read operation requests that each slave node can process in the database may be stored in advance, and the number of read operation requests that each slave node is currently processing may be obtained from each slave node.
And judging whether the slave nodes capable of processing the currently received read operation request exist according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests which are currently processed by each slave node. As long as the number of the read operation requests currently being processed by any slave node does not reach the corresponding maximum number capable of being processed, the slave node can process the currently received read operation request.
S102: if the read operation request does not exist, forwarding the read operation request to a main node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
If no slave node capable of processing the currently received read operation request exists, that is, the number of the read operation requests currently processed by all the slave nodes is full, the read operation request can be forwarded to the master node and processed by the master node; if the slave nodes capable of processing the currently received read operation request exist, determining one processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node. Specifically, the operation of forwarding the read operation request may be performed through the self-developed gateway middleware.
In the embodiment of the application, when the read operation request is received, whether the slave node capable of processing the read operation request exists is judged according to the maximum number of the slave nodes capable of processing the read operation request in the stored database and the number of the read operation requests currently processed by each slave node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if not, forwarding the read operation request to the main node; the phenomenon that certain slave nodes are high in connection number and pressure and are idle is avoided, all nodes of the database are dispatched in a balanced mode, the resource utilization rate of the database is improved, and the processing efficiency of the read operation request can be improved.
Example 2:
on the basis of the foregoing embodiments, in this embodiment, in order to improve resource utilization of a database, the determining to process a slave node from slave nodes capable of processing the read operation request includes:
and determining the slave node with the minimum number according to the saved number of the read operation requests currently processed by each slave node, and determining the slave node with the minimum number as a processing slave node.
Generally, the processing capabilities of the slave nodes configured in a database are the same, that is, the maximum number of read operation requests that can be processed by each slave node is generally the same, and at this time, only one maximum number needs to be saved, and each slave node corresponds to the maximum number.
Specifically, in the embodiment of the present application, the intelligent counting module of the device that schedules database resources may record and store the number of read operation requests currently being processed by each slave node by using the memory hash index table, and after receiving a read operation request, according to the stored number of read operation requests currently being processed by each slave node, may determine the slave node with the smallest number as a processing slave node, and forward the newly received read operation request to the processing slave node.
In the embodiment of the application, according to the number of the stored read operation requests currently processed by each slave node, the slave node with the minimum number can be determined, the newly received read operation requests are forwarded to the slave node with the minimum number, and the connection of each slave node is in a balanced state through effective scheduling of the read operation requests, so that the resource utilization rate of the database can be improved.
If the processing capacity of each slave node is different, namely the maximum number of the read operation requests which can be processed is different, in order to determine to process the slave nodes, the number of the read operation requests which can be processed by each slave node can be determined according to the maximum number of the read operation requests which can be processed by the slave node and the saved number of the read operation requests which are currently processed by the slave node, after the number of the read operation requests which can be processed by each slave node is determined, the slave node with the maximum number is determined to be the processing slave node, and if the number of the slave nodes with the maximum number is more than one, one of the slave nodes can be selected as the processing slave node.
Example 3:
in order to improve the query efficiency of the database, on the basis of the foregoing embodiments, in an embodiment of the present application, the read operation request carries identification information of data to be queried, and before determining whether there is a slave node capable of processing the read operation request, the method further includes:
acquiring stored historical read operation requests, and judging whether a target historical read operation request consistent with data inquired by the read operation request exists or not according to identification information of historical inquiry data carried in each historical read operation request and identification information of the data to be inquired;
if not, the subsequent step of judging whether a slave node capable of processing the read operation request exists is carried out.
In the embodiment of the present application, the read operation request received by the device for scheduling database resources carries identification information of data to be queried, that is, information that uniquely identifies the data to be queried, such as a name of the data to be queried and the like, which needs to be read by the read operation request. And determining the data to be queried required to be read by the read operation request according to the identification information.
The query cache module of the device can also record historical read operation requests and identification information of historical query data carried in each historical read operation request by using the memory hash index table, and can judge whether the historical query data consistent with the data to be queried inquired by the read operation request exists according to the identification information of the data to be queried and the stored identification information of the historical query data carried in each historical read operation request.
If the identification information of the historical query data which is consistent with the identification information of the data to be queried does not exist, it indicates that the historical query data which is consistent with the data to be queried does not exist, that is, the target historical read operation request which is consistent with the read operation request does not exist, and then the subsequent step of judging whether the slave node which can process the read operation request exists is performed, that is, the step of judging whether the slave node which can process the read operation request exists in embodiment 1 is performed.
On the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
if the historical query data consistent with the data to be queried exists, determining a target slave node of a target historical read operation request corresponding to the consistent historical query data according to the stored historical slave nodes receiving the historical read operation requests, and forwarding the read operation request to the target slave node.
If the historical query data consistent with the data to be queried exists, which indicates that the target historical read operation request consistent with the data queried by the read operation request exists, determining the historical read operation request carrying the historical query data, determining the historical read operation request as the target historical read operation request, determining a target slave node receiving the target historical read operation request by the query cache module of the equipment according to the stored historical slave nodes receiving the historical read operation requests, and forwarding the read operation request to the target slave node. Since the target slave node processes the same target history read operation request, the read operation request can be processed more efficiently. According to the embodiment of the application, the query cache hit module is introduced into the self-research gateway middleware, the query statement which is repeatedly executed, namely the read operation request, is directed to the same node, and the query performance of the database can be improved by fully utilizing the query cache of the database.
In the embodiment of the application, before judging whether a slave node capable of processing the read operation request exists, the stored historical read operation requests can be acquired, and whether a target historical read operation request consistent with data inquired by the read operation request exists is judged according to the identification information of the historical inquiry data carried in each historical read operation request and the identification information of the data to be inquired; if the historical read operation request exists, the target slave node receiving the target historical read operation request is determined according to the stored historical slave nodes receiving the historical read operation requests, the read operation request is forwarded to the target slave node, the processing efficiency of the read operation request is improved, and the query efficiency of the database is further improved.
Example 4:
in order to improve query efficiency of a database, on the basis of the foregoing embodiments, in an embodiment of the present application, the determining whether there is a slave node that can process a read operation request according to a maximum number of the slave nodes that can process the read operation request in a saved database and a number of the read operation requests that are currently being processed by each slave node includes:
determining the slave nodes in normal state in the database according to the state of each slave node in the stored database;
and for the slave nodes in the normal state in the database, subsequently judging whether slave node operations capable of processing the read operation requests exist according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests currently processed by each slave node.
Because the master node and the slave node of the database are not fixed, changes may occur, such as changing the master node to the slave node or changing the slave node to the master node; and the node state may be abnormal, and the received read operation request cannot be processed in time. In the embodiment of the application, a monitoring component of a device for scheduling database resources can monitor whether the topological structures of the master node and the slave nodes of the database and the states of the nodes are normal on line, report the topological structures of the master node and the slave nodes and the states of the nodes to a distributed coordination service (zookeeper) in the device, the zookeeper recombines the received information of the topological structures of the master node and the slave nodes and the states of the nodes and then sends the information to a callback function in the device, and the callback function updates and saves the states of the slave nodes in the database. And when the device receives the read operation request, determining the slave nodes in the normal state in the database according to the saved states of the slave nodes in the database. After the read operation request is forwarded to the slave node, the slave node can normally process the read operation request, and therefore query efficiency of the database is guaranteed.
And then, for the slave nodes in the normal state in the database, subsequently judging whether the slave nodes capable of processing the read operation request exist or not according to the saved maximum number of the read operation requests capable of being processed by each slave node in the database and the number of the read operation requests currently processed by each slave node, namely judging whether the slave nodes in the normal state capable of processing the read operation request exist or not.
In the embodiment of the application, the slave nodes in the normal state in the database are determined according to the saved states of the slave nodes in the database, so that the read operation request is ensured to be forwarded to the slave nodes in the normal state for processing, and the query efficiency of the database is ensured.
Example 5:
in order to improve the resource utilization rate of the database, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
and performing quantity increasing and updating on the quantity of the read operation requests currently processed by the nodes receiving the read operation requests.
In the embodiment of the present application, an intelligent counting statistics module of a device that schedules database resources may record, by using a memory hash index table, the number of read operation requests currently being processed by each slave node, and may also record the number of read operation requests currently being processed by a master node. After the received read operation requests are forwarded to any node, the number of the read operation requests currently processed by the node receiving the read operation requests is increased and updated, namely, the number of the read operation requests currently processed by the node receiving the read operation requests is increased by one in the hash index table, so that the number of the read operation requests currently processed by each slave node can be accurately obtained.
If the master node receives the read operation request, the number of the currently processed read operation requests corresponding to the master node is added by one, and if the slave node receives the read operation request, the number of the currently processed read operation requests corresponding to the slave node is added by one.
In order to improve the resource utilization rate of the database, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
and if any node is detected to finish processing any read operation request, performing quantity reduction updating on the quantity of the read operation requests currently processed by the node.
And if any node is detected to finish processing any read operation request, namely after the connection between the node and the terminal sending the read operation request is released, performing quantity reduction updating on the quantity of the read operation requests currently processed by the node. That is, in the embodiment of the present application, the intelligent counting module of the device that schedules database resources subtracts, by one, the number of read operation requests currently being processed by the node that has completed processing the read operation request in the hash index table, so as to ensure that the number of read operation requests currently being processed by each slave node can be accurately obtained.
If the master node finishes processing the read operation request, the number of the currently processed read operation requests corresponding to the master node is reduced by one, and if the slave node finishes processing the read operation request, the number of the currently processed read operation requests corresponding to the slave node is reduced by one.
In the embodiment of the application, the number of the read operation requests currently processed by the node receiving the read operation request is increased and updated, and if it is detected that any node finishes processing any read operation request, the number of the read operation requests currently processed by the node is decreased and updated, so that the number of the read operation requests currently processed by each slave node can be accurately obtained, the received read operation requests are distributed to each node of the database in a balanced manner, and the resource utilization rate of the database is improved.
The following describes a method for scheduling database resources according to an embodiment of the present application with a specific example.
Taking the above fig. 2 as an example for explanation, when identifying an SQL statement carrying a service type and input by a user, the device for scheduling database resources, i.e., the self-developed gateway middleware in fig. 2, may analyze the SQL statement by using a standard and efficient SQL syntax parser through its own database syntax parsing module to obtain the service type of the SQL statement, and if the SQL statement is a write type, the SQL statement is a write operation request; if the SQL statement is of the read type, the SQL statement is a read operation request.
Fig. 3 is a third schematic process diagram of database resource scheduling according to an embodiment of the present application, and as shown in fig. 3, if a read operation request is received, that is, an SQL statement with a service statement type of read operation is received, the processing is performed through the following steps:
s301: and determining the slave nodes with normal states in the database according to the states of the slave nodes in the stored database.
S302: and aiming at the slave nodes in the normal state in the database, judging whether the slave nodes capable of processing the currently received read operation request exist according to the maximum number of the slave nodes capable of processing the read operation request in the saved database and the number of the read operation requests currently processed by each slave node.
S303: if there is no slave node capable of processing the currently received read operation request, the read operation request is forwarded to the master node, and then the operation of S307 is executed.
S304: if the slave node capable of processing the currently received read operation request exists, the stored historical read operation requests are obtained, and whether a target historical read operation request consistent with the data inquired by the read operation request exists is judged according to the identification information of the historical inquiry data carried in each historical read operation request and the identification information of the data to be inquired.
S305: and if the target historical read operation request which is consistent with the data inquired by the read operation request exists, determining the target slave node receiving the target historical read operation request according to the stored historical slave node receiving each historical read operation request, and forwarding the read operation request to the target slave node. The operation of S307 is then performed.
S306: if no target historical read operation request consistent with the data queried by the read operation request exists, determining the slave node with the minimum number according to the saved number of the read operation requests currently processed by each slave node, and forwarding the received read operation request to the slave node with the minimum number.
S307: and performing quantity increasing update on the quantity of the operation requests currently processed by the node receiving the operation requests.
S308: and if any node is detected to finish processing any read operation request, performing quantity reduction updating on the quantity of the read operation requests currently processed by the node.
In the embodiment of the application, when a read operation request is received, whether a slave node capable of processing the read operation request exists is judged according to the maximum number of the read operation requests which can be processed by each slave node in a stored database and the number of the read operation requests which are currently processed by each slave node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if the read operation request does not exist, the read operation request is forwarded to the main node; the phenomena that the connection number of certain nodes is high, the pressure is high, and certain nodes are idle are avoided, all the nodes of the database are dispatched in a balanced mode, and the resource utilization rate of the database is improved.
On the basis of the foregoing embodiments, the present application further provides a database resource scheduling apparatus, and fig. 4 is a schematic structural diagram of the database resource scheduling apparatus provided in the embodiments of the present application, and as shown in fig. 4, the apparatus includes:
a determining module 401, configured to determine, if a read operation request is received, whether a slave node capable of processing the read operation request exists according to the maximum number of the read operation requests that can be processed by each slave node in the stored database and the number of the read operation requests that are currently processed by each slave node;
a forwarding module 402, configured to forward the read operation request to a master node if there is no slave node capable of processing the read operation request; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In a possible implementation manner, the forwarding module 402 is further configured to determine, according to the saved number of read operation requests currently being processed by each slave node, the slave node with the smallest number, and forward the received read operation request to the slave node with the smallest number.
In a possible implementation manner, the determining module 401 is further configured to obtain stored historical read operation requests, and determine whether a target historical read operation request consistent with data queried by the read operation request exists according to identification information of historical query data carried in each historical read operation request and identification information of the data to be queried; if not, the subsequent step of judging whether the slave node capable of processing the read operation request exists is carried out.
In a possible implementation manner, the forwarding module 402 is further configured to, if there are target historical read operation requests that are consistent with the data queried by the read operation request, determine, according to the stored historical slave nodes that receive the historical read operation requests, a target slave node that receives the target historical read operation request, and forward the read operation request to the target slave node.
In a possible implementation manner, the determining module 401 is specifically configured to determine, according to the saved states of the slave nodes in the database, a slave node in a normal state in the database; and for the slave nodes in the normal state in the database, subsequently judging whether slave node operation capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests currently processed by each slave node.
In a possible embodiment, the apparatus further comprises:
and the updating module is used for increasing and updating the number of the read operation requests currently processed by the nodes receiving the read operation requests.
In a possible implementation manner, the update module is further configured to perform quantity reduction update on the number of read operation requests currently being processed by any node if it is detected that the node finishes processing any read operation request.
The apparatus may be specifically deployed in a device that performs database resource scheduling, and for other functions of the device, reference is specifically made to the description of the other embodiments above.
On the basis of the foregoing embodiments, the present application further provides an electronic device, which may be the foregoing device for scheduling database resources. Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
As shown in fig. 5, includes: the system comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 are communicated with each other through the communication bus 504;
the memory 503 has stored therein a computer program which, when executed by the processor 501, causes the processor 501 to perform the steps of:
if the read operation request is received, judging whether a slave node capable of processing the read operation request exists according to the maximum number of the slave nodes capable of processing the read operation request in the stored database and the number of the read operation requests currently processed by each slave node;
if the read operation request does not exist, forwarding the read operation request to a main node;
if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In a possible implementation manner, the processor 501 is further configured to determine, according to the saved number of read operation requests currently being processed by each slave node, the slave node with the minimum number, and forward the received read operation request to the slave node with the minimum number.
In a possible implementation manner, the processor 501 is further configured to obtain stored historical read operation requests, and determine whether a target historical read operation request consistent with data queried by the read operation request exists according to identification information of historical query data and identification information of the data to be queried, where the identification information is carried in each historical read operation request; if not, the subsequent step of judging whether the slave node capable of processing the read operation request exists is carried out.
In a possible implementation manner, the processor 501 is further configured to determine, according to the saved history slave nodes receiving the history read operation requests, a target slave node receiving the target history read operation request, and forward the read operation request to the target slave node, if there is a target history read operation request that is consistent with the data queried by the read operation request.
In a possible implementation manner, the processor 501 is further configured to determine a slave node in the database, which is in a normal state, according to the saved state of each slave node in the database; and for the slave nodes with normal states in the database, subsequently judging whether slave node operations capable of processing the read operation requests exist according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests which are currently processed by each slave node.
In a possible implementation, the processor 501 is further configured to perform a quantity increase update on the number of read operation requests currently being processed by the node that receives the read operation request.
In a possible implementation, the processor 501 is further configured to, if it is detected that any node finishes processing any read operation request, perform a quantity reduction update on the number of read operation requests currently being processed by the node.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 502 is used for communication between the above-described electronic apparatus and other apparatuses.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
In the embodiment of the application, when the read operation request is received, whether the slave node capable of processing the read operation request exists is judged according to the maximum number of the read operation requests which can be processed by each slave node in the stored database and the number of the read operation requests which are currently processed by each slave node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if not, forwarding the read operation request to the main node; the phenomenon that certain slave nodes are high in connection number and pressure and are idle is avoided, all nodes of the database are dispatched in a balanced mode, the resource utilization rate of the database is improved, the processing efficiency of the read operation request can be improved, and the data query efficiency of the database is improved.
On the basis of the foregoing embodiments, the present invention provides a computer-readable storage medium, in which a computer program executable by an electronic device is stored, and when the program is run on the electronic device, the electronic device is caused to execute the following steps:
if the read operation request is received, judging whether a slave node capable of processing the read operation request exists according to the maximum number of the slave nodes capable of processing the read operation request in the stored database and the number of the read operation requests currently processed by each slave node;
if the read operation request does not exist, forwarding the read operation request to a main node;
if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
In one possible implementation, determining to process the slave node among the slave nodes capable of processing the read operation request includes: and determining the slave node with the minimum number according to the saved number of the read operation requests currently processed by each slave node, and forwarding the received read operation requests to the slave node with the minimum number.
In a possible implementation manner, obtaining stored historical read operation requests, and judging whether a target historical read operation request consistent with data queried by the read operation request exists or not according to identification information of historical query data carried in each historical read operation request and identification information of the data to be queried; if not, the subsequent step of judging whether a slave node capable of processing the read operation request exists is carried out.
In a possible implementation manner, if a target historical read operation request consistent with data queried by the read operation request exists, according to the stored historical slave node receiving each historical read operation request, the target slave node receiving the target historical read operation request is determined, and the read operation request is forwarded to the target slave node.
In a possible implementation mode, according to the saved state of each slave node in the database, the slave node with a normal state in the database is determined; and for the slave nodes in the normal state in the database, subsequently judging whether slave node operation capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests currently processed by each slave node.
In one possible implementation, the number of read operation requests currently being processed by the node receiving the read operation request is updated by increasing the number.
In a possible implementation manner, if it is detected that any node finishes processing any read operation request, the number of the read operation requests currently being processed by the node is updated in a number reduction manner.
The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memory such as floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc., optical memory such as CDs, DVDs, BDs, HVDs, etc., and semiconductor memory such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid State Disks (SSDs), etc.
In the embodiment of the application, when the read operation request is received, whether the slave node capable of processing the read operation request exists is judged according to the maximum number of the read operation requests which can be processed by each slave node in the stored database and the number of the read operation requests which are currently processed by each slave node; if the read operation request exists, determining a processing slave node from the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node; if not, forwarding the read operation request to the main node; the phenomenon that certain slave nodes are high in connection number and pressure and are idle is avoided, all nodes of the database are dispatched in a balanced mode, the resource utilization rate of the database is improved, the processing efficiency of the read operation request can be improved, and the data query efficiency of the database is improved.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method for scheduling database resources, the method comprising:
if the read operation request is received, judging whether a slave node capable of processing the read operation request exists according to the maximum number of the read operation requests which can be processed by each slave node in the stored database and the number of the read operation requests which are currently processed by each slave node;
if the read operation request does not exist, forwarding the read operation request to a main node;
if the processing slave node exists, determining the processing slave node in the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
2. The method of claim 1, wherein determining to process a slave node among slave nodes capable of processing the read operation request comprises:
and determining the slave node with the minimum number according to the saved number of the read operation requests currently processed by each slave node, and forwarding the received read operation requests to the slave node with the minimum number.
3. The method according to claim 1, wherein the read operation request carries identification information of data to be queried, and before the determining whether there is a slave node capable of processing the read operation request, the method further comprises:
acquiring stored historical read operation requests, and judging whether a target historical read operation request consistent with data inquired by the read operation request exists or not according to identification information of historical inquiry data carried in each historical read operation request and identification information of the data to be inquired;
if not, the subsequent step of judging whether a slave node capable of processing the read operation request exists is carried out.
4. The method of claim 3, further comprising:
and if the target historical read operation request which is consistent with the data inquired by the read operation request exists, determining the target slave node receiving the target historical read operation request according to the stored historical slave nodes receiving the historical read operation requests, and forwarding the read operation request to the target slave node.
5. The method of claim 1, wherein the determining whether there is a slave node capable of processing the read operation request according to the maximum number of the slave nodes capable of processing the read operation requests in the saved database and the number of the read operation requests currently being processed by each slave node comprises:
determining slave nodes with normal states in the database according to the states of all the slave nodes in the stored database;
and for the slave nodes in the normal state in the database, subsequently judging whether slave node operation capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node in the saved database and the number of the read operation requests currently processed by each slave node.
6. The method of any of claims 1-5, further comprising:
and performing quantity increasing and updating on the quantity of the read operation requests currently processed by the nodes receiving the read operation requests.
7. The method according to any one of claims 1-5, further comprising:
and if any node is detected to finish processing any read operation request, performing quantity reduction updating on the quantity of the read operation requests currently processed by the node.
8. An apparatus for scheduling database resources, the apparatus comprising:
the judging module is used for judging whether a slave node capable of processing the read operation request exists or not according to the maximum number of the read operation requests which can be processed by each slave node and the number of the read operation requests which are currently processed by each slave node in the stored database if the read operation request is received;
the forwarding module is used for forwarding the read operation request to a main node if the slave node capable of processing the read operation request does not exist; if the processing slave node exists, determining the processing slave node in the slave nodes capable of processing the read operation request, and forwarding the read operation request to the processing slave node.
9. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being configured to implement the steps of the database resource scheduling method according to any of claims 1-7 when executing a computer program stored in the memory.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the database resource scheduling method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210920192.2A CN115438020A (en) | 2022-08-02 | 2022-08-02 | Database resource scheduling method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210920192.2A CN115438020A (en) | 2022-08-02 | 2022-08-02 | Database resource scheduling method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115438020A true CN115438020A (en) | 2022-12-06 |
Family
ID=84242464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210920192.2A Pending CN115438020A (en) | 2022-08-02 | 2022-08-02 | Database resource scheduling method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115438020A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149885A (en) * | 2023-10-31 | 2023-12-01 | 天津神舟通用数据技术有限公司 | Read-write separation implementation method based on database master node forwarding |
-
2022
- 2022-08-02 CN CN202210920192.2A patent/CN115438020A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149885A (en) * | 2023-10-31 | 2023-12-01 | 天津神舟通用数据技术有限公司 | Read-write separation implementation method based on database master node forwarding |
| CN117149885B (en) * | 2023-10-31 | 2024-01-26 | 天津神舟通用数据技术有限公司 | Read-write separation implementation method based on database master node forwarding |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112597153B (en) | Block chain-based data storage method, device and storage medium | |
| US20170220647A1 (en) | Pluggable architecture for embedding analytics in clustered in-memory databases | |
| CN110445828B (en) | Data distributed processing method based on Redis and related equipment thereof | |
| CN109271435A (en) | A kind of data pick-up method and system for supporting breakpoint transmission | |
| CN111767009A (en) | Disk cleaning method, device and equipment | |
| CN111651595A (en) | Abnormal log processing method and device | |
| CN115002143A (en) | Node election method and device, storage medium and electronic equipment | |
| CN112988679A (en) | Log collection control method and device, storage medium and server | |
| CN110069217B (en) | Data storage method and device | |
| CN115438020A (en) | Database resource scheduling method, device, equipment and medium | |
| CN111224814B (en) | Message processing method and device | |
| CN111125240B (en) | Distributed transaction realization method and device, electronic equipment and storage medium | |
| CN110599004A (en) | Risk control method, equipment, medium and device | |
| CN109992408B (en) | Resource allocation method, device, electronic equipment and storage medium | |
| CN110298677B (en) | Cloud computing resource charging method and device, electronic equipment and storage medium | |
| CN112241362A (en) | Test method, test device, server and storage medium | |
| CN107689969B (en) | Method and device for determining cache strategy | |
| CN109976896B (en) | Service re-ranking processing method and device | |
| CN112711606A (en) | Database access method and device, computer equipment and storage medium | |
| CN117130979A (en) | Service resource migration method and device and electronic equipment | |
| CN111078418A (en) | Operation synchronization method and device, electronic equipment and computer readable storage medium | |
| CN116151631A (en) | Service decision processing system, service decision processing method and device | |
| CN114936187A (en) | Data file processing method, device, equipment and storage medium | |
| CN114398178A (en) | Task execution method and device and electronic equipment | |
| CN109426505B (en) | Software installation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |