CN113297229A - Method for routing read request and feedback message, respective device and database - Google Patents
Method for routing read request and feedback message, respective device and database Download PDFInfo
- Publication number
- CN113297229A CN113297229A CN202010581310.2A CN202010581310A CN113297229A CN 113297229 A CN113297229 A CN 113297229A CN 202010581310 A CN202010581310 A CN 202010581310A CN 113297229 A CN113297229 A CN 113297229A
- Authority
- CN
- China
- Prior art keywords
- database
- log
- message
- site information
- update
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012216 screening Methods 0.000 claims abstract description 32
- 230000004044 response Effects 0.000 claims abstract description 18
- 238000001914 filtration Methods 0.000 claims description 2
- 238000000926 separation method Methods 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 18
- 238000004891 communication Methods 0.000 description 6
- 230000000977 initiatory effect Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the specification provides methods for routing a read request and a feedback message, respective devices and databases. The method for routing the read request is applied to a read-write separation database, and comprises the following steps: receiving a message sent by a master database, wherein the message carries log site information corresponding to an update in a master database log file, so that the log site information can be obtained from the message, comparing the log site information with current log site information of each slave database log file in response to receiving a read request in a session corresponding to the update to obtain a comparison result, screening out a database which is finished with the update according to the comparison result, and routing the read request to the screened database.
Description
Technical Field
The embodiment of the specification relates to the technical field of databases, in particular to a method for routing read requests. One or more embodiments of the present specification also relate to an apparatus for routing read requests, a method for feeding back messages, an apparatus for feeding back messages, a database, a computing device, and a computer-readable storage medium.
Background
To improve performance, access pressure is often distributed by reading and writing separate databases. In the read-write separated database, a master database is responsible for read-write operation, data is synchronized to a slave database from the master database, the slave database is responsible for read operation, and an agent layer is responsible for routing a write request to the master database and routing a read request to the master database and the slave database.
However, there is a certain delay in the data synchronization process, and if the route is not proper, the client initiates a read request within a very short time after initiating a write request, and the latest data cannot be read.
Disclosure of Invention
In view of this, the embodiments of the present specification provide a method for routing a read request. One or more embodiments of the present disclosure relate to an apparatus for routing a read request, a method for feeding back a packet, an apparatus for feeding back a packet, a database, a computing device, and a computer-readable storage medium, so as to solve technical shortcomings in the prior art.
According to a first aspect of the embodiments of the present specification, there is provided a method for routing a read request, which is applied to a read-write separated database, and includes: receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file; acquiring the log site information from the message; responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
Optionally, the receiving the message sent by the master database includes: and receiving a ReadyForQuery message sent by the main database after the updating is finished.
Optionally, the method further comprises: erasing the log site information carried in the message; and returning the message with the erased log site information to the client.
Optionally, the method further comprises: recording the log site information in a session corresponding to the update; in response to receiving a read request in the session, the log location information is read from the session to compare the log location information with current log location information of each slave database log file.
Optionally, the comparing, in response to receiving the read request in the session corresponding to the update, the log location information with current log location information of each slave database log file, and obtaining a comparison result includes: sending a request for acquiring current log site information of the slave database log files to each slave database which establishes connection by taking a preset time length as a period; receiving current log site information fed back by each slave database respectively in response to the request for acquiring the current log site information of the slave database log file; and responding to the reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information fed back from the database latest to obtain a comparison result.
Optionally, the filtering out the databases that have completed the update according to the comparison result includes: and screening out the database of which the LSN is more than or equal to the LSN corresponding to the update in the master library log file as the database which finishes the update according to the comparison result.
According to a second aspect of the embodiments of the present specification, there is provided an apparatus for routing a read request, configured in a read-write separated database, including: and the message receiving module is configured to receive a message sent by the master database, wherein the message carries log site information corresponding to the update in the master database log file. And the acquisition site module is configured to acquire the log site information from the message. And the comparison module is configured to respond to the reading request received in the session corresponding to the updating, and compare the log site information with the current log site information of each slave database log file to obtain a comparison result. And the screening module is configured to screen out the database which is updated according to the comparison result. A routing module configured to route the read request to the screened database.
According to a third aspect of the embodiments of the present specification, there is provided a method for feeding back a packet, which is applied to a master database in a read-write separated database, and includes: responding to the generated message after the update execution of the master database is finished, and writing log site information corresponding to the update in a master database log file into the message; returning a message carrying the log site information to an agent layer; the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
According to a fourth aspect of the embodiments of the present specification, there is provided an apparatus for feeding back a packet, configured in a master database in a read-write separated database, including: and the site writing module is configured to respond to the main database to generate a message after the update execution is finished, and write log site information corresponding to the update in a log file of the main database in the message. And the message returning module is configured to return the message carrying the log site information to the proxy layer. The message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
According to a fifth aspect of embodiments herein, there is provided a database comprising: the master database is configured to receive a write request sent by the proxy layer, execute the write request, respond to a message generated after the update corresponding to the write request is executed, write log site information corresponding to the update in a master database log file in the message, and return the message carrying the log site information to the proxy layer. The proxy layer is configured to receive a write request of a client; routing a write request of a client to the master database; receiving the message fed back by the main database; acquiring the log site information from the message; responding to a read request received in the session of the write request, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the updated database corresponding to the write request according to the comparison result; and routing the read request to the screened database. A plurality of slave databases configured to transmit current log location information of slave database log files to the agent layer.
According to a sixth aspect of embodiments herein, there is provided a computing device comprising: a memory and a processor; the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to: receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file; acquiring the log site information from the message; responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
According to a seventh aspect of embodiments herein, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the method of routing read requests according to any one of the embodiments herein.
According to an eighth aspect of embodiments herein, there is provided a computing device comprising: a memory and a processor; the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to: responding to a message generated after the updating execution of the master database is finished, and writing log site information corresponding to the updating in a master database log file into the message; returning a message carrying the log site information to an agent layer; the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
According to a ninth aspect of the embodiments of the present specification, there is provided a computer-readable storage medium storing computer instructions, which when executed by a processor, implement the steps of the method for feeding back a message according to the embodiments of the present specification.
In one aspect, an embodiment of the present specification provides a method for routing a read request, where the method can receive a message carrying log location information corresponding to an update in a master library log file, so as to accurately obtain the log location information of the master library after the update corresponding to the write request is completed. And when a read request is received in a session corresponding to the update, comparing the log site information corresponding to the update with the current log site information of each slave database log file, screening out the updated database corresponding to the write request, and routing the read request to the screened database, so that subsequent read requests after the write request in the session can be routed to the updated database, correct routing is realized, and read-write separation read consistency is really realized.
In another aspect, an embodiment of this specification provides a method for feeding back a message, in which a master database of the method adds log site information corresponding to an update in a master database log file to a message generated after the update is completed, and sends the message to an agent layer, so that the agent layer can accurately obtain the log site information of the master library after the update corresponding to the write request is completed, compare the log site information corresponding to the update in the master library log file with the current log site information of each slave database log file, screen out the updated database corresponding to the write request, route the read request to the screened database, therefore, the subsequent read request after the write request in the session can be routed to the updated database by the proxy layer, so that correct routing is realized, and read consistency of read-write separation is really realized.
Drawings
FIG. 1 is a flow diagram of a method of routing read requests provided by one embodiment of the present description;
fig. 2 is a schematic diagram of a ReadyForQuery message provided in an embodiment of the present specification.
FIG. 3 is a diagram illustrating a process for a client to initiate a write request according to an embodiment of the present disclosure;
FIG. 4 is a diagram illustrating a process for a client to initiate a read request according to an embodiment of the present disclosure;
FIG. 5 is a block diagram illustrating an apparatus for routing a read request according to an embodiment of the present disclosure;
FIG. 6 is a block diagram of an apparatus for routing read requests according to another embodiment of the present disclosure;
fig. 7 is a flowchart of a method for feeding back a message according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an apparatus for feeding back a packet according to an embodiment of the present disclosure;
FIG. 9 is a block diagram of a database provided in one embodiment of the present disclosure;
fig. 10 is a block diagram of a computing device according to an embodiment of the present disclosure.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
First, the noun terms to which one or more embodiments of the present specification relate are explained.
The LSN (Log sequence number, Log serial number) is used to identify the position of the Log file recorded in the Log file, and may also be used as a location point information for updating once.
Read-write separation, a technique in which write requests are routed to the master library and read requests are routed to the slave library, thereby alleviating the pressure on the master library.
In the present specification, a method for routing a read request is provided, and the present specification relates to an apparatus for routing a read request, a method for feeding back a packet, an apparatus for feeding back a packet, a database, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments one by one.
FIG. 1 illustrates a flow diagram of a method of routing read requests according to one embodiment of the present description. The method of routing read requests may be applied to read and write separate databases. For example, the method may include steps 102 through 110.
Step 102: and receiving a message sent by the master database, wherein the message carries log site information corresponding to the update in the master database log file.
The type of the message is not limited, as long as the message is sent by the master database after the update corresponding to the write request is completed. The current log location information is the latest location information that is continuously updated as transactions occur and entries are written to the log file, and may be, for example, the latest LSN value.
Step 104: and acquiring the log site information from the message.
Step 106: and responding to the reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result.
The current log site information of each slave library log file may be obtained by the agent layer periodically from the slave database for a preset time period, or obtained by sending a request to each slave database when a read request is received.
Step 108: and screening out the database which is updated according to the comparison result.
For example, in the case that the log location information is LSN, a database whose LSN is greater than or equal to the LSN corresponding to the update in the master library log file may be screened as the database that has completed the update according to the comparison result. It is understood that LSN represents the logical sequence of data writing to the database, and represents the specific location of the operation log corresponding to the data in the log file, which is itself a series of numbers, and can be compared in size.
For another example, in addition to screening out the slave database as the route candidate node, the master database may be used as the route candidate node.
For another example, in order to reduce the pressure of the master database, the master database may be selected as the route candidate node only in a case where there is no slave database that has completed the update corresponding to the write request. Specifically, the method comprises the following steps: if the updated slave database corresponding to the write request is determined to exist according to the comparison result, the updated slave database corresponding to the write request is used as the screened database; and if the fact that the updated slave database corresponding to the write request does not exist is determined according to the comparison result, the master database is used as the screened database.
Step 110: and routing the read request to the screened database.
Therefore, the method utilizes the message generated by the master database after the update corresponding to the write request is completed to carry the log site information corresponding to the update in the master database log file, so that the log site information of the master database after the update corresponding to the write request is completed can be accurately obtained by receiving the message. And when a read request is received in the session of the write request subsequently, comparing the log site information corresponding to the update in the master database log file with the current log site information of each slave database log file, screening out the updated database corresponding to the write request, and routing the read request to the screened database.
For example, the method for routing read requests described in the embodiments of the present specification may be applied to a proxy layer for reading and writing separate databases. The proxy layer may belong to any type of read-write separated database. For example, the method for routing a read request provided by the embodiments of the present specification may be applied to the Proxy layer Proxy. According to the embodiment, in the PGSQL database, the Proxy may route the read request to the database that has completed updating according to the log site information corresponding to the updating in the master library log file carried in the packet, so as to implement correct routing and really implement read consistency of read-write separation.
In one or more embodiments of the present disclosure, the current Log site information may be a LSN (Log sequence number). And the LSN is used for identifying the position of the XLOG, namely the position of the log file recorded in the log file, and updating the corresponding current log site information at one time. The LSN is directly related to the update of the log file, and the LSN is continuously increased along with the occurrence of a transaction and the writing of an entry into the log file, so that the LSN can accurately represent the current log site information of the log file, a slave database to which a read request needs to be routed is determined according to the comparison of the LSNs of a master database and a slave database, correct routing can be realized, and the read consistency of read-write separation is really realized.
The embodiment of the present specification is not limited to the type of the message carrying the current log site information of the master database log file, as long as the message is sent by the master database after the update corresponding to the write request is completed. For example, the result set message may be generated after the master database performs the write operation corresponding to the write request, or may be a non-result set message transmitted between two write operations, such as a message generated by the master database in response to the read request or an operation of setting a session variable, or a notification message actively transmitted to the client by the master database.
Specifically, in the implementation mode that the method is applied to the PGSQL database proxy layer, the packet may be a ReadyForQuery packet. The ReadyForQuery is an identifier for finishing the processing of a request, and is the last message in a result set message after the execution of the request. In PGSQL server, after each request is processed, a ReadyForQuery must be returned, which indicates that the current request is completely processed and that the next request can be received. For example, according to this embodiment, a patch program may be added to a PGSQL kernel, that is, a source code item of a PGSQL database, so that the PGSQL server carries the latest LSN of the current server when returning the ReadyForQuery message. Therefore, the Proxy layer Proxy can accurately acquire the latest LSN value of the master database after the current update request is executed. Subsequent read requests in the same session can read the data corresponding to the update as long as the data is routed to the slave database having a LSN value greater than or equal to the LSN value corresponding to the update request. Therefore, the ReadyForQuery carries the LSN, so that the Proxy can accurately acquire the LSN corresponding to the current write request, the subsequent read request can obtain a correct route, and the read-write separation read consistency is really realized.
Considering that a message carries current log site information of a master library log file, the message may not be in a communication protocol between a database server and a client, and in order to avoid error reporting by the client, the method for routing a read request provided in the embodiment of the present specification may further erase the log site information carried in the message; and returning the message with the erased log site information to the client. Therefore, the message returned to the client by the agent layer applying the method is a message conforming to the communication protocol of the database server and the client, and the error report of the client is avoided. It can be understood that if the message is allowed to carry the log site information in the communication protocol between the database server and the client, the message carrying the log site information can be directly returned to the client.
In one or more embodiments of the present disclosure, in order to improve routing efficiency, log location information corresponding to the update in the master library log file is recorded in the write request, that is, a session corresponding to the update, so that when a read request is received in the session, log location information corresponding to the update in the master library log file can be directly read from the session, so as to compare the log location information corresponding to the update in the master library log file with current log location information of each slave database log file. Because the log site information corresponding to the update in the master library log file is directly recorded in the client session state, when comparison is needed, the log site information is directly read from the session state, and the routing efficiency is effectively improved. It is understood that there may be multiple sessions at the proxy layer. E.g. two sessions as a and B. The log location information of Session a corresponding to the last update of the master pool is lsn _ a. Lsn _ a is recorded in Session a state. The log location information of Session B corresponding to the last update of the master pool is lsn _ B. Lsn _ B is recorded in Session B's state.
In one or more embodiments of the present disclosure, in order to effectively screen out an updated database that has completed the corresponding update of the write request, the current log location information of each slave database is obtained by sending a request for obtaining the current log location information of the slave database log file to each slave database that has established a connection with a preset time period as a cycle. Specifically, for example, the comparing, in response to receiving a read request in a session of the write request, log location information corresponding to the update in the master library log file with current log location information of each slave database log file, and obtaining a comparison result includes: sending a request for acquiring current log site information of the slave database log files to each slave database which establishes connection by taking a preset time length as a period; receiving current log site information fed back by each slave database respectively in response to the request for acquiring the current log site information of the slave database log file; and responding to the reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information fed back from the database latest to obtain a comparison result. In the embodiment, the current log site information of each slave database is obtained by sending the request to each slave database by taking the preset time length as a period, so that the obtained current log site information of the slave database is continuously updated, the latest current log site information is used for comparison, the synchronization condition of each slave database can be accurately reflected, and the updated database corresponding to the write request is effectively screened out.
In order to make the method for routing a read request provided in the foregoing embodiment of the present specification easier to understand, the following describes in detail by taking the Proxy layer Proxy of PGSQL as an example.
For example, as shown in fig. 2, the schematic diagram of the ReadyForQuery message is shown. The method provided by the embodiment of the specification enables the current server to carry the latest LSN value when the ReadyForQuery is returned each time by modifying the PGSQL server kernel code. The length of the LSN value is 8 bytes, and the LSN value follows the content of the ReadyForQuery original message. As shown in fig. 2, in the ReadyForQuery, the first bit is type, and the message type identifier; the second bit to the fifth bit are packet len, packet length; the sixth bit is transaction status, transaction status code; the seventh bit is followed by an 8 byte LSN.
Fig. 3 is a schematic diagram illustrating a process of a client initiating a write request. The Proxy to which the method for routing a read request provided in the embodiments of the present specification is applied may receive an update write request initiated by a client. The Proxy routes the write request behind the PG master library. The master library returns a ReadyForQuery carrying LSNs such as "LSN 2". Proxy acquires "lsn 2" and records it in session state. And then returning the ReadyForQuery after the LSN information is erased to the client. If the session has recorded an LSN such as "LSN 1" received after the client's last write request, then "LSN 1" is replaced with the newly acquired "LSN 2" in the session state. In fig. 3, "mxs _ LSN" represents a session state for recording LSN information; "PG master" represents the master database; "PG Slave" denotes a Slave database; "reproduction" indicates data synchronization.
Fig. 4 is a schematic diagram illustrating a process of a client initiating a read request. And the Proxy receives a select read request sent by the client in the session of the write request. And (4) the Proxy reads the LSN information of the master library from the session state and compares the LSN information of the master library with the LSN of each server. Servers with LSN values equal to or greater than the last update value of the master pool are selected for routing. As shown in fig. 4, since the current LSN of "PG slave 1" is greater than or equal to the LSN information of the master library, "PG slave 1" may become a routing candidate node, and since the current LSN of "PG slave 2" is less than the LSN information of the master library, "PG slave 2" may not become a routing candidate node, thereby implementing read-write separation and read consistency.
In the above fig. 2, 3 and 4, "T", "D", "C" and "Z" are used to describe the message transmitted between the Proxy and the client or the server, where "T" represents "RowDescription"; "D" means "DataRow"; "C" means "CommandComplete"; "Z" means "ReadyForQuery".
In addition, the Proxy can be always connected with each slave database after being started, and is disconnected when the PGSQL server is abnormal and reconnected when the PGSQL server is recovered. The Proxy sends a request every preset time, for example, 10ms, to obtain the LSN of the current PGSQL server, or the PGSQL server may also send the LSN to the Proxy actively, which is not limited in this specification. The Proxy can achieve the purpose of reading consistency as long as routing the next reading request to the PGSQL server of the LSN corresponding to the last updating operation, wherein the LSN is already larger than the LSN corresponding to the last updating operation.
Corresponding to the above method embodiment for routing a read request, this specification further provides an embodiment of an apparatus for routing a read request, and fig. 5 illustrates a schematic structural diagram of an apparatus for routing a read request provided in an embodiment of this specification. The device may be configured to read and write separate databases. As shown in fig. 5, the apparatus includes: a message receiving module 502, a site obtaining module 504, a comparing module 506, a screening module 508 and a routing module 510.
The message receiving module 502 may be configured to receive a message sent by the master database, where the message carries log site information corresponding to an update in a master database log file.
The get location module 504 may be configured to get the log location information from the packet.
The comparison module 506 may be configured to compare the log location information with current log location information of each slave database log file in response to receiving a read request in a session corresponding to the update, and obtain a comparison result.
The screening module 508 may be configured to screen out the database that has completed the update according to the comparison result.
The routing module 510 may be configured to route the read request to the screened database.
The device utilizes the message generated by the master database after the update corresponding to the write request is completed to carry the log site information corresponding to the update in the master database log file, so that the device can accurately obtain the log site information of the master database after the update corresponding to the write request is completed by receiving the message. And when a read request is received in the session of the write request subsequently, comparing the log site information corresponding to the update in the master database log file with the current log site information of each slave database log file, screening out the updated database corresponding to the write request, and routing the read request to the screened database.
In one or more embodiments of the present disclosure, the apparatus for routing a read request may be configured in a Proxy layer Proxy of PGSQL. The current log location information may be an LSN. In this embodiment, for example, the message receiving module 502 may be configured to receive a ReadyForQuery message sent by the master database after the update corresponding to the write request is completed. In the PGSQL server, after each request is processed, a ReadyForQuery must be returned to indicate that the current request is completely processed, and the next request can be received, so that the Proxy can accurately acquire the LSN corresponding to the current write request by carrying the LSN, so that the subsequent read request can obtain a correct route, and read-write separation and read consistency is really realized.
In one or more embodiments of the present disclosure, as shown in fig. 6, the apparatus may further include: the site deleting module 512 may be configured to erase the log site information carried in the message. The message sending module 514 may be configured to return the message with the erased log site information to the client. According to the embodiment, the message returned to the client is a message conforming to the communication protocol of the database server and the client, so that the error report of the client is avoided.
In one or more embodiments of the present disclosure, as shown in fig. 6, the apparatus may further include: a location recording module 516 configured to record log location information corresponding to the update in the master library log file in a session of the write request. A location extraction module 518 configured to, in response to receiving a read request in the session, read the log location information from the session so as to trigger the comparison module 506 to compare the log location information with current log location information of each slave database log file. In the embodiment, the log site information corresponding to the update in the master library log file is directly recorded in the client session state, and when comparison is needed, the log site information is directly read from the session state, so that the routing efficiency is effectively improved.
In one or more embodiments of the present disclosure, as shown in fig. 6, the comparing module 506 may include: the request slave repository site sub-module 5062 may be configured to send a request for obtaining current log site information of the slave repository log file to each slave database to which the connection is established, at a preset time period. The get slave repository site sub-module 5064 may be configured to receive current log site information fed back by each slave database in response to the request to get current log site information of the slave repository log file. The comparison sub-module 5066 may be configured to, in response to receiving a read request in a session corresponding to the update, compare the log location information with current log location information that is newly fed back from the database, and obtain a comparison result. In the embodiment, the current log site information of each slave database is obtained by sending the request to each slave database by taking the preset time length as a period, so that the obtained current log site information of the slave database is continuously updated, the latest current log site information is used for comparison, the synchronization condition of each slave database can be accurately reflected, and the updated database corresponding to the write request is effectively screened out.
In one or more embodiments of the present description, the screening module 508 may be configured to screen out, as the database having completed the update, a database having an LSN greater than or equal to the LSN corresponding to the update in the master library log file according to the comparison result.
The above is an illustrative scheme of an apparatus for routing a read request according to this embodiment. It should be noted that the technical solution of the apparatus for routing a read request belongs to the same concept as the technical solution of the method for routing a read request, and details that are not described in detail in the technical solution of the apparatus for routing a read request can be referred to the description of the technical solution of the method for routing a read request.
Fig. 7 is a flowchart illustrating a method for feeding back a message according to an embodiment of the present disclosure. The method for feeding back the message can be applied to a main database in a read-write separated database. For example, the method may include steps 702 through 704.
Step 702: and responding to the main database to generate a message after the update corresponding to the write request is executed, and writing log site information corresponding to the update in a log file of the main database in the message.
Step 704: and returning the message carrying the log site information to the proxy layer.
The message is used for enabling the agent layer to acquire log site information corresponding to the update in the master database log file from the message, receive a read request in a session corresponding to the update, and compare the log site information with current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
Therefore, the method utilizes the message generated by the master database after the update corresponding to the write request is completed to carry the log site information corresponding to the update in the master database log file, so that the agent layer can accurately obtain the log site information of the master database after the update corresponding to the write request is completed by receiving the message. And when the agent layer receives the read request in the session of the write request, the log site information corresponding to the update in the master database log file is compared with the current log site information of each slave database log file, the updated database corresponding to the write request is screened out, and the read request is routed to the screened database.
Corresponding to the above method embodiment for feeding back a packet, this specification further provides an embodiment of a device for feeding back a packet, and fig. 8 illustrates a schematic structural diagram of a device for feeding back a packet according to an embodiment of this specification. As shown in fig. 8, the apparatus includes: a site writing module 802 and a message returning module 804.
The site writing module 802 may be configured to, in response to the master database generating a message after the update is completed, write log site information corresponding to the update in a master database log file in the message.
The message returning module 804 may be configured to return a message carrying the log site information to the proxy layer.
The message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
Therefore, the device utilizes the message generated by the master database after the update corresponding to the write request is completed to carry the log site information corresponding to the update in the master database log file, so that the agent layer can accurately obtain the log site information of the master database after the update corresponding to the write request is completed by receiving the message. And when the agent layer receives the read request in the session of the write request, the log site information corresponding to the update in the master database log file is compared with the current log site information of each slave database log file, the updated database corresponding to the write request is screened out, and the read request is routed to the screened database.
The above is a schematic scheme of a device for feeding back a packet in this embodiment. It should be noted that the technical solution of the apparatus for feeding back a packet belongs to the same concept as the technical solution of the method for feeding back a packet, and details of the technical solution of the apparatus for feeding back a packet, which are not described in detail, can be referred to the description of the technical solution of the method for feeding back a packet.
Fig. 9 is a schematic diagram illustrating a structure of a database provided according to an embodiment of the present disclosure. For example, the database may include: a master database 902, a proxy layer 904, a plurality of slave databases 906.
The master database 902 may be configured to receive a write request sent by the proxy layer, execute the write request, generate a packet in response to that an update corresponding to the write request is executed, write log location information corresponding to the update in a master database log file in the packet, and return the packet carrying the log location information to the proxy layer.
The proxy layer 904 may be configured to receive a write request of a client; routing a write request of a client to the master database; receiving the message fed back by the main database; acquiring the log site information from the message; responding to a read request received in the session of the write request, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the updated database corresponding to the write request according to the comparison result; and routing the read request to the screened database.
The slave database 906 may be configured to send current log location information of the slave database log file to the proxy layer.
Because the message generated by the main database in the database system after the update corresponding to the write request is completed carries the log site information corresponding to the update in the log file of the main database, the agent layer can accurately obtain the log site information of the main database after the update corresponding to the write request is completed by receiving the message. And when the agent layer receives the read request in the session of the write request, the log site information corresponding to the update in the master database log file is compared with the current log site information of each slave database log file, the updated database corresponding to the write request is screened out, and the read request is routed to the screened database.
FIG. 10 illustrates a block diagram of a computing device 1000 provided in accordance with one embodiment of the present description. The components of the computing device 1000 include, but are not limited to, memory 1010 and a processor 1020. The processor 1020 is coupled to the memory 1010 via a bus 1030 and the database 1050 is used to store data.
Computing device 1000 also includes access device 1040, access device 1040 enabling computing device 1000 to communicate via one or more networks 1060. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 1040 may include one or more of any type of network interface, e.g., a Network Interface Card (NIC), wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 1000 and other components not shown in FIG. 10 may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 10 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.
Computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 1000 may also be a mobile or stationary server.
In one aspect, the processor 1020 may be configured to execute the following computer-executable instructions:
receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file;
acquiring the log site information from the message;
responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result;
screening out the database which is updated according to the comparison result;
and routing the read request to the screened database.
The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above method for routing a read request belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method for routing a read request.
In another aspect, the processor 1020 may be configured to execute the following computer-executable instructions:
responding to a message generated after the updating execution of the master database is finished, and writing log site information corresponding to the updating in a master database log file into the message;
returning a message carrying the log site information to an agent layer;
the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above method for feeding back a packet belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method for feeding back a packet.
An embodiment of the present specification also provides a computer readable storage medium storing computer instructions that, when executed by a processor, are operable to:
receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file;
acquiring the log site information from the message;
responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result;
screening out the database which is updated according to the comparison result;
and routing the read request to the screened database.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above method for routing a read request, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above method for routing a read request.
An embodiment of the present specification also provides another computer-readable storage medium storing computer instructions that, when executed by a processor, are operable to:
responding to a message generated after the updating execution of the master database is finished, and writing log site information corresponding to the updating in a master database log file into the message;
returning a message carrying the log site information to an agent layer;
the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the method for feeding back a packet belong to the same concept, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the method for feeding back a packet.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts, but those skilled in the art should understand that the present embodiment is not limited by the described acts, because some steps may be performed in other sequences or simultaneously according to the present embodiment. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for an embodiment of the specification.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the embodiments. The specification is limited only by the claims and their full scope and equivalents.
Claims (14)
1. A method for routing read requests, applied to a read-write separated database, includes:
receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file;
acquiring the log site information from the message;
responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result;
screening out the database which is updated according to the comparison result;
and routing the read request to the screened database.
2. The method of claim 1, wherein receiving the message sent by the master database comprises: and receiving a ReadyForQuery message sent by the main database after the updating is finished.
3. The method of claim 1, further comprising:
erasing the log site information carried in the message;
and returning the message with the erased log site information to the client.
4. The method of claim 1, further comprising:
recording the log site information in a session corresponding to the update;
in response to receiving a read request in the session, the log location information is read from the session to compare the log location information with current log location information of each slave database log file.
5. The method of claim 1, wherein comparing the log location information with current log location information of each slave database log file in response to receiving a read request in a session corresponding to the update comprises:
sending a request for acquiring current log site information of the slave database log files to each slave database which establishes connection by taking a preset time length as a period;
receiving current log site information fed back by each slave database respectively in response to the request for acquiring the current log site information of the slave database log file;
and responding to the reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information fed back from the database latest to obtain a comparison result.
6. The method of claim 1, wherein filtering out databases that have completed the update based on the comparison comprises:
and screening out the database of which the LSN is more than or equal to the LSN corresponding to the update in the master library log file as the database which finishes the update according to the comparison result.
7. An apparatus for routing read requests configured to read from and write to separate databases, comprising:
the message receiving module is configured to receive a message sent by a master database, wherein the message carries log site information corresponding to the update in a master database log file;
an obtaining location module configured to obtain the log location information from the packet;
a comparison module configured to compare the log location information with current log location information of each slave database log file in response to receiving a read request in a session corresponding to the update, to obtain a comparison result;
a screening module configured to screen out the database that has completed the update according to the comparison result;
a routing module configured to route the read request to the screened database.
8. A method for feeding back message is applied to a main database in a read-write separated database, and comprises the following steps:
responding to the generated message after the update execution of the master database is finished, and writing log site information corresponding to the update in a master database log file into the message;
returning a message carrying the log site information to an agent layer;
the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
9. A device for feeding back message is configured in a main database in a read-write separated database, and comprises:
the site writing module is configured to respond to a message generated by the master database after the update execution is finished, and write log site information corresponding to the update in a master database log file into the message;
the message returning module is configured to return a message carrying the log site information to the proxy layer;
the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
10. A database, comprising:
the master database is configured to receive a write request sent by the proxy layer, execute the write request, respond to a message generated after the update corresponding to the write request is executed, write log site information corresponding to the update in a master database log file in the message, and return the message carrying the log site information to the proxy layer;
the proxy layer is configured to receive a write request of a client; routing a write request of a client to the master database; receiving the message fed back by the main database; acquiring the log site information from the message; responding to a read request received in the session of the write request, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the updated database corresponding to the write request according to the comparison result; routing the read request to the screened database;
a plurality of slave databases configured to transmit current log location information of slave database log files to the agent layer.
11. A computing device, comprising:
a memory and a processor;
the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to:
receiving a message sent by a master database, wherein the message carries log site information corresponding to updating in a master database log file;
acquiring the log site information from the message;
responding to a reading request received in the session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result;
screening out the database which is updated according to the comparison result;
and routing the read request to the screened database.
12. A computer readable storage medium storing computer instructions which, when executed by a processor, carry out the steps of the method of routing read requests according to any one of claims 1 to 6.
13. A computing device, comprising:
a memory and a processor;
the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to:
responding to a message generated after the updating execution of the master database is finished, and writing log site information corresponding to the updating in a master database log file into the message;
returning a message carrying the log site information to an agent layer;
the message is used for enabling the agent layer to acquire the log site information from the message, responding to a reading request received in a session corresponding to the updating, and comparing the log site information with the current log site information of each slave database log file to obtain a comparison result; screening out the database which is updated according to the comparison result; and routing the read request to the screened database.
14. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the method of feeding back a message according to claim 8.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581310.2A CN113297229B (en) | 2020-06-23 | 2020-06-23 | Method for routing read request and feedback message, respective device and database |
| PCT/CN2021/101182 WO2021259188A1 (en) | 2020-06-23 | 2021-06-21 | Method for routing read request, method for feeding back message and respective apparatuses, and database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581310.2A CN113297229B (en) | 2020-06-23 | 2020-06-23 | Method for routing read request and feedback message, respective device and database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113297229A true CN113297229A (en) | 2021-08-24 |
| CN113297229B CN113297229B (en) | 2024-03-08 |
Family
ID=77318095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010581310.2A Active CN113297229B (en) | 2020-06-23 | 2020-06-23 | Method for routing read request and feedback message, respective device and database |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113297229B (en) |
| WO (1) | WO2021259188A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114048068A (en) * | 2021-10-28 | 2022-02-15 | 天翼数字生活科技有限公司 | MYSQL database operation and maintenance method, device, equipment and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149885B (en) * | 2023-10-31 | 2024-01-26 | 天津神舟通用数据技术有限公司 | Read-write separation implementation method based on database master node forwarding |
| CN117251500B (en) * | 2023-11-20 | 2024-02-09 | 深圳市雁联计算系统有限公司 | Dynamic read-write separation method for database |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
| CN106933843A (en) * | 2015-12-29 | 2017-07-07 | 阿里巴巴集团控股有限公司 | database heartbeat detecting method and device |
| CN110019502A (en) * | 2017-08-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Synchronous method, Database Systems and equipment between primary database and standby database |
| US20200050692A1 (en) * | 2018-08-10 | 2020-02-13 | Microsoft Technology Licensing, Llc | Consistent read queries from a secondary compute node |
-
2020
- 2020-06-23 CN CN202010581310.2A patent/CN113297229B/en active Active
-
2021
- 2021-06-21 WO PCT/CN2021/101182 patent/WO2021259188A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN106933843A (en) * | 2015-12-29 | 2017-07-07 | 阿里巴巴集团控股有限公司 | database heartbeat detecting method and device |
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
| CN110019502A (en) * | 2017-08-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Synchronous method, Database Systems and equipment between primary database and standby database |
| US20200050692A1 (en) * | 2018-08-10 | 2020-02-13 | Microsoft Technology Licensing, Llc | Consistent read queries from a secondary compute node |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114048068A (en) * | 2021-10-28 | 2022-02-15 | 天翼数字生活科技有限公司 | MYSQL database operation and maintenance method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021259188A1 (en) | 2021-12-30 |
| CN113297229B (en) | 2024-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110609872B (en) | Method and apparatus for synchronizing node data | |
| EP2834759B1 (en) | Telemetry system for a cloud synchronization system | |
| WO2021259188A1 (en) | Method for routing read request, method for feeding back message and respective apparatuses, and database | |
| CN108874946B (en) | A kind of ID management method and device | |
| CN105407117A (en) | Distributed data backup method, device and system | |
| CN112486074B (en) | Data processing system, method and device | |
| US10579644B2 (en) | Record property synchronization in a network computing system | |
| CN111949324A (en) | Distributed serial number generation method and device | |
| CN114610532A (en) | Database processing method and device | |
| CN113297007A (en) | Data processing method and device | |
| CN113886350A (en) | Data processing method and system | |
| CN111475492A (en) | Data processing method and device | |
| CN115913742A (en) | Unidirectional data transmission method and system and external network client | |
| CN114610533A (en) | Database processing method and device | |
| CN113204395A (en) | Service processing method and device | |
| CN113296887A (en) | Method and device for starting safety container | |
| CN111949418A (en) | Subscription message processing method and device | |
| CN112286947B (en) | Method and device for keeping data consistency of different storage systems | |
| CN112463206A (en) | Configuration information processing method and device in configuration center | |
| CN113468103B (en) | Method and device used in snapshot migration | |
| CN114168589A (en) | Index construction method and device | |
| CN113301084A (en) | Data processing method and device | |
| CN113094441A (en) | Data synchronization method and device | |
| CN114363640B (en) | Data storage method, device and system | |
| CN117972250B (en) | Data storage method, system and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40058614 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |