CN114594907A - Data read-write processing method, device, equipment and storage medium - Google Patents
Data read-write processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114594907A CN114594907A CN202210174003.1A CN202210174003A CN114594907A CN 114594907 A CN114594907 A CN 114594907A CN 202210174003 A CN202210174003 A CN 202210174003A CN 114594907 A CN114594907 A CN 114594907A
- Authority
- CN
- China
- Prior art keywords
- data
- read
- serial number
- database
- slave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0622—Securing storage systems in relation to access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The application provides a data read-write processing method, a data read-write processing device, data read-write processing equipment and a storage medium. The technical scheme provided by the application comprises the following steps: in the data writing phase, the master device allocates a corresponding serial number to the data when writing the data into the master database, and synchronizes the data and the corresponding serial number to the slave device. And the slave equipment writes the newly added data and the corresponding serial number into the slave database, and sets the serial number corresponding to the newly added data as the serial number information stored in the slave database. And in the data reading stage, whether the data is stored in the slave database is determined by the slave equipment according to the serial number of the data and the serial number information of the slave database. If the data is stored in the slave database, the slave device reads the data from the slave database; if the data is not stored in the slave database, the data is read from the master database. By the technical scheme, the problem of data reading abnormity caused by inconsistent data reading and writing in the related technology is solved, and the data reading efficiency is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data read/write processing method, apparatus, device, and storage medium.
Background
With the rapid increase of the data volume, the reading and writing pressure of the database is increased. In order to share the reading and writing pressure of the database, the data reading and writing separation is realized by using a master-slave data storage mode. The master-slave data storage mode is provided with a master database and a plurality of slave databases, data is preferentially written into the master database, and then the data stored in the master database is synchronized to the slave databases so as to read the data from the slave databases. Because the slave databases are provided in plurality, the slave databases can be simultaneously used, the data reading efficiency is improved, and the database load is reduced.
The existing slave database and master database may be arranged across the machine rooms, i.e. the master database and slave database are arranged under servers of different machine rooms. Under the cross-room scene, different rooms are provided with corresponding slave databases, and low delay of data reading of each room is ensured through the slave databases. However, based on factors such as network delay among different computer rooms, the newly written data in the master database is difficult to be synchronized to the slave database in real time, and the consistency of data reading and writing cannot be guaranteed, so that the slave database cannot read the data, data reading abnormity is caused, and the data reading efficiency is reduced.
Disclosure of Invention
The application provides a data read-write processing method, a data read-write processing device, data read-write processing equipment and a storage medium, which are used for solving the problem of data read abnormity caused by inconsistent data read-write in the related technology and improving the data read efficiency.
In a first aspect, the present application provides a data read-write processing method, including:
when the main database writes new data, distributing corresponding serial numbers to the new data;
writing the newly added data and the corresponding serial number into the main database;
and synchronizing the newly added data and the corresponding serial number to the slave equipment in a write diffusion mode so that the slave equipment writes the newly added data and the corresponding serial number into the slave database.
In a second aspect, the present application provides a data read-write processing method, including:
receiving newly added data and a corresponding serial number sent by a master device, wherein the serial number is obtained by distributing the newly added data when the master device writes the newly added data;
and writing the newly added data and the corresponding serial number into a slave database, and adjusting the stored serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data request terminal is stored in the slave database.
In a third aspect, the present application provides a data read-write processing method, including:
receiving a data reading request sent by a data request end, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when master equipment writes in new data corresponding to the data to be read;
comparing the serial number of the data to be read with serial number information recorded in a slave database, and determining whether the data to be read is stored in the slave database, wherein the serial number information is obtained based on information synchronization with the master device;
and responding to the judgment result that the data to be read is stored in the slave database, reading the data to be read from the slave database, and sending the data to be read to the data request end.
In a fourth aspect, the present application provides a data read-write processing method, including:
receiving a data reading request sent by slave equipment, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when the master equipment writes in newly added data corresponding to the data to be read;
acquiring corresponding data to be read from a master database according to the serial number of the data to be read;
and sending the data to be read to the slave equipment so that the slave equipment sends the data to be read to a data request end.
In a fifth aspect, the present application provides a data read/write processing apparatus, including:
the serial number generation module is configured to distribute a corresponding serial number to newly added data when the main database writes the newly added data;
a master database write module configured to write the newly added data and the corresponding serial number into the master database;
and the data synchronization module is configured to synchronize the newly added data and the corresponding serial numbers to the slave equipment in a write diffusion mode, so that the slave equipment writes the newly added data and the corresponding serial numbers into the slave database.
In a sixth aspect, the present application provides a data read/write processing apparatus, including:
the data receiving module is configured to receive newly added data and a corresponding serial number, wherein the newly added data and the corresponding serial number are sent by a master device, and the serial number is obtained by distributing the newly added data when the master device writes the newly added data;
and the slave database writing module is configured to write the newly added data and the corresponding serial number into the slave database, and adjust the saved serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data requesting terminal is stored in the slave database.
In a seventh aspect, the present application provides a data read/write processing apparatus, including:
the data reading device comprises a first request receiving module, a second request receiving module and a data reading module, wherein the first request receiving module is configured to receive a data reading request sent by a data request end, the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when master equipment writes new data corresponding to the data to be read;
a serial number comparison module configured to compare a serial number of the data to be read with serial number information recorded in a slave database, and determine whether the slave database stores the data to be read, where the serial number information is obtained based on information synchronization with the master device;
and the slave database reading module is configured to respond to the judgment result that the data to be read is stored in the slave database, read the data to be read from the slave database and send the data to be read to the data request terminal.
In an eighth aspect, the present application provides a data read/write processing apparatus, including:
the second request receiving module is configured to receive a data reading request sent by a slave device, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when the master device writes in new data corresponding to the data to be read;
the master database reading module is configured to obtain corresponding data to be read from a master database according to the serial number of the data to be read;
the data sending module is configured to send the data to be read to the slave device, so that the slave device sends the data to be read to a data request end.
In a ninth aspect, the present application provides a data read-write processing device, including: a memory and one or more processors;
the memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the data reading and writing processing method according to the first aspect, the second aspect, the third aspect, or the fourth aspect.
In a tenth aspect, the present application provides a storage medium containing computer-executable instructions for performing a data read-write processing method as described in the first, second, third or fourth aspect when executed by a computer processor.
In the data writing stage, corresponding serial numbers are distributed to newly added data when the newly added data are written into a master database through the master device, and the newly added data and the corresponding serial numbers are synchronized to the slave device. And the slave equipment writes the newly added data and the corresponding serial number into the slave database, and sets the serial number corresponding to the newly added data as the serial number information stored in the slave database. In the data reading stage, whether the data to be read is stored in the slave database or not is determined by the slave device according to the serial number of the data to be read and the serial number information of the slave database. If the data to be read is stored in the slave database, the slave device reads the data to be read from the slave database; and if the slave database does not store the data to be read, the slave device sends the serial number of the data to be read to the master device so that the master device reads the data to be read from the master database. Through the technical means, the sequence number of the data to be read can represent the sequence of writing the data into the master database, the sequence number information stored in the slave database can represent the data which is newly synchronized to the slave database in the master database, and the sequence number of the data to be read is compared with the sequence number information stored in the slave database, so that whether the data to be read in the master database is synchronized to the slave database can be determined. If the data to be read is synchronized to the slave database, the data to be read is read from the slave database, and if the data to be read is not synchronized to the slave database, the data to be read is read from the master database, so that the consistency of data reading and writing is ensured, abnormal data reading is avoided, and the data reading efficiency is improved.
Drawings
FIG. 1 is a block diagram of a data write phase according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a data writing processing method according to an embodiment of the present application;
FIG. 3 is a flow chart of another data writing method provided by an embodiment of the present application;
FIG. 4 is a block diagram of a data read phase provided by an embodiment of the present application;
FIG. 5 is a flowchart of a data reading method provided in an embodiment of the present application;
FIG. 6 is a flow chart of another data reading method provided by the embodiments of the present application;
fig. 7 is a schematic structural diagram of a data read/write processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of another data read/write processing apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of another data read/write processing apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of another data read/write processing apparatus according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of a data read-write processing device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application. It should be further noted that, for the convenience of description, only some but not all of the relevant portions of the present application are shown in the drawings. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
The data read-write processing method provided in the embodiment of the present application may be executed by a data read-write processing device, where the data read-write processing device may be implemented in a software and/or hardware manner, and the data read-write processing device may be formed by two or more physical entities or may be formed by one physical entity. In this embodiment, the data read-write processing device includes a master device and a slave device, where the master device is a proxy server configured in a master room, the slave device is a proxy server configured in a slave room, a database managed by the master room is a master database, and a database managed by the slave room is a slave database.
In one embodiment, a data read/write system in a master-slave mode is generally configured with a master database and a plurality of slave databases. When the data read-write system receives the new data, the new data is preferentially written into the master database, the data in the master database is synchronized to the slave database, and when the data read-write system receives a data read request, the data is read from the slave database. Because a plurality of slave databases are configured in the data reading and writing system, the data reading pressure of the slave databases is low, and the master database is only responsible for data synchronization. Therefore, the master-slave mode can effectively relieve the data reading pressure of the database. At present, as the service scope increases, the master database and the slave database are arranged in machine rooms in different areas to quickly respond to data reading requests in different areas. However, in the data reading and writing system arranged across the machine room, the network delay between the master database and the slave database is high, and it cannot be guaranteed that the data newly written in the master database can be synchronized to the slave database in real time, so that the data cannot be read from the slave database, and the data reading and writing are inconsistent, which causes data reading abnormality. And if the data is read from the slave database after the data is synchronized to all the slave databases, high data reading delay is caused, and the real-time performance of the data is influenced.
In order to solve the above problem, an embodiment of the present application provides a data read-write processing method, so as to solve the problem of data read abnormality caused by inconsistent data read-write of a master-slave database across a machine room, and improve data read efficiency. The data read-write processing method provided by the embodiment of the application is divided into two stages, namely a data write-in stage and a data read-out stage.
In an embodiment, fig. 1 is a schematic diagram of a framework of a data writing phase provided in an embodiment of the present application. As shown in fig. 1, after receiving the new data, the master device assigns a corresponding serial number to the new data, and writes the new data and the corresponding serial number into the master database. After the newly added data are written into the master database, the master device sends the newly added data and the corresponding serial numbers to the synchronization server, and the synchronization server sends the newly added data and the corresponding serial numbers to each slave device. And the slave equipment writes the newly added data and the corresponding serial number into the slave database, and saves the serial number of the newly added data as serial number information. In this embodiment, in the data writing phase, the master device and the slave device respectively execute corresponding data writing methods, and cooperatively complete the data writing operation.
In this embodiment, fig. 2 is a flowchart of a data writing processing method provided in an embodiment of the present application. The main execution body of the method is a master device, and referring to fig. 2, the data writing processing method includes:
and S110, when the main database writes the newly added data, distributing a corresponding serial number to the newly added data.
Illustratively, after receiving the new data, the master device prepares to write the new data into the master database. When the new data is written into the main database, the serial number of the new data is generated according to the serial number of the data written at the moment in the main database, so that the serial number of the new data and the serial number of the data in the main database can be sorted according to the writing sequence.
In one embodiment, according to the sequence of the new data, a monotonically increasing serial number is sequentially assigned to each new data. For example, assuming that the first newly added data and the second newly added data are the nth data and the (n + 1) th data written in the master database, respectively, the serial number n is assigned to the first newly added data, and the serial number n +1 is assigned to the second newly added data. In another embodiment, the serial number of the newly added data may also be monotonically decreased, for example, if the serial number of the first newly added data written into the primary database is M, the serial number of the nth newly added data written into the primary database is M-n +1, and the serial number of the nth +1 second newly added data written into the primary database is M-n. However, if the amount of data stored in the master database is too large, the monotonically decreasing serial number setting method needs to set a larger initial serial number, and there may be a case where the amount of data in the master database is larger than the initial serial number, so that the monotonically increasing method is more suitable than the monotonically decreasing serial number setting method. It should be noted that the increasing value or decreasing value between data is not necessarily 1, and may be other positive integer values as long as the sequence number is ensured to be monotonically increasing or monotonically decreasing according to the writing order.
In this embodiment, description is made in such a manner that the master database sets the serial number in monotonically increasing increments. And setting the serial number of the newly added data as the serial number information of the main database every time the main database writes one newly added data. The serial number information may be regarded as the maximum serial number of data stored in the master database, i.e., the serial number of data newly written in the master database. When the master device writes the new data into the master database, the serial number larger than the serial number information is distributed to the new data according to the serial number information of the master database, and the monotone increasing of the serial number can be ensured. For example, if the serial number information of the master device is m, a serial number m +1 is assigned to the new addition data.
And S120, writing the newly added data and the corresponding serial number into a main database.
In this embodiment, the serial number corresponding to the newly added data is stored in the data identifier of the newly added data, and the newly added data and the corresponding adjusted data identifier are written into the master database. Illustratively, each data item is associated with a data identifier, and the data identifier characterizes the identity of the data item. And storing the serial number into the data identifier of the data so as to determine the serial number of the data according to the data identifier.
In an embodiment, after the master device writes the newly added data and the data identifier with the serial number into the master database, the data identifier with the serial number is sent to the data requesting end, so that the data requesting end determines that the writing of the newly added data is completed, and sends a data reading request to the slave device based on the adjusted data identifier. For example, the data identifier with the stored serial number may uniquely characterize the corresponding data, and when the data requesting end receives the data identifier with the stored serial number, it may be determined that the data corresponding to the data identifier has been written into the master database. After determining that the data is written into the master database, the data request terminal can generate a corresponding data reading request according to the data identifier, and send the data reading request to the slave device to request to read the data corresponding to the data identifier. The data request end does not need to wait for data to be written into each slave device and then request to read the data, and the data request end can request to read the data after the data is written into the master database, so that the data writing performance is greatly improved, and the low delay of data writing is ensured.
S130, synchronizing the newly added data and the corresponding serial number to the slave device in a write diffusion mode, so that the slave device writes the newly added data and the corresponding serial number into the slave database.
The write diffusion mode is to transmit data to the synchronization server, and the synchronization server transmits the data to each slave device. The write diffusion mode can ensure that data can be synchronized to the slave device more quickly, the data writing efficiency of the slave database is improved, and the low delay of data writing is ensured. For example, referring to fig. 1, the master device sends the newly added data and the data identifier with the stored serial number to the synchronization server, and the newly added data and the data identifier are sent to each slave device through the synchronization server. And after receiving the newly added data and the data identification sent by the synchronous server, the slave device writes the newly added data and the data identification into the slave database and updates the serial number information of the slave database.
Further, fig. 3 is a flowchart of another data writing method provided in an embodiment of the present application. The execution subject of the method is a slave device, and referring to fig. 3, the data write processing method includes:
s210, receiving the newly added data and the corresponding serial number sent by the main device.
For example, referring to fig. 1, after the master device sends the newly added data and the data identifier with the stored serial number to the synchronization server, the slave device receives the newly added data and the data identifier with the stored serial number sent by the synchronization server.
S220, writing the newly added data and the corresponding serial number into the slave database, and adjusting the stored serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data request end is stored in the slave database.
Illustratively, the slave device writes the new data and the data identifier with the stored serial number into the slave database, and updates the serial number information of the stored slave database to the serial number corresponding to the new data. The serial number information of the slave database is the serial number of the data which is newly written from the database, and when the slave device writes the newly added data into the slave database, the slave data are sequentially written according to the serial numbers, so that the serial numbers of the data in the slave database are monotonically increased according to the writing sequence. The serial number information of the slave device can also be seen as the maximum serial number of the data stored in the slave database.
It should be noted that, since the master database and the slave database maintain data synchronization, the sequence of writing the newly added data into the slave database is the same as the sequence of writing the newly added data into the master database. Therefore, the data corresponding to the maximum serial number of the slave database and the data of the downward serial number in the master database can be found in the slave database, and the data corresponding to the upward maximum serial number of the slave database in the master database cannot be found in the slave database. For example, if the maximum serial number of the slave database is k, the data of serial number 1 to serial number k in the master database are stored in the slave database. While data in the master database greater than the serial number k is not stored in the slave database. Correspondingly, when the slave device receives a data reading request issued by the data request terminal, the serial number in the data reading request can be compared with the serial number information of the slave database, and whether the data requested by the data request terminal is written into the slave database is determined.
On the basis of the above embodiments, fig. 4 is a schematic diagram of a framework of a data reading phase provided in an embodiment of the present application. As shown in fig. 4, after receiving a data reading request sent by a data request end, a slave device parses a serial number of data to be read from the data reading request. And the slave equipment compares the serial number information of the slave database with the serial number of the data to be read and determines whether the data to be read in the master database is synchronized to the slave database. If the data to be read is synchronized to the slave database, the slave device reads the data to be read from the slave database; and if the data to be read is not synchronized to the slave database, the slave equipment forwards the data reading request to the master equipment, and the master equipment reads the data to be read from the master database. In the data reading stage, the master device and the slave device respectively execute corresponding data reading methods to cooperatively complete data reading operation.
In this embodiment, fig. 5 is a flowchart of a data reading method provided in an embodiment of the present application. The execution subject of the method is a slave device, and referring to fig. 5, the data reading method includes:
s310, receiving a data reading request sent by a data request end, wherein the data reading request comprises a serial number of data to be read.
For example, after receiving a data identifier which is sent by a master device and is stored with a serial number, a data request end generates a corresponding data reading request according to the data identifier, and sends the data reading request to a slave device, so that the slave device reads data to be read corresponding to the data identifier, and sends the data to the data request end. Further, the slave device obtains the data identifier of the data to be read from the data reading request sent by the data request terminal, and analyzes the serial number of the data to be read from the data identifier.
S320, comparing the serial number of the data to be read with the serial number information recorded in the slave database, and determining whether the data to be read is stored in the slave database.
In one embodiment, if the serial number of the data to be read is greater than the serial number recorded in the serial number information in the slave database, it is determined that the data to be read is not stored in the slave database; and if the serial number of the data to be read is less than or equal to the serial number recorded in the serial number information in the database, determining that the data to be read is stored in the database. For example, assume that the master database sets the sequence number in a monotonically increasing manner. As can be seen from the above, if the serial number of the data to be read is greater than the serial number information of the slave database, it may be determined that the data to be read in the master database is not synchronized to the slave database, that is, the slave database does not store the data to be read. If the serial number of the data to be read is less than or equal to the serial number information of the slave database, it can be determined that the data to be read in the master database is synchronized to the slave database, i.e., the slave database stores the data to be read.
In contrast, if the master database sets the serial number in a monotonically decreasing manner, the serial number information of the slave database is the minimum serial number of data in the slave database. If the serial number of the data to be read is smaller than the serial number information of the slave database, determining that the data to be read is not stored in the slave database; if the serial number of the data to be read is greater than or equal to the serial number information of the slave database, it may be determined that the data to be read is stored from the database.
S330, responding to the judgment result that the data to be read is stored in the database, reading the data to be read from the database, and sending the data to be read to the data request end.
Referring to fig. 1, when the slave device determines that the serial number of the data to be read is less than or equal to the serial number information of the slave database, the slave device reads the read data from the slave database and sends the read data to the data request side.
And S340, responding to the judgment result that the data to be read is not stored in the slave database, and sending the data reading request to the master device so that the master device reads the data to be read from the master database and then sends the data to be read.
Referring to fig. 1, when the slave device determines that the serial number of the data to be read is greater than the serial number of the slave database, the slave device forwards the data read request to the master device to notify the master device that the data to be read is not read by itself. And after receiving the data request, the master device determines that the slave device does not read the data to be read, reads the data to be read from the master database and sends the data to be read to the data request terminal.
Further, fig. 6 is a flowchart of another data reading method provided in the embodiment of the present application. The main execution body of the method is a master device, and referring to fig. 6, the data reading method includes:
s410, receiving a data reading request sent by the slave device, and acquiring corresponding data to be read from the master database according to a serial number in the data reading request.
For example, after receiving a data reading request sent by a slave device, a master device obtains a data identifier storing a serial number of data to be read from the data reading request. And reading the data to be read from the main database according to the data identifier.
And S420, sending the data to be read to the slave device so that the slave device sends the data to be read to the data request end.
For example, after the master device reads the data to be read, the data to be read is sent to the slave device, so that the slave device is notified to send the data to be read to the corresponding data request terminal. And after receiving the data to be read, the slave device sends the data to be read to a data request end to ensure the closing of a request link.
It should be noted that, in this embodiment, data synchronization between the master database and the slave database is performed in a write-diffusion manner, the synchronization speed of the synchronization manner is fast, the delay is low, and a certain time is also required for the master database to upload the data identifier stored with the serial number to the data request end. Therefore, the situation that the data to be read in the master database is not synchronized to the slave database is rare, that is, most data reading operations do not need to be executed across a machine room, and low delay of data reading is guaranteed.
In summary, the data read-write processing method provided in this embodiment represents the sequence in which the data is written into the master database according to the serial number of the data to be read, and the serial number information stored in the slave database represents the data that is newly synchronized to the slave database in the master database. And determining whether the data to be read in the master database is synchronized to the slave database or not by comparing the serial number of the data to be read with the serial number information stored in the slave database. If the data to be read is synchronized to the slave database, the data to be read is read from the slave database, and if the data to be read is not synchronized to the slave database, the data to be read is read from the master database, so that the consistency of data reading and writing is ensured, abnormal data reading is avoided, and the data reading efficiency is improved.
On the basis of the foregoing embodiments, fig. 7 is a schematic structural diagram of a data read/write processing apparatus according to an embodiment of the present application. As shown in fig. 7, the data read/write processing apparatus includes a serial number generation module 51, a master database writing module 52, and a data synchronization module 53.
The serial number generation module is configured to distribute a corresponding serial number to newly added data when the main database writes the newly added data;
a master database writing module configured to write the newly added data and the corresponding serial number into a master database;
and the data synchronization module is configured to synchronize the newly added data and the corresponding serial numbers to the slave device in a write diffusion mode, so that the slave device writes the newly added data and the corresponding serial numbers into the slave database.
On the basis of the foregoing embodiments, fig. 8 is a schematic structural diagram of another data read/write processing apparatus according to an embodiment of the present application. As shown in fig. 8, the data read/write processing apparatus includes a data receiving module 61 and a slave database writing module 62.
The data receiving module is configured to receive newly added data and a corresponding serial number sent by the main device, wherein the serial number is obtained by distributing the newly added data when the main device writes the newly added data;
and the slave database writing module is configured to write the newly added data and the corresponding serial number into the slave database, and adjust the saved serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data request terminal is stored in the slave database.
On the basis of the foregoing embodiments, fig. 9 is a schematic structural diagram of another data read/write processing apparatus provided in the embodiments of the present application. As shown in fig. 9, the data read/write processing apparatus includes a first request receiving module 71, a serial number comparing module 72, and a database reading module 73.
The first request receiving module is configured to receive a data reading request sent by a data request end, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when the master device writes in new data corresponding to the data to be read;
the serial number comparison module is configured to compare the serial number of the data to be read with serial number information recorded in the slave database, and determine whether the data to be read is stored in the slave database, wherein the serial number information is obtained based on information synchronization with the master device;
and the database reading module is configured to respond to the judgment result that the data to be read is stored in the database, read the data to be read from the database and send the data to be read to the data request terminal.
On the basis of the foregoing embodiments, fig. 10 is a schematic structural diagram of another data read/write processing apparatus according to an embodiment of the present application. As shown in fig. 10, the data read/write processing apparatus includes a second request receiving module 81, a master database reading module 82, and a data transmitting module 83.
The second request receiving module is configured to receive a data reading request sent by the slave device, the data reading request includes a serial number of data to be read, and the serial number of the data to be read is allocated by the master device when writing of new data corresponding to the data to be read is performed;
the main database reading module is configured to obtain corresponding data to be read from the main database according to the serial number of the data to be read;
the data sending module is configured to send the data to be read to the slave device, so that the slave device sends the data to be read to the data request end.
In summary, the data reading and writing device provided in the embodiment of the present application represents the sequence in which data is written into the master database according to the serial number of the data to be read, and the serial number information stored in the slave database represents the latest synchronization of the master database to the data in the slave database. And determining whether the data to be read in the master database is synchronized to the slave database or not by comparing the serial number of the data to be read with the serial number information stored in the slave database. If the data to be read is synchronized to the slave database, the data to be read is read from the slave database, and if the data to be read is not synchronized to the slave database, the data to be read is read from the master database, so that the consistency of data reading and writing is ensured, abnormal data reading is avoided, and the data reading efficiency is improved.
The data read-write processing device provided by the embodiment of the application can be used for executing the data read-write processing method provided by the embodiment, and has corresponding functions and beneficial effects.
The embodiment of the application also provides data read-write processing equipment which can integrate the data read-write processing device provided by the embodiment of the application. Fig. 11 is a schematic structural diagram of a data read-write processing device according to an embodiment of the present application. Referring to fig. 11, the data read-write processing apparatus includes: an input device 93, an output device 94, a memory 92, and one or more processors 91; a memory 92 for storing one or more programs; when the one or more programs are executed by the one or more processors 91, the one or more processors 91 implement the data reading and writing processing method provided in the above embodiment. The data read-write processing device, the data read-write processing equipment and the computer can be used for executing the data read-write processing method provided by any embodiment, and have corresponding functions and beneficial effects.
Embodiments of the present application further provide a storage medium containing computer-executable instructions, which are used to execute the data reading and writing processing method provided by the above embodiments when executed by a computer processor. Of course, the storage medium provided in the embodiments of the present application and containing computer-executable instructions is not limited to the data reading and writing processing method described above, and may also perform related operations in the data reading and writing processing method provided in any embodiment of the present application. The data read-write processing apparatus, the device, and the storage medium provided in the foregoing embodiments may execute the data read-write processing method provided in any embodiment of the present application, and refer to the data read-write processing method provided in any embodiment of the present application without detailed technical details described in the foregoing embodiments.
The foregoing is considered as illustrative of the preferred embodiments of the invention and the technical principles employed. The present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the claims.
Claims (15)
1. A data read-write processing method is characterized by comprising the following steps:
when the main database writes new data, distributing corresponding serial numbers to the new data;
writing the newly added data and the corresponding serial number into the main database;
and synchronizing the newly added data and the corresponding serial number to the slave equipment in a write diffusion mode so that the slave equipment writes the newly added data and the corresponding serial number into the slave database.
2. The data read-write processing method according to claim 1, wherein the assigning a corresponding sequence number to the newly added data includes:
and according to the sequence of the newly added data, sequentially allocating a monotonically increasing serial number to each newly added data.
3. The data read-write processing method according to claim 1 or 2, wherein the writing the newly added data and the corresponding serial number into the master database includes:
and storing the serial number corresponding to the newly added data into the data identifier of the newly added data, and writing the newly added data and the data identifier with the stored serial number into the main database.
4. The data read-write processing method according to claim 3, after writing the newly added data and the data identifier with the stored serial number into the master database, further comprising:
and sending the data identifier stored with the serial number to a data request end so that the data request end determines that the newly added data is completely written, and sending a data reading request to the slave device based on the adjusted data identifier.
5. A data read-write processing method is characterized by comprising the following steps:
receiving newly added data and a corresponding serial number sent by a master device, wherein the serial number is obtained by distributing the newly added data when the master device writes the newly added data;
and writing the newly added data and the corresponding serial number into a slave database, and adjusting the stored serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data request terminal is stored in the slave database.
6. A data read-write processing method is characterized by comprising the following steps:
receiving a data reading request sent by a data request end, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when master equipment writes in new data corresponding to the data to be read;
comparing the serial number of the data to be read with serial number information recorded in a slave database, and determining whether the data to be read is stored in the slave database, wherein the serial number information is obtained based on information synchronization with the master device;
and responding to the judgment result that the data to be read is stored in the slave database, reading the data to be read from the slave database, and sending the data to be read to the data request end.
7. The data read-write processing method according to claim 6, wherein in response to a determination that the data to be read is not stored in the slave database, the data read request is sent to the master device, so that the master device reads the data to be read from the master database and sends the data to be read.
8. The data read-write processing method according to claim 6 or 7, characterized in that the serial number is sequentially assigned to each newly added data in a monotonically increasing manner by the master device according to the sequence of the newly added data;
correspondingly, the comparing the serial number of the data to be read with the serial number information recorded in the slave database to determine whether the slave database stores the data to be read includes:
if the serial number of the data to be read is larger than the serial number recorded by the serial number information in the slave database, determining that the slave database does not store the data to be read;
and if the serial number of the data to be read is less than or equal to the serial number recorded by the serial number information in the slave database, determining that the data to be read is stored in the slave database.
9. A data read-write processing method is characterized by comprising the following steps:
receiving a data reading request sent by slave equipment, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when the master equipment writes in newly added data corresponding to the data to be read;
acquiring corresponding data to be read from a master database according to the serial number of the data to be read;
and sending the data to be read to the slave equipment so that the slave equipment sends the data to be read to a data request end.
10. A data read/write processing apparatus, comprising:
the serial number generation module is configured to distribute a corresponding serial number to newly added data when the main database writes the newly added data;
a master database write module configured to write the newly added data and the corresponding serial number into the master database;
and the data synchronization module is configured to synchronize the newly added data and the corresponding serial number to the slave device in a write diffusion mode, so that the slave device writes the newly added data and the corresponding serial number into the slave database.
11. A data read/write processing apparatus, comprising:
the data receiving module is configured to receive newly added data and a corresponding serial number, wherein the newly added data and the corresponding serial number are sent by a master device, and the serial number is obtained by distributing the newly added data when the master device writes the newly added data;
and the slave database writing module is configured to write the newly added data and the corresponding serial number into the slave database, and adjust the saved serial number information based on the serial number, wherein the serial number information is used for determining whether the data requested to be read by the data requesting terminal is stored in the slave database.
12. A data read/write processing apparatus, comprising:
the data reading device comprises a first request receiving module, a second request receiving module and a data reading module, wherein the first request receiving module is configured to receive a data reading request sent by a data request end, the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when master equipment writes new data corresponding to the data to be read;
a serial number comparison module configured to compare a serial number of the data to be read with serial number information recorded in a slave database, and determine whether the slave database stores the data to be read, where the serial number information is obtained based on information synchronization with the master device;
and the slave database reading module is configured to respond to the judgment result that the data to be read is stored in the slave database, read the data to be read from the slave database and send the data to be read to the data request terminal.
13. A data read/write processing apparatus, comprising:
the second request receiving module is configured to receive a data reading request sent by a slave device, wherein the data reading request comprises a serial number of data to be read, and the serial number of the data to be read is obtained by distribution when the master device writes in new data corresponding to the data to be read;
a master database reading module configured to obtain corresponding data to be read from a master database according to the serial number of the data to be read
The data sending module is configured to send the data to be read to the slave device, so that the slave device sends the data to be read to a data request end.
14. A data read-write processing apparatus, characterized by comprising: a memory and one or more processors;
the memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the data read-write processing method of any one of claims 1-9.
15. A storage medium containing computer-executable instructions for performing the data read-write processing method of any one of claims 1-9 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210174003.1A CN114594907A (en) | 2022-02-24 | 2022-02-24 | Data read-write processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210174003.1A CN114594907A (en) | 2022-02-24 | 2022-02-24 | Data read-write processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114594907A true CN114594907A (en) | 2022-06-07 |
Family
ID=81806456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210174003.1A Pending CN114594907A (en) | 2022-02-24 | 2022-02-24 | Data read-write processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114594907A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714916A (en) * | 2009-11-26 | 2010-05-26 | 成都市华为赛门铁克科技有限公司 | Method, equipment and system for backing up |
| CN102760126A (en) * | 2011-04-26 | 2012-10-31 | 阿里巴巴集团控股有限公司 | Data synchronization method in distributive system, and distributive system |
| CN103577277A (en) * | 2012-07-27 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Method and device for processing data |
| CN103618640A (en) * | 2013-11-19 | 2014-03-05 | 武汉烽火网络有限责任公司 | Ethernet performance testing method and device based on point-to-point testing |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN105468718A (en) * | 2015-11-18 | 2016-04-06 | 腾讯科技（深圳）有限公司 | Data consistency processing method, device and system |
-
2022
- 2022-02-24 CN CN202210174003.1A patent/CN114594907A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714916A (en) * | 2009-11-26 | 2010-05-26 | 成都市华为赛门铁克科技有限公司 | Method, equipment and system for backing up |
| CN102760126A (en) * | 2011-04-26 | 2012-10-31 | 阿里巴巴集团控股有限公司 | Data synchronization method in distributive system, and distributive system |
| CN103577277A (en) * | 2012-07-27 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Method and device for processing data |
| CN103618640A (en) * | 2013-11-19 | 2014-03-05 | 武汉烽火网络有限责任公司 | Ethernet performance testing method and device based on point-to-point testing |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN105468718A (en) * | 2015-11-18 | 2016-04-06 | 腾讯科技（深圳）有限公司 | Data consistency processing method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11899689B2 (en) | Blockchain-based data synchronization method, apparatus, and computer-readable storage medium | |
| CN111399764B (en) | Data storage method, data reading device, data storage equipment and data storage medium | |
| CN109561151B (en) | Data storage method, device, server and storage medium | |
| CN113094430B (en) | Data processing method, device, equipment and storage medium | |
| CN110837505A (en) | State data storage method, state data synchronization device and storage medium | |
| CN113407637A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| WO2022134797A1 (en) | Data fragmentation storage method and apparatus, a computer device, and a storage medium | |
| CN111291062B (en) | Data synchronous writing method and device, computer equipment and storage medium | |
| CN112351068A (en) | Information synchronization method, system, device, computer equipment and storage medium | |
| CN111338834B (en) | Data storage method and device | |
| CN113051102A (en) | File backup method, device, system, storage medium and computer equipment | |
| CN113377863B (en) | Data synchronization method and device, electronic equipment and computer readable storage medium | |
| CN113190620A (en) | Method, device, equipment and storage medium for synchronizing data between Redis clusters | |
| CN110113217B (en) | Micro-service management method, device, management platform and storage medium | |
| CN112231324B (en) | System and method for realizing incremental data comparison | |
| CN113810216A (en) | Cluster fault switching method and device and electronic equipment | |
| CN114594907A (en) | Data read-write processing method, device, equipment and storage medium | |
| CN111966650B (en) | Operation and maintenance big data sharing data table processing method and device and storage medium | |
| CN114706923A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN113094431A (en) | Read-write separation method and device and server | |
| CN114969206A (en) | Data processing method, device, equipment and storage medium | |
| CN111221857B (en) | Method and apparatus for reading data records from a distributed system | |
| CN113253924A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN107332679B (en) | Centerless information synchronization method and device | |
| CN111666338B (en) | Data replication method, control node and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |