CN113609199A - Database system, server, and storage medium - Google Patents
Database system, server, and storage medium Download PDFInfo
- Publication number
- CN113609199A CN113609199A CN202110853140.3A CN202110853140A CN113609199A CN 113609199 A CN113609199 A CN 113609199A CN 202110853140 A CN202110853140 A CN 202110853140A CN 113609199 A CN113609199 A CN 113609199A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- server
- database server
- load balancer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a database system, a server and a storage medium, and belongs to the field of databases. The database system includes: at least two database servers and a load balancer; the load balancer is used for sending a data writing request to the database server; the database server is used for storing target data according to the received data writing request; the database server is also used for sending the target data to the associated groups in the message queue; and the database server is also used for extracting the queue data in the message queue target group and storing the queue data. In the embodiment of the application, each database server has data writing capacity, and data synchronization is performed through the message queue while target data is stored, so that the data writing efficiency and the safety and high availability of a database system are improved; the load balancer can realize data synchronization by only sending a data writing request to one database server at random, and the load of the load balancer is reduced.
Description
Technical Field
The embodiment of the application relates to the technical field of databases, in particular to a database system, a server and a storage medium.
Background
A database is a warehouse that organizes, stores, and manages data according to a data structure, and is a collection of large amounts of data that is stored in a computer for a long time, organized, sharable, and uniformly managed. The database in the internet of things generally needs to have a condition that data is complete and can be queried in real time, and in order to prevent data loss caused by database abnormality and incapability of querying the data, developers generally deploy the database in a high-availability database system.
In the related technology, the working mode of the high-availability database system includes a master-slave mode, that is, when the host works normally, data is backed up to the standby machine through the shared storage system, when the host is abnormal, the standby machine takes over the work of the host, and when the host recovers to be normal, the standby machine recovers to be in a standby monitoring state; the multi-active mode is that the controller distributes data processing tasks to one host, and when the host is abnormal, the controller distributes the tasks to other hosts.
However, if the database system in the related art only includes one host, the data reading and writing efficiency of the database is low, and after the host recovers abnormally, the time required for completing the data is long, which also reduces the data processing efficiency; if the system comprises a plurality of hosts, a controller is required to be arranged to judge the working state of each host, the cost is high, and the efficiency of reporting the abnormal state is limited.
Disclosure of Invention
The embodiment of the application provides a database system, a server and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides a database system, where the database system includes: the system comprises at least two database servers and a load balancer, wherein the at least two database servers are associated with each other and store the same data;
the load balancer is used for sending a data writing request to the database server, wherein the data writing request comprises target data;
the database server is used for storing the target data according to the received data writing request;
the database server is further configured to send the target data to an associated packet in a message queue, where the associated packet is a packet corresponding to the associated database server;
the database server is further configured to extract queue data in the message queue target packet and store the queue data, where the queue data is sent to the target packet by the associated database server, and the target packet is a packet corresponding to the database server.
In another aspect, an embodiment of the present application provides a server, which includes a processor and a memory; the memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded and executed by the processor to implement the functions of a database server or load balancer in a database system as described in the above aspects.
In another aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the functions of a database server or a load balancer in a database system as described in the above aspect.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the server reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the server implements the functions of the database server or the load balancer in the database system provided in the various alternative implementations of the above aspects.
The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:
in the embodiment of the application, each database server of the database system has data writing capability, and the database servers automatically synchronize data through the message queue while storing the target data, so that the data stored by each database server in the database system are the same, and the data writing efficiency, the safety and the high availability of the database system are improved; the load balancer can realize data synchronization by only sending a data writing request to one database server at random, the running state of each database server does not need to be monitored before each data request is sent, the load of the load balancer is reduced, an additional controller is not needed, and the requirement of various databases on high availability can be met.
Drawings
FIG. 1 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application;
FIG. 2 is a flow chart of data writing provided by an exemplary embodiment of the present application;
FIG. 3 is a flow chart of data writing provided by another exemplary embodiment of the present application;
FIG. 4 is a flow chart of database modification provided by an exemplary embodiment of the present application;
FIG. 5 is a flow chart of a data read provided by an exemplary embodiment of the present application;
FIG. 6 is a flow chart of data writing provided by another exemplary embodiment of the present application;
FIG. 7 is a flow chart of a synchronizer for data writing provided by an exemplary embodiment of the present application;
FIG. 8 is a flow chart of data writing provided by another exemplary embodiment of the present application;
FIG. 9 is a flow chart of a data read provided by another exemplary embodiment of the present application;
fig. 10 is a block diagram of a server according to another exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
In the related technology, the working mode of the high-availability database system includes a master-slave mode, that is, when the host works normally, data is backed up to the standby machine through the shared storage system, when the host is abnormal, the standby machine takes over the work of the host, and when the host recovers to be normal, the standby machine recovers to be in a standby monitoring state; the multi-active mode is that the controller distributes data processing tasks to one host, and when the host is abnormal, the controller distributes the tasks to other hosts; in the shared storage mode, for a relational database management system (MySQL), a binary log (Binlog) file of a MySQL database is stored in a shared disk, and when a main server is down, a standby machine acquires data through the Binlog file and is used as the main server to read and write the data.
However, if the database system in the related art only includes one host, the data reading and writing efficiency of the database is low, and after the host recovers abnormally, the time required for completing the data is long, which also reduces the data processing efficiency; if the system comprises a plurality of hosts, a controller is required to be arranged to judge the working state of each host, the cost is high, and the abnormal reporting efficiency is limited; or, the main server and the standby server need to share data through special files, and the method cannot be applied to other types of database systems.
In order to solve the above problem, an embodiment of the present application provides a database system, and the structure of the database system is shown in fig. 1.
The load balancer 110 is configured to receive a data read-write request, a database modification instruction, and the like, and randomly send the received request and instruction to any one of the database servers in the database system. For a data reading request, if the load balancer 110 determines that the database server receiving the data reading and writing request is abnormal and cannot read data, the load balancer 110 sends the data reading and writing request to other database servers in the database system.
The database server 120, the database server 130, and the database server 140 are associated with each other, and are used for reading and writing data, modifying the database, synchronizing the data among the databases, and the like. In a possible implementation manner, when one of the database servers receives a data writing request, after the data writing is completed, the target data is sent to the message queue, and each database server accesses the message queue, extracts the data stored in the other database servers, and writes the data into the database. For the database modification instruction, after one database server receives the database modification instruction, the instruction is executed and forwarded to other database servers, so that each data block server in the database system can carry out the same modification, and the consistency of the database is ensured.
Compared with the database systems provided in the related art, the database servers in the embodiment of the application automatically synchronize data through the message queue while storing the target data, so that the data stored by each database server in the database system are the same, and the data writing efficiency, the safety and the high availability of the database system are improved; the load balancer can realize data synchronization by only sending a data writing request to one database server at random, the running state of each database server does not need to be monitored before each data request is sent, the load of the load balancer is reduced, an additional controller is not needed, and the requirement of various databases on high availability can be met.
Referring to fig. 2, it shows a flowchart of interaction when a database system writes data according to an embodiment of the present application, where the flowchart includes:
step 201, the load balancer sends a data write request to the database server, where the data write request includes target data.
The load balancer is a server for receiving data write requests and other requests, instructions (e.g., data read requests, database modification requests), and the like, and feeding back request responses. The load balancer is connected with the Internet of things equipment, computer equipment of Internet of things users or computer equipment responsible for data acquisition and the database server.
In order to improve the high availability of a database system and avoid the situations that when one or more database servers are abnormal, operation and maintenance personnel cannot reach the location of the abnormal server in time for maintenance to cause data loss and cannot read data, in a possible implementation mode, the database system comprises at least two database servers, the database servers in the database system are mutually associated, the data stored in the database in each database server is the same, and therefore the situation that when one database server is down, the data loss is caused is avoided.
For a data write request, the load balancer only needs to send the data write request to one database server or a part of database servers, and the data write request does not need to be forwarded to all database servers in the database system.
Optionally, the load balancer sequentially sends the received data write-in requests to the database servers according to a certain cyclic sequence, for example, the database system includes three database servers, namely a database server a, a database server B and a database server C, if the load balancer sends the currently received data write-in request to the database server a, the currently received data write-in request is sent to the database server B when the data write-in request is next received, and the data write-in request and other requests or instructions are sent cyclically according to the sequence of the database server a, the database server B, the database server C and the database server a; or the load balancer randomly sends the received data write-in request to one database server, and developers set request distribution parameters in the program code of the load balancer, so that the probability that the load balancer sends the data write-in request to each database server is the same, the workload of the database servers is evenly distributed, and the data storage efficiency is improved. The embodiments of the present application do not limit this.
Step 202, the database server stores the target data according to the received data write request.
The database servers respectively perform data storage according to the received data writing requests, for example, when the database server a receives a data writing request containing the target data a, the target data a is stored in the database, and when the database server B receives a data writing request containing the target data B, the target data B is stored in the database.
Step 203, the database server stores the target data into an associated packet in a Message Queue (MQ), and the associated packet is a packet corresponding to the associated database server.
The database system provided by the embodiment of the application is a multi-activity high-availability system, and in order to improve the high availability of the database system, the data stored by each database server needs to be ensured to be consistent, so that when any one database server in the database system is down, other database servers still store complete data and can continue to read and write the data.
In one possible embodiment, the database servers use message queues to synchronize data. The message queue is a container for storing messages in the message transmission process, and the database server puts data (messages) to be transmitted in the queue, and the message transmission is realized by using a queue mechanism. In order to facilitate data synchronization, the message queue in the embodiment of the application is divided into n groups, wherein n is the number of database servers in the database system, and each database server corresponds to a group of the message queue. And after the database server stores the target data, storing the target data into a message queue group corresponding to the associated database server, namely an associated group.
Illustratively, the message queue is stored separately from the load balancer and in a server other than the database server.
Illustratively, the database system comprises a database server a, a database server B and a database server C, the database server a receives the data writing request, stores the target data in the database, and then stores the target data in the packets corresponding to the database server B and the database server C in the message queue, so that the database server B and the database server C subsequently extract the target data for storage, thereby realizing that all the database servers in the database system store the target data.
And step 204, the database server extracts the queue data in the message queue target group and stores the queue data, the queue data is stored to the target group by the associated database server, and the target group is a group corresponding to the database server.
In one possible implementation, the database server actively accesses the target packet of the message queue, checks whether queue data exists, and if so, extracts and stores the queue data into the database.
It should be noted that step 204 and steps 202 and 203 are in parallel, that is, the database server can extract and store the queue data in the message queue to the database while storing the target data and storing the target data in the message queue.
In summary, in the embodiment of the present application, each database server of the database system has a data writing capability, and the database servers automatically synchronize data with each other through a message queue while storing target data, so that the data stored by each database server in the database system are the same, and the data writing efficiency and the security and high availability of the database system are improved; the load balancer can realize data synchronization by only sending a data writing request to one database server at random, the running state of each database server does not need to be monitored before each data request is sent, the load of the load balancer is reduced, an additional controller is not needed, and the requirement of various databases on high availability can be met.
In one possible embodiment, the database server includes a synchronizer and a database, so that the database server realizes data storage and data synchronization. Fig. 3 shows a flowchart of interaction when a database system writes data, where the flowchart includes:
step 301, the load balancer sends a data write request to the synchronizer, where the data write request includes target data.
In one possible implementation, the load balancer communicates with a synchronizer in the database server, sending data write requests to the synchronizer.
For a specific implementation of step 301, reference may be made to step 201 described above, and details of this embodiment are not described herein again.
Step 302, the synchronizer receives a data write request and sends the target data to the database.
In step 303, the synchronizer starts a timer.
In one possible implementation, the synchronizer determines target data according to the data writing request, and sends the target data to the database, so that the database stores the data. In order to monitor whether the database is abnormal or not, the synchronizer executes step 302, simultaneously executes step 303 to start a timer, starts timing, and obtains the storage condition of the database for the target data within the time length of the timer, if normal feedback of the database can be obtained, the database is not abnormal, the synchronizer can continue to execute the subsequent steps, and if normal feedback of the database cannot be obtained within the time length of the timer or abnormal feedback of the database is obtained, the database is determined to be abnormal.
The database stores the target data, step 304.
And when the database works normally, the target data sent by the synchronizer can be received and stored. If the database is abnormal, the target data sent by the synchronizer cannot be received or cannot be stored.
After completing the storage of the target data, the database sends a stored message to the synchronizer, step 305.
In one possible embodiment, after the database writes data normally, the synchronizer is notified so that the synchronizer determines that the database is normal and continues data processing. And immediately sending the stored message to the synchronizer by the database after the target data is stored.
In another possible implementation, if the database cannot store the target data, a storage failure message is sent to the synchronizer, so that the synchronizer can timely know that the database is abnormal.
In step 306, the synchronizer stores the target data in the association packet after receiving the stored message.
In a possible implementation manner, if the synchronizer receives the stored message before the timer reaches the time length of the timer, it is determined that the database is normal, and the target data is successfully stored, so that only the target data needs to be stored in the association group, so that the synchronizer corresponding to the association server extracts the target data, and the writing of the target data in each database server in the database system is completed.
In order to facilitate the synchronizer to correctly store and extract data, in one possible embodiment, a configuration file is stored in the database server, and the configuration file includes configuration information of the database server and an associated database server, that is, configuration information of all database servers in the database system. The configuration information comprises at least one of server identification, server ports, grouping identification and grouping ports, the server identification corresponds to the server ports one to one, and the grouping identification corresponds to the grouping ports one to one. The database server stores the configuration file before data reading and writing work is carried out.
Based on the information contained in the configuration file, step 306 further includes the following steps:
step 306a, the synchronizer determines an associated packet port according to the packet identifier corresponding to the associated database server in the configuration file, and stores the target data into the associated packet through the associated packet port.
In one possible embodiment, the configuration file includes the server identification of each database server, the packet identification of the corresponding packet, and the packet port. And the synchronizer inquires and obtains an associated packet port of the associated packet based on the server identifier and the packet identifier, so that the server where the message queue is located communicates through the associated packet port, and the target data is stored in the associated packet.
And 307, when the timer reaches the time length of the timer and the stored message is not received, the synchronizer determines that the database is in an abnormal state and stores the target data into the association group and the target group.
If the timer reaches the time length of the timer and the synchronizer does not receive the stored message, the synchronizer determines that the database is in an abnormal state, at the moment, the target data not only needs to be stored in the association group to enable the association server to store the target data, but also needs to be stored in the target group, and the target data which cannot be written in is conveniently acquired from the target group to be recovered after the subsequent database is recovered to be normal. By the method, data are written in, the state of each database server does not need to be judged by the controller, and automatic data backup can be realized among the database servers.
Optionally, when the timer reaches the time length of the timer and does not receive the stored message, the synchronizer determines that the database is in an abnormal state; and when the timer reaches the time length of the timer and the storage failure message is received, the synchronizer determines that the database is in an abnormal state.
In a possible implementation mode, the target group stores the queue data stored by the associated database server and/or the target data stored by the synchronizer during the abnormal period of the database, and when the database is in a normal state, the database needs to store the data in the target group. The data writing process in the embodiment of the present application further includes the following steps:
step one, when the database is not in an abnormal state, the synchronizer extracts the queue data in the target grouping and sends the queue data to the database.
Optionally, the synchronizer sends the target data included in the received data write-in request to the database in real time, judges whether the database is abnormal or not by monitoring whether the stored message is received within the time length of the timer, stores the target data to the target permission if the database is abnormal, and sends the queue data in the target group to the database if the database is normal; or after determining that the database is abnormal, the synchronizer directly stores the target data into the target group and sends the queue data in the target group to the server every preset time so as to judge whether the database is normal or not.
Based on the information in the configuration file, in one possible embodiment, the first step includes: and the synchronizer extracts the queue data through a target packet port corresponding to the target packet in the configuration file.
The synchronizer utilizes the configuration file to inquire a target group and a target group port corresponding to a database server where the synchronizer is located, communicates with the server where the message queue is located through the target group port, extracts queue data contained in the target group and sends the queue data to the database.
And step two, the database stores the queue data.
And step three, after the database finishes storing the queue data, sending the stored message to the synchronizer.
In order to facilitate the synchronizer to know whether the database is abnormal in time, the database needs to send the stored message to the synchronizer after successfully storing the queue data. If the synchronizer does not receive the stored message sent by the database within the time length of the timer, determining that the database is abnormal and storing the queue data back to the target group; if the database receives a stored message within the timer duration, it is determined that the queue data has been stored by the database.
In a possible implementation manner, when the synchronizer determines that the database is abnormal, an abnormality warning is sent to the computer device corresponding to the operation and maintenance personnel through the load balancer, or the database server directly sends the abnormality warning to the computer device to inform the operation and maintenance personnel to handle the abnormality in time.
In the embodiment of the application, the synchronizer judges whether the database is abnormal or not by detecting whether feedback of successfully storing the target data in the database is received within the time length of the timer or not, and stores the target data into the associated packet and the target packet in the message queue when the database is determined to be abnormal, so that the associated database server can store the target data, and can automatically recover the data by storing the queue data in the target packet after the database is recovered to be normal, the high availability of the database system is improved, and the data loss under the abnormal condition is avoided.
The foregoing embodiment describes a flow of the database system in the present application when writing data, and when the load balancer receives a database modification instruction, it is also necessary that each database server in the database system performs the same modification, so that it is possible to ensure that data stored in each database is consistent. Fig. 4 shows an interaction flowchart when the database system performs a database modification operation, where the interaction flowchart includes:
step 401, the load balancer sends a database modification instruction to the database server, where the database modification instruction is generated by the computer device.
The user carries out database and table modification operations of the database through the computer equipment, wherein the database and table modification operations comprise library and table addition, deletion, adjustment and the like, and the computer equipment generates a corresponding database modification instruction according to the user operation and sends the database modification instruction to the database server.
The load balancer randomly sends the received database modification instruction to a database server, or performs average distribution according to a preset sequence, as the data writing instruction.
Step 402, the database server executes a database modification operation according to the database modification instruction.
After receiving the database modification instruction forwarded by the load balancer, the database server performs corresponding modification operation on the database according to the database modification instruction.
In step 403, the database server sends a modified message or a modification failure message to the load balancer.
In order to facilitate the load balancer to obtain the database modification result, the database server sends a modified message or a modification failure message to the load balancer according to the modification result after the database executes the database modification instruction, so that the load balancer feeds back to the computer equipment according to the received message, and a user can take measures in time when finding an abnormality.
Step 404, the database server forwards the database modification instruction to the associated database server through a server port corresponding to the associated database server, and the associated database server is configured to perform a database modification operation according to the database modification instruction and send a modified message or a modification failure message to the load balancer.
Because data synchronization is needed among the database servers in the database system, after the database servers perform database modification operation, other databases in the database system, namely, associated databases, also need to perform the same database modification operation.
In one possible embodiment, a synchronizer in a database server forwards database modification instructions to a synchronizer in an associated database server. The synchronizer queries and acquires a server port of the associated database server based on the configuration file, and sends a database modification instruction to the synchronizer in the associated database server through the server port.
And after the synchronizer in the associated database server receives the database modification instruction, modifying the database in the same way, and sending a modified message or a modification failure message to the load balancer based on the modification result so that the load balancer can acquire the execution result of each database server in the database system.
In step 405, the load balancer sends modification failure prompt information to the computer device when receiving the modification failure message.
When the load balancer receives a modification failure message once or does not receive modified messages sent by all the database servers within a preset time, determining that the database server fails to execute the database modification instruction, and sending modification failure prompt information to the computer equipment.
In a possible implementation manner, the load balancer sends modification failure prompt information to the computer device, and prompts a user that corresponding operation needs to be performed again when the modification fails. If the number of times that the load balancer continuously receives the modification failure message reaches the number threshold value and the database servers which fail to be modified each time are the same, the load balancer sends the server identification of the database server which correspondingly sends the modification failure message to the computer equipment, so that a user can check and maintain the data block server.
In the embodiment of the application, after receiving the database modification instruction, the load balancer only needs to randomly send the database modification instruction to one database server, and the database servers forward messages through the synchronizer, so that the synchronous modification of the database is realized, and the consistency of the database is ensured; when the operation of the database server fails, the load balancer sends prompt information of the operation failure to the computer equipment so as to prompt the user to operate again or perform exception handling.
In addition to data writing and database modification operations, the database system in the embodiment of the present application further has a data reading function, and fig. 5 shows an interaction flow chart when the database system performs data reading according to an embodiment of the present application, where the interaction flow chart includes:
step 501, the load balancer sends a data reading request to the database server, where the data reading request is used to indicate data to be read.
The data reading method includes the steps that a user carries out data reading operation through computer equipment, the computer equipment generates a data reading request based on the user operation, the data reading request is used for indicating data to be read, for example, when the user needs to inquire data generated by the Internet of things equipment A within a certain time period, the data reading request comprises an equipment identifier corresponding to the Internet of things equipment A in a database and the time period in an inquiry condition.
The load balancer randomly sends the received data read request to a database server, or performs average distribution according to a preset sequence, as with the data write request and the database modification instruction.
Step 502, the database server sends data to be read to the load balancer.
And the database server screens the data to be read from the database based on the query conditions in the data reading request, sends the data to be read to the load balancer, and returns the query result to the computer equipment by the load balancer.
In step 503, when the data to be read is not received within the preset time, the load balancer sends a data read request to the associated database server.
In order to avoid the situation that the data cannot be read due to the fact that a database server receiving a data reading instruction is abnormal, timing is started after the load balancer sends the data reading instruction to the database server, if the load balancer does not receive data to be read sent by the database server within a preset time length or receives a data reading failure message sent by the database server, it is determined that the database server is abnormal, a data reading request is sent to any one associated database server, and the associated database server reads the data.
In a possible implementation manner, if the number of times of data reading failure is greater than the preset number of times, it is determined that the database does not contain data to be read which meets the data reading instruction, or data in the database is lost, and the load balancer reports prompt information of data reading failure to the computer device. And if the data reading of the database server fails and the data reading of the associated database server succeeds, determining that the data of the database server is lost or abnormal, and sending a database abnormal warning to the corresponding computer equipment so that operation and maintenance personnel can process the data in time.
In the embodiment of the application, the load balancer randomly sends a data reading request to the database server, and if the database server is abnormal and the load balancer does not receive data to be read, the load balancer forwards the data reading request to other database servers, so that data query can be still performed under the condition that one database server is abnormal and cannot read the data, and the real-time query function of the database system is realized.
In conjunction with the above embodiments, in an illustrative example, the flow of data writing performed by the database system is shown in fig. 6. The figure shows a database system comprising two database servers (database server a and database server B), but in other embodiments, the database system may comprise more database servers, and the examples of the present application are not limited thereto.
In step 601, the load balancer sends a data write request to synchronizer a.
In step 602, synchronizer A sends the target data to database A.
In step 603, synchronizer a stores the target data in packet B of the message queue.
Synchronizer B extracts the data in packet B, step 604.
In step 605, the synchronizer B sends the extracted data to the database B.
The specific flow of the steps executed by the synchronizer is shown in fig. 7.
Step 701, determining whether data exists in a target packet in a message queue. If yes, go to step 702, otherwise go to step 701.
Step 702, write the data into the database.
Step 703, confirming whether a data write request is received. If so, steps 704 and 705 are performed, otherwise, step 703 is continued.
Step 704, write the data into the database.
Step 705, store data to associated packets in the message queue.
The steps 701 to 702 and the steps 703 to 705 are parallel.
In connection with the above embodiments, in an illustrative example, when there is an abnormality in the database server a, the flow of data writing performed by the database system is as shown in fig. 8. The figure shows a database system comprising two database servers (database server a and database server B), but in other embodiments, the database system may comprise more database servers, and the examples of the present application are not limited thereto.
In step 801, the load balancer sends a data write request to synchronizer A.
In step 802, synchronizer A sends the target data to database A. If it is determined that there is an exception in database a, step 803 is continued.
In step 803, synchronizer a stores the target data in packet a and packet B of the message queue.
In step 804, synchronizer B extracts the data in packet B.
In step 805, the synchronizer B sends the extracted data to the database B.
When synchronizer A determines that database A is back to normal, step 806 is performed.
In step 806, after the database a recovers to normal, the synchronizer a sends the data in the packet a to the database a.
In conjunction with the above embodiments, in an illustrative example, the flow of data reading by the database system is shown in fig. 9.
Step 901, randomly selecting a database server and sending a data reading request.
Step 902, determine whether the database server is normal. If yes, go to step 903, otherwise return to step 901.
Step 903, sending a data reading request.
Referring to fig. 10, a schematic diagram of a server according to an exemplary embodiment of the present application is shown, where the computer device may be implemented as a load balancer in a database system or as a database server.
The server 1000 includes a Central Processing Unit (CPU) 1001, a Random Access Memory (RAM) 1002, a system Memory 1004 of a Read-Only Memory (ROM) 1003, and a system bus 1005 connecting the system Memory 1004 and the CPU 1001. The server 1000 also includes a basic Input/Output system (I/O system) 1006, which facilitates the transfer of information between devices within the computer, and a mass storage device 1007, which stores an operating system 1013, application programs 1014, and other program modules 1015.
The basic input/output system 1006 includes a display 1008 for displaying information and an input device 1009, such as a mouse, keyboard, etc., for user input of information. Wherein a display 1008 and an input device 1009 are connected to the central processing unit 1001 via an input-output controller 1010 connected to the system bus 1005. The basic input/output system 1006 may also include an input/output controller 1010 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input-output controller 1010 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 1007 is connected to the central processing unit 1001 through a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1007 and its associated computer-readable media provide non-volatile storage for the server 1000. That is, the mass storage device 1007 may include a computer-readable medium (not shown) such as a hard disk or a Compact disk Read-Only Memory (CD-ROM).
Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, computer Memory (EPROM), Read-write Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 1004 and mass storage device 1007 described above may be collectively referred to as memory.
The memory stores one or more programs configured to be executed by the one or more central processing units 1001, the one or more programs containing instructions for implementing the functions of the server described above, and the central processing unit 1001 executes the one or more programs to implement the functions of the server in the respective embodiments described above.
According to various embodiments of the present application, the server 1000 may also operate as a remote computer connected to a network through a network, such as the Internet. That is, the server 1000 may be connected to the network 1012 through a network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 may be used to connect to another type of network or a remote computer system (not shown).
The memory also includes one or more programs, stored in the memory, which include instructions for performing the steps provided by the embodiments of the present application as performed by the server.
Embodiments of the present application further provide a computer-readable storage medium, which stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the functions of the database server or the load balancer in the database system according to the above embodiments.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the server reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the server implements the functions of the database server or the load balancer in the database system provided in the various alternative implementations of the above aspects.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A database system, the database system comprising: the system comprises at least two database servers and a load balancer, wherein the at least two database servers are associated with each other and store the same data;
the load balancer is used for sending a data writing request to the database server, wherein the data writing request comprises target data;
the database server is used for storing the target data according to the received data writing request;
the database server is further configured to store the target data into an associated packet in a message queue, where the associated packet is a packet corresponding to the associated database server;
the database server is further configured to extract queue data in the message queue target packet and store the queue data, where the queue data is stored to the target packet by the associated database server, and the target packet is a packet corresponding to the database server.
2. The system of claim 1, wherein the database server comprises a synchronizer and a database;
the synchronizer is used for receiving the data writing request and sending the target data to the database;
the database is used for storing the target data;
the database is further used for sending a stored message to the synchronizer after the target data is stored;
the synchronizer is further configured to store the target data in the association packet after receiving the stored message.
3. The system of claim 2,
the synchronizer is used for starting a timer when the target data is sent to the database;
the synchronizer is further configured to determine that the database is in an abnormal state and store the target data to the association packet and the target packet when the timer reaches a timer duration and the stored message is not received.
4. The system of claim 2,
the synchronizer is used for extracting the queue data in the target grouping and sending the queue data to the database when the database is not in the abnormal state;
the database is used for storing the queue data;
the database is further configured to send the stored message to the synchronizer after the storage of the queue data is completed.
5. The system according to any one of claims 2 to 4, wherein a configuration file is stored in the database server, the configuration file includes configuration information of the database server and the associated database server, the configuration information includes at least one of a server identifier, a server port, a group identifier, and a group port, and the server identifier corresponds to the server port one to one, and the group identifier corresponds to the group port one to one.
6. The system of claim 5, wherein the message queue resides in a message server;
the synchronizer is used for determining an associated grouping port according to the grouping identification corresponding to the associated database server in the configuration file and storing the target data to the associated grouping through the associated grouping port;
the synchronizer is further configured to extract the queue data through a target packet port corresponding to the target packet in the configuration file.
7. The system of claim 5 or 6,
the load balancer is used for sending a database modification instruction to the database server, and the database modification instruction is generated by computer equipment;
the database server is used for executing database modification operation according to the database modification instruction and sending a modified message or a modification failure message to the load balancer;
the database server is further configured to forward the database modification instruction to the associated database server through the server port corresponding to the associated database server, where the associated database server is configured to execute the database modification operation according to the database modification instruction, and send the modified message or the modification failure message to the load balancer;
and the load balancer is further used for sending modification failure prompt information to the computer equipment when receiving the modification failure message.
8. The system according to any one of claims 1 to 4,
the load balancer is further used for sending a data reading request to the database server, wherein the data reading request is used for indicating data to be read;
the database server is used for sending the data to be read to the load balancer;
and the load balancer is further used for sending the data reading request to the associated database server when the data to be read is not received within a preset time length.
9. A server, comprising a processor and a memory; the memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded and executed by the processor to implement the functions of a database server or load balancer in a database system as claimed in any one of claims 1 to 8.
10. A computer-readable storage medium having at least one program code stored therein, the program code being loaded and executed by a processor to implement the functions of a database server or a load balancer in a database system as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110853140.3A CN113609199B (en) | 2021-07-27 | 2021-07-27 | Database system, server, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110853140.3A CN113609199B (en) | 2021-07-27 | 2021-07-27 | Database system, server, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113609199A true CN113609199A (en) | 2021-11-05 |
| CN113609199B CN113609199B (en) | 2023-09-12 |
Family
ID=78305688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110853140.3A Active CN113609199B (en) | 2021-07-27 | 2021-07-27 | Database system, server, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113609199B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765798A (en) * | 2015-03-26 | 2015-07-08 | 深圳市一科互联有限公司 | System and method for achieving Mysql remote synchronous fault tolerance enhancement |
| US20160292019A1 (en) * | 2015-03-31 | 2016-10-06 | Mckesson Corporation | Systems and Methods for Servicing Database Events |
| CN106201666A (en) * | 2016-06-24 | 2016-12-07 | 浪潮软件股份有限公司 | A kind of distributed transaction implementation method |
| CN108134745A (en) * | 2017-12-19 | 2018-06-08 | 深圳交控科技有限公司 | Distribution type data collection method and device |
| CN109739929A (en) * | 2018-12-18 | 2019-05-10 | 中国人民财产保险股份有限公司 | Method of data synchronization, apparatus and system |
| CN109783573A (en) * | 2018-12-18 | 2019-05-21 | 北京华夏电通科技有限公司 | The method of data synchronization and terminal of multichannel push |
| US20190251180A1 (en) * | 2018-02-13 | 2019-08-15 | Citrix Systems, Inc. | Live zero downtime migration of databases with disparate schemata |
| CN110691124A (en) * | 2019-09-24 | 2020-01-14 | 华润网络(深圳)有限公司 | Data processing method and system under remote multi-active architecture |
| CN111708637A (en) * | 2020-06-18 | 2020-09-25 | 深圳市金斧子网络科技有限公司 | Data processing method and device and computer readable medium |
| CN113094434A (en) * | 2021-04-08 | 2021-07-09 | 中国工商银行股份有限公司 | Database synchronization method, system, device, electronic equipment and medium |
-
2021
- 2021-07-27 CN CN202110853140.3A patent/CN113609199B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765798A (en) * | 2015-03-26 | 2015-07-08 | 深圳市一科互联有限公司 | System and method for achieving Mysql remote synchronous fault tolerance enhancement |
| US20160292019A1 (en) * | 2015-03-31 | 2016-10-06 | Mckesson Corporation | Systems and Methods for Servicing Database Events |
| CN106201666A (en) * | 2016-06-24 | 2016-12-07 | 浪潮软件股份有限公司 | A kind of distributed transaction implementation method |
| CN108134745A (en) * | 2017-12-19 | 2018-06-08 | 深圳交控科技有限公司 | Distribution type data collection method and device |
| US20190251180A1 (en) * | 2018-02-13 | 2019-08-15 | Citrix Systems, Inc. | Live zero downtime migration of databases with disparate schemata |
| CN109739929A (en) * | 2018-12-18 | 2019-05-10 | 中国人民财产保险股份有限公司 | Method of data synchronization, apparatus and system |
| CN109783573A (en) * | 2018-12-18 | 2019-05-21 | 北京华夏电通科技有限公司 | The method of data synchronization and terminal of multichannel push |
| CN110691124A (en) * | 2019-09-24 | 2020-01-14 | 华润网络(深圳)有限公司 | Data processing method and system under remote multi-active architecture |
| CN111708637A (en) * | 2020-06-18 | 2020-09-25 | 深圳市金斧子网络科技有限公司 | Data processing method and device and computer readable medium |
| CN113094434A (en) * | 2021-04-08 | 2021-07-09 | 中国工商银行股份有限公司 | Database synchronization method, system, device, electronic equipment and medium |
Non-Patent Citations (2)
| Title |
|---|
| DHARMIT PATEL 等: "Towards in-order and exactly-once delivery using hierarchical distributed message queues", 《CCGRID \'14: PROCEEDINGS OF THE 14TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD, AND GRID COMPUTING》, pages 883 * |
| 王冠男: "多源日志分析平台的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 09, pages 138 - 573 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113609199B (en) | 2023-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108170768B (en) | Database synchronization method, device and readable medium | |
| US20220269564A1 (en) | Processing node management method, configuration method, and related apparatus | |
| CN103455368B (en) | A kind of deadlock detection method, node and system | |
| JP4722944B2 (en) | System, method and software for distributed loading of databases | |
| US11157373B2 (en) | Prioritized transfer of failure event log data | |
| WO2018214887A1 (en) | Data storage method, storage server, storage medium and system | |
| US11409711B2 (en) | Barriers for dependent operations among sharded data stores | |
| US20180032567A1 (en) | Method and device for processing data blocks in a distributed database | |
| WO2021112909A1 (en) | Managing replication state for deleted objects | |
| CN108140035B (en) | Database replication method and device for distributed system | |
| CN109586989B (en) | State checking method, device and cluster system | |
| US20230376470A1 (en) | Moving Window Data Deduplication in Distributed Storage | |
| CN113609199B (en) | Database system, server, and storage medium | |
| CN110837428B (en) | Storage device management method and device | |
| CN111400100B (en) | Management method and system for distributed software backup | |
| CN111324668B (en) | Database data synchronous processing method, device and storage medium | |
| US11381642B2 (en) | Distributed storage system suitable for sensor data | |
| CN106168919A (en) | A kind of database backup method, Apparatus and system | |
| US10885014B2 (en) | Assigning monitoring responsibilities in distributed systems using optimistic concurrency | |
| CN108023914B (en) | Memory data sharing system, and memory data writing and reading method | |
| US20220405306A1 (en) | Database replication system and method, source end device, and destination end device | |
| CN117349384B (en) | Database synchronization method, system and equipment | |
| US20210248108A1 (en) | Asynchronous data synchronization and reconciliation | |
| CN115168109A (en) | Data recovery method, device, equipment and storage medium | |
| JPH11232153A (en) | Data base system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |