CN111339131A - Data reading and writing method and system - Google Patents
Data reading and writing method and system Download PDFInfo
- Publication number
- CN111339131A CN111339131A CN202010120955.6A CN202010120955A CN111339131A CN 111339131 A CN111339131 A CN 111339131A CN 202010120955 A CN202010120955 A CN 202010120955A CN 111339131 A CN111339131 A CN 111339131A
- Authority
- CN
- China
- Prior art keywords
- database
- server
- data
- sql statement
- sql
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24549—Run-time optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2445—Data retrieval commands; View definitions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data reading and writing method and system. The method comprises the following steps: when the application server receives the data operation request, judging whether the application server is connected with the database proxy server or not; when the application server determines to connect the database proxy server, the application server sends a data operation request to the connected database proxy server; the database proxy server determines a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request; the database proxy server sends each SQL statement to a database server of an operation database corresponding to the SQL statement; and the database server receives the SQL sentences and executes the SQL sentences to the operation database to complete data reading and writing. By adopting the method, the response speed and the usability can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data reading and writing method and system.
Background
With the development of computer and internet applications, online business systems have been commonly used in business processes in various industries. And, with the expansion of services, more and more users use the service application system. The use of the users is increased, so that the access amount of the business application system and the operation amount of the database are increased greatly. Since the availability of the business application system depends on the response speed of the database, the use of the business application system is affected once the database has response problems.
In order to relieve the pressure of the database, a caching technology is generally adopted. However, since the caching technology is usually only capable of caching the frequently used data in advance, the caching technology is only suitable for the business system with less data quantity and data updating. Availability is not high for systems that have frequent access, large amounts of data, and fast data updates.
Disclosure of Invention
In view of the above, it is necessary to provide a data read/write method and system capable of improving system availability in view of the above technical problems.
A method of reading and writing data, the method comprising:
when an application server receives a data operation request, judging whether the application server is connected with a database proxy server or not;
when the application server determines that the database proxy server is connected, the application server sends the data operation request to the connected database proxy server;
the database proxy server determines a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request;
the database proxy server sends each SQL statement to a database server of the operation database corresponding to the SQL statement;
and the database server of the operation database receives the SQL statement and executes the SQL statement to the operation database to complete data reading and writing.
In one embodiment, the determining, by the database proxy server, a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request includes:
when the SQL statement is an inserted SQL-like statement, determining a readable and writable main database as an operation database of the SQL statement;
and when the SQL statement is a query SQL-like statement, determining the readable slave database as an operation database of the SQL statement.
In one embodiment, the method further comprises:
when the application server determines that the database proxy server is not connected, the application server determines the request type of the data operation request according to the type of the SQL statement corresponding to the data operation request;
the application server determines an operation database according to the request type and sends the data operation request to a database server corresponding to the operation database;
and the database server receives the data operation request, executes SQL (structured query language) statements in the data operation request on the operation database and finishes data reading and writing.
In one embodiment, the determining, by the application server, the request type of the data operation request according to the type of the SQL statement corresponding to the data operation request includes:
when the data operation request comprises an insert SQL-like statement, determining the request type of the data operation request as an insert type;
and when the data operation request only comprises a query SQL type statement, determining the request type of the data operation request as a query type.
In one embodiment, the determining, by the application server, the operation database according to the request type includes:
when the request type is an insertion type, determining a readable and writable main database as an operation database;
and when the request type is a query type, determining the readable slave database as an operation database.
In one embodiment, the database server comprises a master database server and a slave database server; the method further comprises the following steps:
when a master database has data updating, the master database server generates an updating event and writes the updating event into a binary log;
the master database server sends the file name of the binary log and the updating position of the binary log to the slave database server through a log backup thread;
the slave database server requests the master database server to acquire a copy of the update event in the binary log according to the file name and the update position;
and the database server stores the copy of the update event into a local relay log, and redos the copy of the update event according to the copy of the update event to complete data synchronization with a main database.
In one embodiment, the determining, by the application server, whether to connect to the database proxy server includes:
acquiring a state value of a filter;
and determining whether to connect the database proxy server according to the state value of the filter.
A data reading and writing system, the system comprising:
the application server is used for judging whether the application server is connected with the database proxy server or not when receiving the data operation request;
the application server is also used for sending the data operation request to the connected database proxy server when the database proxy server is determined to be connected;
the database proxy server is used for determining a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request;
the database proxy server is also used for respectively sending each SQL statement to the database server of the operation database corresponding to the SQL statement;
and the database server is used for receiving the SQL sentences and executing the SQL sentences to the operation database to complete data reading and writing.
In one embodiment, the database proxy server is further configured to:
when the SQL statement is an inserted SQL-like statement, determining a readable and writable main database as an operation database of the SQL statement;
and when the SQL statement is a query SQL-like statement, determining the readable slave database as an operation database of the SQL statement.
In one embodiment, the application server is further configured to determine a request type of the data operation request according to a type of an SQL statement corresponding to the data operation request when it is determined that the database proxy server is not connected;
the application server is also used for determining an operation database according to the request type and sending the data operation request to a database server corresponding to the operation database;
and the database server is also used for receiving the data operation request, executing SQL sentences in the data operation request on the operation database and finishing data reading and writing.
According to the data reading and writing method and system, after the application server receives the data operation request, the data operation request is sent to the connected database proxy server, the database proxy server determines the corresponding operation database for each SQL statement according to the type of the SQL statement in the data operation request, and then the SQL statement is sent to the database server of the corresponding operation database to be executed. According to the method, different types of request SQL statements are subdivided to corresponding operation databases through the database proxy server, and data operation separation is realized through a fine-grained working mode of the database proxy server, so that the response speed of the database is improved, the database is not limited by the type of a system, and the usability is improved.
Drawings
FIG. 1 is a diagram illustrating an exemplary implementation of a method for reading and writing data;
FIG. 2 is a diagram illustrating an application scenario of a data read/write method in another embodiment;
FIG. 3 is a flow chart illustrating a method for reading and writing data according to an embodiment;
FIG. 4 is a flowchart illustrating a data read/write method according to an embodiment;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data reading and writing method provided by the application can be applied to the application environment shown in fig. 1. The application environment relates to a terminal 10, an application server 20, a database proxy server 30, a database server 40. The terminal 10 communicates with the application server 20 through a network, and the application server 20, the database proxy server 30, and the database server 40 may communicate with each other through a network. After the application server 20 receives the data operation request sent by the terminal 10, judging whether the application server 10 is connected with the database proxy server 30; when the application server 10 determines to connect the database proxy server 30, the application server 10 transmits a data operation request to the connected database proxy server 30; the database proxy server 30 determines a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request; the database proxy server 30 sends each SQL statement to the database server 40 of the operation database corresponding to the SQL statement; the database server 40 receives the SQL statements and executes the SQL statements on the operation database to complete data reading and writing. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the application server 20, the database proxy server 30, and the database server 40 may be implemented by independent servers or a server cluster formed by a plurality of servers.
In one embodiment, as shown in FIG. 2, database server 40 includes a master database server 402 and a slave database server 404. The master database server 402, the slave database server 404, the application server 20, and the database proxy server 30 communicate with each other via a network. When the database proxy server 30 determines that the corresponding operation database is the master database according to the type of the SQL statement, the database proxy server 30 transmits the SQL statement to the master database server 402 corresponding to the master database. When the database proxy server 30 determines that the corresponding operation database is the slave database according to the type of the SQL statement, the database proxy server 30 sends the SQL statement to the slave database server 404 corresponding to the slave database. The master database server 402 and the slave database server 404 may be implemented by separate servers or a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 3, a method for reading and writing data is provided, the method comprising the steps of:
step S301, when the application server receives the data operation request, judging whether the application server is connected with the database proxy server.
Step S302, when the application server determines to connect the database proxy server, the application server sends a data operation request to the database proxy server.
The application server is a server providing service logic for the application program, and may be understood as a server corresponding to the service application system. For example, the server developed by the steam distribution trader is used for the corresponding server of the steam distribution trading platform. The data operation request is an instruction for requesting the application server to perform a relevant operation on the specified data. Such as adding data, querying data, deleting data, and so forth. The database proxy server is a proxy server which performs data operation on the application server.
Specifically, the terminal automatically generates a corresponding data operation request according to the operation of the user on each interface on the service application system. For example, if the user clicks the login interface, the corresponding data operation request is generated according to the business logic rule of the login interface. And when the user clicks the registration interface, generating a corresponding data operation request according to the service logic rule of the registration interface. That is, when the business application system is developed, the business logic rules of the interfaces are inconsistent according to the functional requirements. Therefore, when a user operates different interfaces, inconsistent data operation requests can be generated according to the business logic rules of the interfaces. The specific method can be governed by the logic of the business application system actually used. And after the terminal generates the data operation request, sending the data operation request to an application server corresponding to the service application system. When receiving a data operation request, the application server firstly judges whether the application server is connected with a database proxy server. And after determining that the database proxy server is connected, sending the data operation request to the connected database proxy server.
In one embodiment, the generation of the data operation request is described in detail by taking a business application system as an example of a steam distribution transaction platform. Specifically, the steam distribution transaction platform comprises a registration interface, a login interface and a creation interface. When the automobile distribution transaction platform is developed, the business logic rule of the registration interface needs to firstly inquire whether the user information registered at this time exists in the database. And when the user information registered this time does not exist in the database, adding the registered information this time into the database. Therefore, after the user inputs the registration information in the input box of the automobile distribution transaction platform terminal and clicks the registration interface, the terminal can automatically generate two SQL statements. The two SQL sentences are the data operation request generated this time.
The business logic rule of the login interface of the automobile distribution trading platform is to firstly inquire whether the user exists according to the login information of the user. And if so, inquiring the affiliated organization information of the user. After the user clicks the login interface at the terminal of the automobile distribution transaction platform, the terminal can automatically generate two SQL query statements as the data operation request. And the business logic rule of the creation interface of the steam distribution trading platform is to newly add a corresponding information sheet according to the creation information input by the user. After the user clicks the creation interface at the terminal of the automobile distribution transaction platform, the terminal automatically generates a corresponding number of newly added SQL statements as the data operation request according to the number created by the user.
Step S304, the database proxy server determines corresponding operation databases for the SQL sentences according to the types of the SQL sentences in the data operation request.
Among them, SQL statements (Structured Query Language) are a kind of database Query and programming Language, and users store data and Query, update, and manage database systems. Therefore, the instructions generated by operating on the data are all SQL statements. The types of SQL statements include a query class and an insert class. It can be understood that, when the SQL statement is a statement that needs to change data in the database, such as new data, updated data, deleted data, and the like, the SQL statement is an insertion class. When the SQL statement queries data from the database, statements which do not modify the data and other operations are query classes. The SQL statement corresponding to the newly added data is insert (), the SQL statement corresponding to the query data is select (), the SQL statement corresponding to the updated data is update (), and the SQL statement corresponding to the deleted data is delete (). The operation database refers to a database for performing data operation on the operation database, and in the embodiment, the operation database comprises a readable and writable master database and a readable and writable slave database. The master database and the slave database are independent databases, and any database can be used as long as the database is supported by the Mycat data read-write separation middleware, and the database includes but is not limited to MySQL, Oracle, DB2, SQL Server, PostgreSQL and other types of databases.
In one embodiment, the determining, by the database proxy server, the corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request specifically includes: when the SQL statement is an inserted SQL-like statement, determining that the readable and writable main database is an operation database of the SQL statement; when the SQL statement is a query SQL-like statement, readable operation data of the SQL statement from the database is determined.
Specifically, the database corresponding to the new SQL statement insert (), the update SQL statement update (), or the delete SQL statement delete () in the data operation request is a readable and writable main database. The database corresponding to the query SQL statement select () in the data operation request is a readable slave database.
Step S306, the database proxy server sends each SQL statement to the database server of the operation database corresponding to the SQL statement.
Specifically, after the database proxy server determines the operation database corresponding to each SQL statement, the SQL statement may be sent to the database server corresponding to the operation database. A database server refers to a server that deploys an operational database. Since the operation database in this embodiment includes a master database and a slave database, that is, the database server includes a master database server corresponding to the master database and a slave database server corresponding to the slave database. And when the operation database determined by the SQL statement is a main database, sending the SQL statement to a main database server corresponding to the main database. And when the operation database determined by the SQL statement is the slave database, sending the SQL statement to a slave database server corresponding to the slave database.
Step S308, the database server of the operation database receives the SQL statement and executes the SQL statement to the operation database to complete data reading and writing.
Specifically, after the database server receives the SQL statement, the SQL statement may be executed on the operation database to complete reading and writing of data. For example, the newly added data is written into the operation database according to the newly added SQL statement insert (). And querying or acquiring corresponding data from the database according to the query SQL statement select ().
According to the data reading and writing method, after the application server receives the data operation request, the data operation request is sent to the connected database proxy server, the database proxy server determines the corresponding operation database for each SQL statement according to the type of the SQL statement in the data operation request, and then the SQL statement is sent to the database server of the corresponding operation database to be executed. According to the method, different types of request SQL statements are subdivided to corresponding operation databases through the database proxy server, and data operation separation is realized through a fine-grained working mode of the database proxy server, so that the response speed of the database is improved, the database is not limited by the type of a system, and the usability is improved.
In one embodiment, as shown in fig. 4, another data reading and writing method is provided, which includes the following steps:
step S402, when the application server determines that the database proxy server is not connected, the application server determines the request type of the data operation request according to the SQL statement corresponding to the data operation request.
In particular, the use of the business application system is prevented from being influenced by the interruption of data operation caused by the downtime or the failure of the database proxy server. After the application server receives the data operation request, the database proxy server connected with the application server is judged. When the database proxy server is disconnected from the database proxy server, the database proxy server is not available. The application server automatically responds to the data operation request, and determines the request type of the operation request according to the SQL statement of the data operation request, wherein the request type comprises an insertion type and a query type.
Because the application server has no technical support such as Mycat middleware and the like. Therefore, the application server cannot perform fine-grained work as a database proxy server, that is, the application server cannot divide each SQL statement in the data operation request into corresponding operation databases. The application server can only perform coarse-grained work, and one database is designated to be unified as an operation database of SQL statements in the data operation request. Therefore, the request type needs to be determined according to the SQL statement of the data operation request.
In one embodiment, the determining, by the application server, the request type of the data operation request according to the SQL statement corresponding to the data operation request specifically includes: when the data operation request comprises an insertion SQL-like statement, determining the request type of the data operation request as an insertion type; and when the data operation request only comprises the query SQL-like statement, determining the request type of the data operation request as the query type.
Specifically, the data operation request is of a query type only when all SQL statements of the data operation request are query-type statements. If the data operation request comprises any insert SQL-like statement, the data operation request is of an insert type.
Step S404, the application server determines an operation database according to the request type, and sends the data operation request to a database server corresponding to the operation database.
In one embodiment, the determining, by the application server, the operation database according to the request type specifically includes: when the request type is an insertion type, determining a readable and writable main database as an operation database; and when the request type is the query type, determining the readable slave database as the operation database.
Specifically, the data operation request of the insertion type includes an insertion SQL-like statement and a query SQL-like statement at the same time. The insert type data operation request needs to be read and written to the database at the same time, and thus the readable and writable master database is designated as the operation database of the insert type data operation request. And the query type data operation request only comprises a query type SQL statement. The query-type data operation request only needs to read the database, thus designating the only readable slave database as the operation database of the query-type data operation request. After the application server determines the operation database corresponding to the data operation request, the data operation request can be sent to the database server corresponding to the operation database. For example, the data operation request is sent to a master database server corresponding to the master database. And sending the data operation request to a slave database server corresponding to the slave database.
For example, taking the example that the data operation request includes two SQL statements, it is assumed that 0 represents the query SQL-like statement, 1 represents the insert SQL-like statement, a represents the connection master database, and B represents the connection slave database. The application server determines that the connected database can be represented by the following boolean algebra:
step S406, the database server receives the data operation request, and executes the SQL statement in the data operation request on the operation database to complete data reading and writing.
Specifically, after the database server receives the SQL statement, the SQL statement may be executed on the operation database to complete reading and writing of data. For example, the data to be deleted is deleted from the operation database according to the delete SQL statement delete ().
In this embodiment, when the database proxy server is unavailable, the application server responds to the instruction in time to perform data operation, thereby ensuring normal operation of the service. And moreover, the corresponding operation database can be classified and determined according to the data operation request, so that read-write data separation is realized, and the input and output pressure of the database is reduced.
In one embodiment, the data synchronization between the master database and the slave database through the shared operation log specifically includes: when the master database has data updating, the master database server generates an updating event and writes the updating event into a binary log; the master database server sends the file name of the binary log and the updating position of the binary log to the slave database server through a log backup thread; the slave database server requests the master database server to acquire a copy of the update event in the binary log according to the file name and the update position; and the database server stores the copy of the update event into a local relay log, and redos the copy according to the update event to complete data synchronization with the main database.
In order to prevent the situation that bidirectional data synchronization easily causes data inconsistency and the slave database is a read-only database, the data synchronization in this embodiment refers to a mechanism for keeping data in the slave database and data in the master database unified when data in the master database is changed.
Specifically, when the master database server changes data in the master database, such as adding, updating, deleting, and the like, by executing an SQL statement on the master database, the master database server simultaneously generates corresponding update events according to a preset binary format, and stores the generated update events into a binary log (bin log). The update event should include the modified data and corresponding data operations, for example, the update event may directly record the executed SQL statement. The binary log is a file for storing an update event, only changes to data are stored in the binary log, and data which is not changed cannot be written. Then, the master database server has a data update event through the slave database server by a log backup thread (log dump), and simultaneously transmits a file name of the binary log and an update location of this time to the slave database. The update position is the position where the update time is written into the binary log this time.
After receiving the file name and the update location of the binary log from the database server, requesting a master database server to obtain a copy of the update event based on the file name and the update location. That is, after the master database server receives a request from the database server, a corresponding update event is queried according to a file name and an update location provided from the database. And backups the update event and returns the backup copy to the slave database server. The received copy is then stored by the slave database server into a local relay log (relay log), which is a file used by the slave database server to save the data update event received from the master database server. And a special thread is arranged from the database server for monitoring the update condition of the relay log. And when the thread monitors that the relay log stores a new updating event, reading the updating event. And redo (redo) the current slave database according to the updating event, thereby completing the data updating of the slave database which is the same as that of the master database, and synchronizing the data of the slave database and the master database. In addition, after the events in the relay log are completely redone, the thread automatically deletes the relay log file. In the embodiment, the data of the master database and the slave database are ensured to be consistent through data synchronization, so that the data reading and writing are prevented from being influenced due to the inconsistent data.
In one embodiment, the step of determining, by the application server, whether to connect to the database proxy server specifically includes: acquiring a state value of a filter; and determining whether to connect the database proxy server according to the state value of the filter.
Wherein, the filter refers to a mechanism for monitoring whether the application server and the database proxy server are connected. And setting a corresponding switch in the filter, and when the state value of the switch indicates that the filter is opened, indicating that the application server and the database proxy server are in a connection state. When the state value of the switch indicates that the filter is closed, it indicates that the application server and the database proxy server are in an unconnected state.
Specifically, when the application server receives the data operation request, it can determine whether there is a connected database proxy server according to the state value of the filter. For example, if the state value 1 indicates on, 0 indicates off. And when the state value acquired by the application server is 1, determining that the database proxy server corresponding to the state value 1 is the database proxy server connected with the application server. When the status value is 0, it indicates that the database proxy server is not connected. In this embodiment, the connection state of the application server and the database proxy server is monitored by the filter, so that whether fine-grained data operation or coarse-grained data operation is performed can be accurately determined.
In one embodiment, the data reading and writing method specifically includes the following steps:
and S1, the terminal generates a corresponding data operation request according to the service logic of the interface clicked by the user, and sends the generated data operation request to the application server.
And S1, the application server receives the data operation request sent by the terminal, responds to the data operation request to acquire the state value of the filter and determines the available database proxy server according to the state value of the filter.
S3, when it is determined that there is an available database proxy server, sending a data operation request to the database proxy server.
And S4, the database proxy server receives the data operation request, and sends the inserted SQL like statement in the data operation request to the master database server and the query SQL like statement to the slave database server. The received SQL statements are executed by the master database server and the slave database server, respectively.
S5, when it is determined that there is no available database proxy server, the application server determines the operation type of the data operation request according to the type of the SQL statement in the data operation request. When the data operation request only comprises the query SQL-like statement, the data operation request is determined to be of the query type. Otherwise, the data operation request is of an insert type.
And S6, when the operation type of the data operation request is the insertion type, the application server sends the data operation request to the main data server, and the main database server executes the SQL statement in the data operation request.
And S7, when the operation type of the data operation request is the query type, the application server sends the data operation request to the slave data server, and the slave database server executes the SQL statement in the data operation request.
It should be understood that although the various steps in the flow charts of fig. 3-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 3-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, a computer device is provided, which may be an application server, a database proxy server, a master database server or a slave database server, and its internal structure diagram may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data read-write method.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of reading and writing data, the method comprising:
when an application server receives a data operation request, judging whether the application server is connected with a database proxy server or not;
when the application server determines to connect the database proxy server, the application server sends the data operation request to the connected database proxy server;
the database proxy server determines a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request;
the database proxy server sends each SQL statement to a database server of the operation database corresponding to the SQL statement;
and the database server of the operation database receives the SQL statement and executes the SQL statement to the operation database to complete data reading and writing.
2. The method of claim 1, wherein the determining, by the database proxy server, a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request comprises:
when the SQL statement is an inserted SQL-like statement, determining a readable and writable main database as an operation database of the SQL statement;
and when the SQL statement is a query SQL-like statement, determining the readable slave database as an operation database of the SQL statement.
3. The method of claim 1, further comprising:
when the application server determines that the database proxy server is not connected, the application server determines the request type of the data operation request according to the type of the SQL statement corresponding to the data operation request;
the application server determines an operation database according to the request type and sends the data operation request to a database server corresponding to the operation database;
and the database server receives the data operation request, executes SQL (structured query language) statements in the data operation request on the operation database and finishes data reading and writing.
4. The method according to claim 3, wherein the determining, by the application server, the request type of the data operation request according to the type of the SQL statement corresponding to the data operation request includes:
when the data operation request comprises an insert SQL-like statement, determining the request type of the data operation request as an insert type;
and when the data operation request only comprises a query SQL type statement, determining the request type of the data operation request as a query type.
5. The method of claim 3, wherein the application server determines an operation database according to the request type, comprising:
when the request type is an insertion type, determining a readable and writable main database as an operation database;
and when the request type is a query type, determining the readable slave database as an operation database.
6. The method of claim 1, wherein the database server comprises a master database server and a slave database server; the method further comprises the following steps:
when a master database has data updating, the master database server generates an updating event and writes the updating event into a binary log;
the master database server sends the file name of the binary log and the updating position of the binary log to the slave database server through a log backup thread;
the slave database server requests the master database server to acquire a copy of the update event in the binary log according to the file name and the update position;
and the database server stores the copy of the update event into a local relay log, and redos the copy of the update event according to the copy of the update event to complete data synchronization with a main database.
7. The method of claim 1, wherein the determining, by the application server, whether to connect to a database proxy server comprises:
acquiring a state value of a filter;
and determining whether to connect the database proxy server according to the state value of the filter.
8. A data reading and writing system, the system comprising:
the application server is used for judging whether the application server is connected with the database proxy server or not when receiving the data operation request;
the application server is also used for sending the data operation request to the connected database proxy server when the database proxy server is determined to be connected;
the database proxy server is used for determining a corresponding operation database for each SQL statement according to the type of each SQL statement in the data operation request;
the database proxy server is also used for respectively sending each SQL statement to the database server of the operation database corresponding to the SQL statement;
and the database server is used for receiving the SQL sentences and executing the SQL sentences to the operation database to complete data reading and writing.
9. The system of claim 8, wherein the database proxy server is further configured to:
when the SQL statement is an inserted SQL-like statement, determining a readable and writable main database as an operation database of the SQL statement;
and when the SQL statement is a query SQL-like statement, determining the readable slave database as an operation database of the SQL statement.
10. The system according to claim 8, wherein the application server is further configured to determine a request type of the data operation request according to a type of an SQL statement corresponding to the data operation request when it is determined that the database proxy server is not connected;
the application server is also used for determining an operation database according to the request type and sending the data operation request to a database server corresponding to the operation database;
and the database server is also used for receiving the data operation request, executing SQL sentences in the data operation request on the operation database and finishing data reading and writing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010120955.6A CN111339131A (en) | 2020-02-26 | 2020-02-26 | Data reading and writing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010120955.6A CN111339131A (en) | 2020-02-26 | 2020-02-26 | Data reading and writing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111339131A true CN111339131A (en) | 2020-06-26 |
Family
ID=71181950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010120955.6A Pending CN111339131A (en) | 2020-02-26 | 2020-02-26 | Data reading and writing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111339131A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897872A (en) * | 2020-07-03 | 2020-11-06 | 杭州浮云网络科技有限公司 | Large-scale MSSQL management method, system and computer equipment based on service center |
| CN111930789A (en) * | 2020-09-21 | 2020-11-13 | 北京东方通软件有限公司 | Automatic scheduling method and device for middleware of database access layer |
| CN112527817A (en) * | 2020-12-08 | 2021-03-19 | 北京大米科技有限公司 | Information processing method, information processing apparatus, storage medium, and electronic device |
| CN112783860A (en) * | 2021-01-11 | 2021-05-11 | 网易（杭州）网络有限公司 | Method and device for constructing mirror database, storage medium and computer equipment |
| CN114095514A (en) * | 2020-07-29 | 2022-02-25 | 华为技术有限公司 | Database access method and system |
-
2020
- 2020-02-26 CN CN202010120955.6A patent/CN111339131A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897872A (en) * | 2020-07-03 | 2020-11-06 | 杭州浮云网络科技有限公司 | Large-scale MSSQL management method, system and computer equipment based on service center |
| CN114095514A (en) * | 2020-07-29 | 2022-02-25 | 华为技术有限公司 | Database access method and system |
| CN114095514B (en) * | 2020-07-29 | 2023-03-10 | 华为技术有限公司 | Database access method and system |
| CN111930789A (en) * | 2020-09-21 | 2020-11-13 | 北京东方通软件有限公司 | Automatic scheduling method and device for middleware of database access layer |
| CN112527817A (en) * | 2020-12-08 | 2021-03-19 | 北京大米科技有限公司 | Information processing method, information processing apparatus, storage medium, and electronic device |
| CN112783860A (en) * | 2021-01-11 | 2021-05-11 | 网易（杭州）网络有限公司 | Method and device for constructing mirror database, storage medium and computer equipment |
| CN112783860B (en) * | 2021-01-11 | 2023-08-08 | 网易（杭州）网络有限公司 | Method, device, storage medium and computer equipment for constructing mirror image database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339131A (en) | Data reading and writing method and system | |
| US7640276B2 (en) | Backup system, program and backup method | |
| US7546429B1 (en) | Method of detection of data corruption in mirrored storage | |
| US7836162B2 (en) | Transaction processing system and transaction processing method | |
| CN101981550B (en) | Database system, database update method, database, and database update program | |
| JP4419884B2 (en) | Data replication apparatus, method, program, and storage system | |
| JP2020529673A (en) | High availability database with distributed store | |
| CN110781214A (en) | Database reading and writing method and device, computer equipment and storage medium | |
| US7698319B2 (en) | Database system management method, database system, database device, and backup program | |
| US20060167895A1 (en) | Database system and method for adapting a main database components in a main memory thereof | |
| EP4307137A1 (en) | Transaction processing method, distributed database system, cluster, and medium | |
| CN105144162A (en) | Integration database framework | |
| CN112307049A (en) | Method, device and equipment for separating read from write of database and readable storage medium | |
| US10671482B2 (en) | Providing consistency in a distributed data store | |
| CN111090701B (en) | Service request processing method, device, readable storage medium and computer equipment | |
| CN112015819A (en) | Data updating method, device, equipment and medium for distributed graph database | |
| CN116010446A (en) | Database switching method, device, equipment and medium | |
| CN111400273A (en) | Database capacity expansion method and device, electronic equipment and machine-readable storage medium | |
| CN114547108A (en) | Data processing method, device, equipment and medium | |
| CN115168367A (en) | Data configuration method and system for big data | |
| CN115391337A (en) | Database partitioning method and device, storage medium and electronic equipment | |
| CN114168636A (en) | Data processing method, device and equipment based on cache consistency and storage medium | |
| CN114328032A (en) | Disaster recovery switching processing method and device based on big data double activities and computer equipment | |
| CN112765126A (en) | Database transaction management method and device, computer equipment and storage medium | |
| CN114328711A (en) | Data processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |