CN113761038A - Data source read-write separation method and device - Google Patents

Data source read-write separation method and device Download PDF

Info

Publication number
CN113761038A
CN113761038A CN202011050571.8A CN202011050571A CN113761038A CN 113761038 A CN113761038 A CN 113761038A CN 202011050571 A CN202011050571 A CN 202011050571A CN 113761038 A CN113761038 A CN 113761038A
Authority
CN
China
Prior art keywords
data source
request
read
write
linking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011050571.8A
Other languages
Chinese (zh)
Inventor
赵力
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202011050571.8A priority Critical patent/CN113761038A/en
Publication of CN113761038A publication Critical patent/CN113761038A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data source read-write separation method and device, and relates to the technical field of computers. One embodiment of the method comprises: acquiring an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, taking one of the first data source and the second data source as a read data source, and taking the other one as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; when the operation type corresponding to the request is determined to be write operation, the request is linked to a write data source to carry out write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to a read data source for read operation. The implementation mode realizes the read-write separation of the data source in a non-invasive mode, has small invasion to the service code, supports the switching of different types of data sources and reduces the development cost.

Description

Data source read-write separation method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data source read-write separation method and apparatus, an electronic device, and a computer readable medium.
Background
At present, in order to improve the capability of a system for concurrently accessing a database and reduce the influence of a large number of transactions on the overall performance of a service, some schemes can be adopted for service vertical extension. The current common method adopts schemes of data slicing, read-write separation and the like. The read-write separation scheme can be realized by adding a Proxy layer (Proxy) at a server or developing codes at a service level.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: the data fragmentation can not solve the problem of hot data; a Proxy layer (Proxy) mode is added at a server, the Proxy layer needs to analyze a syntax tree of mysql (relational database management system), intercepts sql (the sql is the most common standardized language for accessing a database), identifies read-write operation and forwards the sql, is complex in overall development, develops a Proxy at the server, can only support read-write separation of mysql at the server, and cannot realize Proxy work of non-mysql, such as data of ClickHouse database; the method for developing codes on the business layer needs to reform the business layer again, develop the butt joint work of different data sources, and seriously invade the business.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data source read-write separation method and apparatus, an electronic device, and a computer readable medium, which can implement data source read-write separation in a non-intrusive manner, have small intrusion on service codes, support switching of data sources of different types, and reduce development cost.
To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a data source read-write separation method, including:
acquiring an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, taking one of the first data source and the second data source as a read data source, and taking the other one as a write data source;
intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
Optionally, the method further comprises: creating an agent manager; determining an identification of each of the first data sources and each of the second data sources, the identification including a read identification and a write identification; recording the identification in the agent manager.
Optionally, when it is determined that the operation type corresponding to the request is a write operation, linking the request to the write data source for performing the write operation includes:
if the operation type corresponding to the request is determined to be write operation, searching a write identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform write operation;
when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation comprises: and if the operation type corresponding to the request is determined to be read operation, searching a read identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the read identifier to perform the read operation.
Optionally, the obtaining an original data source of a service system, taking the original data source as a first data source, and copying the first data source to obtain a second data source includes:
acquiring an example of an original data source of the service system to acquire the original data source of the service system;
when the number of the instances is multiple, determining that the business system comprises a plurality of original data sources, and taking each original data source in the plurality of original data sources as a first data source;
and respectively copying the first data sources to obtain a plurality of corresponding second data sources.
Optionally, before identifying the operation type corresponding to the request, the method further includes:
determining a target entity class of the request;
determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class;
when the operation type corresponding to the request is determined to be write operation, the step of linking the request to the write data source for write operation comprises the following steps:
under the condition that the operation type corresponding to the request is determined to be write operation, if the first target data source is a write data source, linking the request to the first target data source for write operation, and if the first target data source is a read data source, linking the request to a second target data source corresponding to the first target data source for write operation;
when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation comprises:
and under the condition that the operation type corresponding to the request is determined to be read operation, if the first target data source is a read data source, linking the request to the first target data source for reading operation, and if the first target data source is a write data source, linking the request to a second target data source corresponding to the first target data source for reading operation.
Optionally, identifying an operation type corresponding to the request includes: and acquiring the annotation of the request, and identifying the operation type corresponding to the annotation.
Optionally, the annotation comprises one or more of: mybatis annotations, JPA annotations and custom annotations.
Optionally, identifying an operation type corresponding to the request includes: and identifying the operation type corresponding to the request through an abstract syntax tree.
To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a data source read-write separation apparatus, including:
the data source initialization module is used for determining a data source of a business system, copying the first data source to obtain a second data source by taking the data source as a first data source, and taking one of the first data source and the second data source as a read data source and the other as a write data source;
the intercepting module is used for intercepting a request for accessing the service system and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
the link module is used for linking the request to the write data source to perform write operation when the operation type corresponding to the request is determined to be write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
Optionally, the data source initialization module is further configured to: creating an agent manager; determining an identification of each of the first data sources and each of the second data sources, the identification including a read identification and a write identification; recording the identification in the agent manager.
Optionally, the link module is further configured to: if the operation type corresponding to the request is determined to be write operation, searching a write identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the read identifier to perform write operation; and if the operation type corresponding to the request is determined to be read operation, searching a read identifier from the proxy manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform the read operation.
Optionally, the data source initialization module is further configured to: acquiring an example of an original data source of the service system to acquire the original data source of the service system; when the number of the instances is multiple, determining that the business system comprises a plurality of original data sources, and taking each original data source in the plurality of original data sources as a first data source; and respectively copying the first data sources to obtain a plurality of corresponding second data sources.
Optionally, the identification module is further configured to: determining a target entity class of the request; determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class;
the linking module is further configured to: under the condition that the operation type corresponding to the request is determined to be write operation, if the first target data source is a write data source, linking the request to the first target data source for write operation, and if the first target data source is a read data source, linking the request to a second target data source corresponding to the first target data source for write operation; and under the condition that the operation type corresponding to the request is determined to be read operation, if the first target data source is a read data source, linking the request to the first target data source for reading operation, and if the first target data source is a write data source, linking the request to a second target data source corresponding to the first target data source for reading operation.
Optionally, the identification module is further configured to: and acquiring the annotation of the request, and identifying the operation type corresponding to the annotation.
Optionally, the annotation comprises one or more of: mybatis annotations, JPA annotations and custom annotations.
Optionally, the identification module is further configured to: and identifying the operation type corresponding to the request through an abstract syntax tree.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided an electronic apparatus including: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors realize the data source read-write separation method of the embodiment of the invention.
To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, the program implementing the data source read-write separation method of the embodiments of the present invention when executed by a processor.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps that an original data source of a service system is obtained, the original data source is used as a first data source, the first data source is copied to obtain a second data source, one of the first data source and the second data source is used as a read data source, and the other one of the first data source and the second data source is used as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; when the operation type corresponding to the request is determined to be read operation, the request is linked to the read data source for read operation, so that read-write separation of the data source can be realized in a non-invasive manner, intrusion to service codes is small, switching of different types of data sources is supported, and development cost is reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a data source read-write separation method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a main flow of a data source read-write separation method according to an embodiment of the present invention;
FIG. 3 is a flow chart of a data source read-write separation method according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of the main blocks of a data source read-write separation apparatus according to an embodiment of the present invention;
FIG. 5 is a system diagram of a data source read-write separation apparatus according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 7 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a data source read-write separation method according to an embodiment of the present invention, as shown in fig. 1, the method includes:
step S101: acquiring an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, taking one of the first data source and the second data source as a read data source, and taking the other one as a write data source;
step S102: intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
step S103: when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
In this embodiment, all information for establishing database connection is stored in the data source (data source), and a path for connecting to the database is defined. By means of the data source, the corresponding database connection can be determined.
For step S101, a raw data source (datasource) of the business system may be obtained by the following process: the Java instance (i.e. bean) of the original data source of the service system is obtained in the context of the starting of the Java Spring framework. And obtaining the original data source by obtaining the Java instance of the original data source. If only one Java instance exists, the service system has only one original data source, and if a plurality of Java instances exist, the service system has a plurality of original data sources. After the original data source is obtained, the original data source is used as a first data source, and the first data source is copied to obtain a second data source. When the first data source is multiple, the first data source is respectively copied to obtain multiple second data sources. Specifically, the process of copying the first data source to obtain the second data source includes: and acquiring configuration information of the first data source, copying the configuration information, replacing the copied configuration information with a preset configuration file, and generating two Java instances.
And after the second data source is obtained by copying, creating an agent manager, wherein the agent manager is used for managing the first data source and the second data source, configuring unique identifiers for the first data source and the second data source, and recording the corresponding relation between the identifiers and the first data source and the second data source, wherein the identifiers comprise read identifiers and write identifiers. And if the first data source is a read data source, determining that the identifier of the first data source is a read identifier, and if the second data source is a write data source, determining that the identifier of the second data source is a write identifier. As an example, the correspondence identifying the first data source and the second data source may be recorded in the form of a key-value.
In an alternative embodiment, one of the first data source and the second data source may be a read data source and the other may be a write data source. The read data source is used for providing a read data service, and the write data source is used for providing a write data service. Preferably, in this embodiment, the first data source is used as a write data source, the second data source is used as a read data source, the identifier of the first data source is a write identifier, and the identifier of the second data source is a read identifier. Therefore, when the first data source is copied (i.e. the configuration information of the first data source is copied), the copied configuration information is replaced by a preset read service configuration file, and the read service configuration file is used as the configuration information of the second data source, so that the second data source serves as a read data source to provide read service.
For step S102, the request to access the business system may be intercepted by the AOP facet. Among them, AOP (Aspect organized Programming) is a technology for implementing unified maintenance of program functions by means of a pre-compilation mode and a dynamic proxy during operation. The type of operation to which the request corresponds may be determined by identifying the annotation of the request. Annotation (annotation) is a mechanism of Spring, which can be obtained by reflection analysis. The Spring framework is an open source Java platform that provides a comprehensive infrastructure for easy and fast development of durable Java applications. In this embodiment, the annotation is used to distinguish between read and write services. By way of example, the annotations may include Mybatis annotations, JPA annotations, and custom annotations. Mybatis is a persistent layer framework that supports customized SQL, stored procedures, and advanced mapping, which may use annotations to configure and map native information. JPA (Java Persistence API) is a JDK 5.0 annotation or XML that describes the object-to-relationship table mappings and persists run-time entity objects into the database. In addition to the annotations already available in Mybatis and JPA, the present embodiment also supports custom annotations, i.e. annotations can be flexibly set according to application requirements.
For step S103, if it is identified that the operation type corresponding to the request is a read operation, the request is linked to a corresponding read data source for a read operation, and if it is identified that the operation type corresponding to the request is a write operation, the request is linked to a corresponding write data source for a write operation. If the operation type corresponding to the request is determined to be read operation, searching a read identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the read identifier to perform read operation; and if the operation type corresponding to the request is determined to be write operation, searching a write identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform write operation.
In an optional embodiment, if the first data source is multiple and the second data source is multiple, the data source corresponding to the request may be determined from the multiple first data sources or the multiple second data sources according to a preset object relationship mapping table and a target entity class in the request, which may specifically include:
determining a target entity class of the request;
determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class;
under the condition that the operation type corresponding to the request is determined to be write operation, if the first target data source is a write data source, linking the request to the first target data source for write operation, and if the first target data source is a read data source, linking the request to a second target data source corresponding to the first target data source for write operation;
and under the condition that the operation type corresponding to the request is determined to be read operation, if the first target data source is a read data source, linking the request to the first target data source for reading operation, and if the first target data source is a write data source, linking the request to a second target data source corresponding to the first target data source for reading operation.
The object relation mapping table refers to an ORM mapping table. ORM (object Relational mapping) is the automatic persistence of objects in an object oriented language program into a Relational database using metadata that describes the mapping between the objects and the database. The ORM mapping table realizes the data source access corresponding to the binding of the entity class, when a program accesses a related repositor method of the ORM entity class, a corresponding data source object can be found, and meanwhile, different database connection pools maintain data source resources, so that the binding of the entity class and the data source is realized. The system can acquire the bound data source of the current thread through a Spring's DataSourceUtils tool class, and further switch the data source according to the read-write decision function, so as to realize read-write separation. The repeatability is between the business layer and the data layer, separates the business layer and the data layer, and encapsulates the logic of data query and storage inside the repeatability. As an example, assume that the service system has 3 original data sources, which are an original data source a, an original data source B, and an original data source C, and the 3 original data sources are used as first data sources, and the 3 original data sources are respectively copied to obtain a second data source a ', a second data source B ', and a third data source C '. Taking a first data source A, a first data source B and a first data source C as writing data sources, taking a second data source A ', a second data source B' and a third data source C 'as reading data sources, setting the identifications of the first data source A, the first data source B and the first data source C as writing identifications, and setting the identifications of the second data source A', the second data source B 'and the third data source C' as reading identifications. The object mapping table (i.e. ORM mapping table) records the mapping relationships of the original data source a, the original data source B, the original data source C and the corresponding entity classes. After receiving a request for accessing a service system, analyzing the request to determine a target entity class of the request, and determining an original data source corresponding to the request according to an object relationship mapping table, wherein if the original data source corresponding to the request is a first data source A, the first data source A is a first target data source. Obtaining the annotation of the request, identifying the annotation to determine the operation type corresponding to the request, linking the request to the first data source A if the operation type corresponding to the request is write operation and the identifier of the first data source A is write identifier, and linking the request to the second data source A 'if the operation type corresponding to the request is read operation and the identifier of the second data source A' is read identifier.
The data source read-write separation method of the embodiment of the invention comprises the steps of obtaining an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, and taking one of the first data source and the second data source as a read data source and the other as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; when the operation type corresponding to the request is determined to be read operation, the request is linked to the read data source for read operation, so that read-write separation of the data source can be realized in a non-invasive manner, intrusion to service codes is small, switching of different types of data sources is supported, and development cost is reduced.
Fig. 2 is a schematic flow chart of a data source read-write separation method according to another embodiment of the present invention, as shown in fig. 2, wherein the data source merge corresponds to step S101 in fig. 1. Thread isolation means that the embodiment of the present invention implements separate management of multiple data sources through an agent manager, and each data source is an independent thread pool. In this embodiment, a request for accessing the service system is linked to a read data source or a write data source corresponding to the operation type of the request, that is, the request is switched to an isolated thread pool for operation. The thread pool in this embodiment is a current data source thread pool scheme such as Druid and Hikari, where Druid and Hikari are both a connection pool technology. The read data source and the write data source of this embodiment are connected to different thread pools. The tangent plane rule identification corresponds to step S102 in fig. 1 for identifying the operation type of the request to access the business system. Dynamic routing corresponds to step S103 in fig. 1 for dynamically linking the request to a read data source for a read operation or to a write data source for a write operation, depending on the type of operation.
The data source read-write separation method of the embodiment of the invention can realize the read-write separation of the data source in a non-invasive way, has small invasion to service codes, supports the switching of different types of data sources and reduces the development cost.
Fig. 3 is a schematic diagram of a main flow of a data source read-write separation method according to another embodiment of the present invention, as shown in fig. 3, the method includes:
step S301: acquiring an example of an original data source of a business system to acquire the original data source of the business system, copying the first data source to obtain a second data source by taking the original data source as a first data source, taking the first data source as a write data source, and taking the second data source as a read data source; when the number of the instances is multiple, determining that the service system comprises multiple original data sources, taking each data source of the multiple original data sources as a first data source, and respectively copying the first data source to obtain multiple corresponding second data sources; when the number of instances is one, it is determined that the business system includes one original data source.
Step S302: an agent manager is created for managing the plurality of first data sources and the second data source.
Step S303: and determining the identification of each first data source and each second data source, and recording the corresponding relation between the identification and the first data source or the second data source in the agent manager, wherein the identification of each first data source is a write identification, and the identification of each second data source is a read identification.
Step S304: and intercepting a request for accessing the service system through the AOP section.
Step S305: determining a target entity class for the request.
Step S306: and determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class.
Step S307: identifying an operation type corresponding to the request, wherein the operation type comprises a read operation and a write operation;
step S308: if the identified operation type is write operation, linking the request to a first target data source; and if the identified operation type is a read operation, linking the request to a second target data source corresponding to the first target data source.
The data source read-write separation method of the embodiment of the invention comprises the steps of obtaining an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, and taking one of the first data source and the second data source as a read data source and the other as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; based on the operation type, the technical means of linking the request to the read data source for reading operation or linking the request to the write data source for writing operation can realize the read-write separation of the data source in a non-invasive way, has small invasion to service codes, supports the switching of different types of data sources and reduces the development cost.
Fig. 4 is a schematic diagram of main blocks of a data source read-write separation apparatus 400 according to an embodiment of the present invention, and as shown in fig. 4, the data source read-write separation apparatus 400 includes:
a data source initialization module 401, configured to determine a data source of a service system, copy the data source as a first data source to obtain a second data source, and use one of the first data source and the second data source as a read data source and the other as a write data source;
an intercepting module 402, configured to intercept a request for accessing the service system, and identify an operation type corresponding to the request, where the operation type includes a read operation and a write operation;
a linking module 403, configured to link the request to the write data source for write operation when it is determined that the operation type corresponding to the request is write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
Optionally, the data source initialization module 401 is further configured to: creating an agent manager; determining an identification of each of the first data sources and each of the second data sources, the identification including a read identification and a write identification; recording the identification in the agent manager.
Optionally, the link module 403 is further configured to: if the operation type corresponding to the request is determined to be write operation, searching a write identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform write operation; and if the operation type corresponding to the request is determined to be read operation, searching a read identifier from the proxy manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform the read operation.
Optionally, the data source initialization module 401 is further configured to: acquiring an example of an original data source of the service system; when the number of the instances is multiple, determining that the business system comprises a plurality of original data sources, and taking each data source in the plurality of original data sources as a first data source; and respectively copying the first data sources to obtain a plurality of corresponding second data sources.
Optionally, the identifying module 402 is further configured to: determining a target entity class of the request; determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class;
the linking module 403 is further configured to: under the condition that the operation type corresponding to the request is determined to be write operation, if the first target data source is a write data source, linking the request to the first target data source for write operation, and if the first target data source is a read data source, linking the request to a second target data source corresponding to the first target data source for write operation; and under the condition that the operation type corresponding to the request is determined to be read operation, if the first target data source is a read data source, linking the request to the first target data source for reading operation, and if the first target data source is a write data source, linking the request to a second target data source corresponding to the first target data source for reading operation.
Optionally, the identifying module 402 is further configured to: and acquiring the annotation of the request, and identifying the operation type corresponding to the annotation.
Optionally, the annotation comprises one or more of: mybatis annotations, JPA annotations and custom annotations.
Optionally, the identifying module 402 is further configured to: and identifying the operation type corresponding to the request through an abstract syntax tree.
The data source read-write separation device and the data source read-write separation method provided by the embodiment of the invention obtain an original data source of a service system, copy the original data source as a first data source to obtain a second data source, and use one of the first data source and the second data source as a read data source and the other as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; when the operation type corresponding to the request is determined to be read operation, the request is linked to the read data source for read operation, so that read-write separation of the data source can be realized in a non-invasive manner, intrusion to service codes is small, switching of different types of data sources is supported, and development cost is reduced.
The device can execute the method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
Fig. 5 is a schematic system diagram of a data source read-write separation device to which an embodiment of the present invention is applied, and as shown in fig. 5, the data source read-write separation device intercepts an external read-write access request, identifies an operation type of the access request, links the access request to a corresponding read data source if the operation type of the access request is a read operation, and links the access request to a corresponding write data source if the operation type of the access request is a write operation, thereby implementing read-write separation.
Fig. 6 shows an exemplary system architecture 600 of a data source read-write separation method or a data source read-write separation apparatus to which an embodiment of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, a server 605, and a database 606. The network 604 serves to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. Network 604 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages or the like. The terminal devices 601, 602, 603 may have various communication applications installed thereon, and send requests to access the server through the communication applications.
The server 605 may be a server that provides various services, such as a server that provides data services for users. Server 605 may link the access request to a read data source or a write data source according to the received access request, and perform data read-write service on database 606 through the linked read data source or write data source.
It should be noted that the data source read-write separation method provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the data source read-write separation apparatus is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks, and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, a block diagram of a computer system 700 suitable for use with a server device implementing an embodiment of the invention is shown. The terminal device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a sending module, an obtaining module, a determining module, and a first processing module. The names of these modules do not in some cases constitute a limitation on the unit itself, and for example, the sending module may also be described as a "module that sends a picture acquisition request to a connected server".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
determining an original data source of a business system to acquire the original data source of the business system, taking the original data source as a first data source, copying the first data source to obtain a second data source, and taking one of the first data source and the second data source as a read data source and the other as a write data source;
intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
According to the technical scheme of the embodiment of the invention, an original data source of a service system is obtained, the original data source is used as a first data source, the first data source is copied to obtain a second data source, one of the first data source and the second data source is used as a read data source, and the other one of the first data source and the second data source is used as a write data source; intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation; when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; when the operation type corresponding to the request is determined to be read operation, the request is linked to the read data source for read operation, so that read-write separation of the data source can be realized in a non-invasive manner, intrusion to service codes is small, switching of different types of data sources is supported, and development cost is reduced.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (11)

1. A data source read-write separation method is characterized by comprising the following steps:
acquiring an original data source of a service system, taking the original data source as a first data source, copying the first data source to obtain a second data source, taking one of the first data source and the second data source as a read data source, and taking the other one as a write data source;
intercepting a request for accessing the service system, and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
when the operation type corresponding to the request is determined to be write operation, linking the request to the write data source for write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
2. The method of claim 1, further comprising:
creating an agent manager;
determining identifications of the first data source and the second data source, wherein the identifications comprise read identifications and write identifications;
recording the identification in the agent manager.
3. The method of claim 2, wherein when it is determined that the operation type corresponding to the request is a write operation, linking the request to the write data source for the write operation comprises:
if the operation type corresponding to the request is determined to be write operation, searching a write identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the write identifier to perform write operation;
when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation comprises:
and if the operation type corresponding to the request is determined to be read operation, searching a read identifier from the agent manager, and linking the request to a first data source or a second data source corresponding to the read identifier to perform the read operation.
4. The method of claim 3, wherein obtaining an original data source of a business system, using the original data source as a first data source, and copying the first data source to obtain a second data source comprises:
acquiring an example of an original data source of the service system to acquire the original data source of the service system;
when the number of the instances is multiple, determining that the business system comprises a plurality of original data sources, and taking each original data source in the plurality of original data sources as a first data source;
and respectively copying the first data sources to obtain a plurality of corresponding second data sources.
5. The method of claim 4, wherein prior to identifying the type of operation to which the request corresponds, the method further comprises:
determining a target entity class of the request;
determining a first target data source corresponding to the request from the plurality of first data sources according to a preset object relationship mapping table and the target entity class, wherein the object relationship mapping table is used for recording the mapping relationship between an original data source of the service system and the corresponding entity class;
when the operation type corresponding to the request is determined to be write operation, the step of linking the request to the write data source for write operation comprises the following steps:
under the condition that the operation type corresponding to the request is determined to be write operation, if the first target data source is a write data source, linking the request to the first target data source for write operation, and if the first target data source is a read data source, linking the request to a second target data source corresponding to the first target data source for write operation;
when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation comprises:
and under the condition that the operation type corresponding to the request is determined to be read operation, if the first target data source is a read data source, linking the request to the first target data source for reading operation, and if the first target data source is a write data source, linking the request to a second target data source corresponding to the first target data source for reading operation.
6. The method of claim 1, wherein identifying the type of operation to which the request corresponds comprises: and acquiring the annotation of the request, and identifying the operation type corresponding to the annotation.
7. The method of claim 1, wherein the annotations include one or more of: mybatis annotations, JPA annotations and custom annotations.
8. The method of claim 1, wherein identifying the type of operation to which the request corresponds comprises:
and identifying the operation type corresponding to the request through an abstract syntax tree.
9. A data source read-write separation apparatus, comprising:
the data source initialization module is used for determining a data source of a business system, copying the first data source to obtain a second data source by taking the data source as a first data source, and taking one of the first data source and the second data source as a read data source and the other as a write data source;
the identification module is used for intercepting a request for accessing the service system and identifying an operation type corresponding to the request, wherein the operation type comprises read operation and write operation;
the link module is used for linking the request to the write data source to perform write operation when the operation type corresponding to the request is determined to be write operation; and when the operation type corresponding to the request is determined to be read operation, linking the request to the read data source for read operation.
10. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-8.
11. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-8.
CN202011050571.8A 2020-09-29 2020-09-29 Data source read-write separation method and device Pending CN113761038A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011050571.8A CN113761038A (en) 2020-09-29 2020-09-29 Data source read-write separation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011050571.8A CN113761038A (en) 2020-09-29 2020-09-29 Data source read-write separation method and device

Publications (1)

Publication Number Publication Date
CN113761038A true CN113761038A (en) 2021-12-07

Family

ID=78785925

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011050571.8A Pending CN113761038A (en) 2020-09-29 2020-09-29 Data source read-write separation method and device

Country Status (1)

Country Link
CN (1) CN113761038A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030212660A1 (en) * 2002-05-10 2003-11-13 Kerwin Douglas W. Database scattering system
CN102541923A (en) * 2010-12-30 2012-07-04 北京新媒传信科技有限公司 Database read-write separating method and device
CN102622427A (en) * 2012-02-27 2012-08-01 杭州闪亮科技有限公司 Method and system for read-write splitting database
CN103793432A (en) * 2012-11-01 2014-05-14 深圳中兴网信科技有限公司 Method and device for splitting database reading and writing
US20150278329A1 (en) * 2014-03-26 2015-10-01 International Business Machines Corporation Replication of a relational database
CN107301179A (en) * 2016-04-14 2017-10-27 北京京东尚科信息技术有限公司 The method and apparatus of data base read-write separation
CN110837531A (en) * 2019-10-12 2020-02-25 中国平安财产保险股份有限公司 Data source read-write separation method and device and computer readable storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030212660A1 (en) * 2002-05-10 2003-11-13 Kerwin Douglas W. Database scattering system
CN102541923A (en) * 2010-12-30 2012-07-04 北京新媒传信科技有限公司 Database read-write separating method and device
CN102622427A (en) * 2012-02-27 2012-08-01 杭州闪亮科技有限公司 Method and system for read-write splitting database
CN103793432A (en) * 2012-11-01 2014-05-14 深圳中兴网信科技有限公司 Method and device for splitting database reading and writing
US20150278329A1 (en) * 2014-03-26 2015-10-01 International Business Machines Corporation Replication of a relational database
CN107301179A (en) * 2016-04-14 2017-10-27 北京京东尚科信息技术有限公司 The method and apparatus of data base read-write separation
CN110837531A (en) * 2019-10-12 2020-02-25 中国平安财产保险股份有限公司 Data source read-write separation method and device and computer readable storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Y.J. CHEN.ET AL: "Recording performance of discrete track patterned media fabricated by focused ion beam", IEEE TRANSACTIONS ON MAGNETICS, 6 June 2005 (2005-06-06) *
饶兰兰;肖杰伟;: "一种异构数据读写分离技术的架构理论和实践", 电脑编程技巧与维护, no. 24, 18 December 2014 (2014-12-18) *

Similar Documents

Publication Publication Date Title
CN109062563B (en) Method and device for generating page
CN111949693B (en) Data processing device, data processing method, storage medium and electronic equipment
US10353723B2 (en) PL/SQL language parsing at a virtual machine
US20200233786A1 (en) Default mock implementations at a server
US10379914B2 (en) System and method for achieving specific behaviors by intercepting file access calls in a mainframe rehosting platform
CN110688096B (en) Method and device for constructing application program containing plug-in, medium and electronic equipment
CN110765187A (en) Data source route management method and device
CN111666293A (en) Database access method and device
US10585678B2 (en) Insertion of custom activities in an orchestrated application suite
US20230259358A1 (en) Documentation enforcement during compilation
CN112988583A (en) Method and device for testing syntax compatibility of database
WO2021012554A1 (en) Method and apparatus for updating data field in block chain, medium, and electronic device
CN113778725A (en) Data verification method and device
CN112860538A (en) Method and device for performing interface regression test based on online log
CN113761038A (en) Data source read-write separation method and device
CN115543967A (en) Data migration method and device, electronic equipment and computer readable medium
CN115640279A (en) Method and device for constructing data blood relationship
CN112835606B (en) Gray release method and device, electronic equipment and medium
CN114489674A (en) Data verification method and device of dynamic data model
CN114780361A (en) Log generation method, device, computer system and readable storage medium
CN108089852B (en) Method, medium, apparatus and computing device for establishing attribute access
US11888937B2 (en) Domain specific provider contracts for core data services
US11943115B2 (en) Locally debugging remote deployment of microservices
US20240126541A1 (en) Inversion process for rules framework
US20230195901A1 (en) Model-based upgrade recommendations using software dependencies

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination