CN107016016B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN107016016B CN107016016B CN201610882919.7A CN201610882919A CN107016016B CN 107016016 B CN107016016 B CN 107016016B CN 201610882919 A CN201610882919 A CN 201610882919A CN 107016016 B CN107016016 B CN 107016016B
- Authority
- CN
- China
- Prior art keywords
- sql statement
- database
- relational database
- service request
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Abstract
The application discloses a data processing method and a device, and a first conversion rule and a second conversion rule are configured in advance, so that after a service request is received, a first SQL statement for operating a first database and a second SQL statement for operating a second database can be generated through the service request, the first conversion rule and the second conversion rule in sequence, and the first SQL statement and the second SQL statement can be executed respectively to realize the operation on the second database. Therefore, by the method provided by the application, the first database and the second database can be operated according to the service request, so that the data in the second database is actually obtained according to the service request of the application, and further the data in the first database does not need to be backed up to the second database, thereby avoiding the subsequent process of verifying the data in the second database, and reducing the cost of database replacement.
Description
Technical Field
The present application relates to the field of information technologies, and in particular, to a method and an apparatus for data processing.
Background
With the development of information technology, the demand of applications to execute services through networks is increasing, and the service data generated by executing services is also increasing. These traffic data are typically stored using a database. When storing service data, in order to ensure the consistency of data storage and prevent dirty data, a relational database is usually used to store each service data.
Generally, since the structure of the relational database used by different applications is not completely consistent, the application server corresponding to the application needs to configure the relational data (referred to as the original database herein) matching the requirements or performance of the application for one application. However, over time, the original database may no longer be adapted to the needs or capabilities of the application, requiring replacement of the original database with another database having superior capabilities (referred to herein as a replacement database).
In the prior art, replacing the original database is usually: and backing up the service data generated by the original database within a period of time after a preset time point to the replacement database in an asynchronous double-writing data processing mode (namely writing the service data to the original relational database firstly and then writing the service data to the replacement database), thereby realizing the data migration from the data in the original database to the replacement database.
However, since the "asynchronous double-write" process is only performed at the database level and has no special relation with the application program and the application server, the replacement database after data migration may not support the normal operation of the application program. In order to prevent this situation, before the application or the application server uses the data in the replacement database, the accuracy of the data in the replacement database must be verified, but various current verification methods are not universal, and special designs of verification methods, verification tools, verification environments and the like are required, so that the verification process is quite complicated and time-consuming, and the replacement cost of the relational database is greatly increased.
Disclosure of Invention
The embodiment of the application provides a data processing method, which is used for solving the problems that in the prior art, after a database is replaced for data processing, the accuracy of data in the database needs to be verified, the verification time is long, more resources are consumed, and the replacement cost is high.
The embodiment of the application provides a data processing device, which is used for solving the problems that in the prior art, after a database is replaced to perform data processing, the accuracy of data in the database needs to be verified, the verification time is long, more resources are consumed, and the replacement cost is high.
The embodiment of the application adopts the following technical scheme:
a method of data processing, comprising:
receiving a service request for a relational database;
generating a first SQL statement corresponding to the service request according to the service request and a first conversion rule configured in advance, and generating a second SQL statement according to the service request and a second conversion rule configured in advance;
and executing corresponding operation on the first relational database through the first SQL statement, and executing corresponding operation on the second relational database through the second SQL statement.
A data processing apparatus comprising:
the receiving module is used for receiving a service request for the relational database;
the generating module generates a first SQL statement corresponding to the service request according to the service request and a first conversion rule configured in advance, and generates a second SQL statement according to the service request and a second conversion rule configured in advance;
and the execution module executes corresponding operation on the first relational database through the first SQL statement and executes corresponding operation on the second relational database through the second SQL statement.
The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:
because the first conversion rule and the second conversion rule are configured in advance, after a service request is received, a first SQL statement for operating a first database can be generated through the service request and the first conversion rule, then a second SQL statement for operating a second database can be generated according to the second conversion rule and the service request, and corresponding operations are executed through the first SQL statement and the second SQL statement respectively, wherein the first database is an original database, and the second database is a replacement database. Therefore, by the method provided by the application, the first database and the second database can be operated according to the service request, so that the data in the second database is actually generated according to the service request for executing the application, further, the data in the first database does not need to be backed up to the second database at a database level, the subsequent process of verifying the data in the second database is avoided, and the cost for replacing the database is reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a block diagram illustrating a data processing process according to an embodiment of the present disclosure;
fig. 2 is a schematic structural relationship diagram of an application server, the first database, and the second database according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a detailed structural relationship among the application server, the first database, and the second database according to an embodiment of the present disclosure.
Fig. 4 is a schematic diagram illustrating a process of executing the first SQL statement and the second SQL statement in the first database and the second database, respectively, according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The data processing process provided in the embodiment of the present application is a process of "migrating", to a second database, business data generated when a business request is executed in a period of time after a predetermined time point by a first database when the first database used by an application needs to be replaced by the second database. The first database is an original database used by the application program, and the second database is a replacement database replacing the original database.
In general, the predetermined time point may be a time point determined by a worker, and a time duration corresponding to a period of time after the predetermined time point may also be determined by the worker, that is, the end time of the "migration" process may also be determined by the worker. Moreover, since the predetermined time point and the duration of the period of time have no relation with the data processing process of the present application, the present application is not limited to this specifically. That is, the data processing process described in this application is a data processing method when the application executes the service request after the predetermined time point until the end of the period of time, and does not refer to when the data processing method starts and when the data processing method ends.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a data processing process provided in an embodiment of the present application, which specifically includes the following steps:
s101: a service request is received for a relational database.
In the prior art, during the execution of a service, an application server may generally receive a service request corresponding to the service, and the application server determines the operation of the subsequent step according to the service request.
Similarly, in the embodiment of the present application, the application server may also receive the service request, where the service request may also be a service request for a relational database because the application corresponds to a process of how to perform data processing on data in the database. Since the data processing process is a process of "migrating" data in the first database to the second database, the application server may correspond to the first database and the second database, the first database is an original database used by the application server, the second database is a replacement database that is expected to be used by the application server to replace the first database after the "migration" process is completed, and both the first database and the second database may be relational databases, as shown in fig. 2.
Fig. 2 is a schematic structural relationship diagram of an application server, the first database, and the second database according to an embodiment of the present application. It can be seen that the application server corresponds to the first database and the second database.
The application server may be an individual device, or may be a system (i.e., a distributed application server) composed of a plurality of devices, and the first database and the second database may be an individual device, or may be a system (i.e., a distributed database) composed of a plurality of devices, which is not limited in this application.
In addition, since the application program usually executes an Object-oriented Language (such as java Language), the service request received by the application server is usually generated by the Object-oriented Language, but if the service request relates to an add, Query, modify, Delete (Create, Retrieve, Update, Delete, CRUD) operation on data in a database corresponding to the application server, because the Language for operating the relational database is Structured Query Language (SQL), which is not consistent with the Object-oriented Language, in this embodiment of the application, when the application server receives the service request for the relational database, the application server may further convert the service request into an SQL statement capable of operating the relational database through a pre-configured Object Relation Mapping (ORM) framework so as to subsequently operate the relational database, the application server may operate the relational database using the ORM framework, which may be ibatis specifically, using data layer middleware.
Fig. 2 may continue to be refined to fig. 3. Fig. 3 is a schematic diagram illustrating a detailed structural relationship among the application server, the first database, and the second database according to an embodiment of the present disclosure. It can be seen that the application server generates SQL statements through the ORM framework, and operates the first and second databases using data layer middleware.
And, since the application server corresponds to the first database and the second database, and the database types and the database sizes of the first and second databases may be different, a first conversion rule for converting the service request into an SQL statement operable on the first database and a second conversion rule for converting the service request into an SQL statement operable on the second database may be configured in advance in the application server, where the first conversion rule is also generally referred to as a sqmap and the second conversion rule is referred to as a second sqmap in the following description because the conversion rule is also generally referred to as a sqmap.
For example, assuming that an application server a corresponds to a first database B and a second database C, where the first database B and the second database C are both relational databases, the application program corresponding to the application server a uses a Java language, further assuming that the first database B is mysql and the second database C is oracle, the application server a is also preconfigured with a first slmap that converts the Java language into a first SQL statement corresponding to the mysql database, so as to convert the Java language into a second slmap that corresponds to the second SQL statement corresponding to the oracle database, and the first slmap and the second slmap are both configured in an ORM framework of the application server, and the application server a receives a service request D for the relational database.
S102: and generating a first SQL statement corresponding to the service request according to the service request and a first conversion rule configured in advance, and generating a second SQL statement according to the service request and a second conversion rule configured in advance.
In the embodiment of the present application, after the application server receives a service request for a relational database, an idle thread may be determined, and the service request may be executed through the thread, where the thread may first convert the service request into a first SQL statement for executing a corresponding operation on the first database, and then convert the first SQL statement into a second SQL statement for executing a corresponding operation on the second database.
Specifically, the application server may determine an idle thread, and enable the thread to generate the first SQL statement according to the service request through the first sqmap preconfigured in the ORM framework of the application server, and because the ORM framework is also preconfigured with the second sqmap, the thread may also generate the second SQL statement according to the service request, so that the application server may execute corresponding operations on the first database and the second data through the first SQL statement and the second SQL statement, respectively.
In addition, in the embodiment of the present application, since the second database is used as a replacement database, the amount of data stored is smaller than the amount of data stored in the first database, and the probability of supporting the normal operation of the application server is lower, if any service request to the relational database generates the first SQL statement and the second SQL statement respectively, the probability of the execution failure of the second SQL statement may be increased. In addition, in the data processing process described in the present application, it is only necessary to make the data change in the first database caused by the service request appear in the second server as well, that is, it is only necessary to ensure that after the preset time point, the value of the data in the second database is consistent with the value of the data after the data change occurs in the first database, so in this embodiment of the present application, the data processing process may only be directed to the operations of adding, deleting, and changing the data, and does not need to perform the data processing process described in the present application on the query operation (the query operation does not change the value of the data stored in the first database).
Since the operator can determine what kind of service request of the application program needs to add, delete, and modify the data in the first database, in the application, a service identifier list determined by the operator may be stored in the application server in advance, where each service identifier in the service identifier list is a service identifier of a service request that needs to add, delete, and modify the first database, and the application server may also determine whether to generate the second SQL statement according to the service identifier corresponding to the service request.
Specifically, in step S101, the service request received by the application server may carry a service identifier corresponding to the service request, so that, first, when the application server calls an idle thread according to the service request, the service identifier may be written into a thread variable of the thread. Secondly, after the thread generates the first SQL statement according to the first sqmap, the data layer middleware may execute the first SQL statement to operate on the first database, and then the data layer middleware may determine the service identifier through a thread variable of the thread. Then, the data layer middleware may determine, according to the pre-stored service identifier list, whether the service identifier is the same as any service identifier in the service identifier list, if so, determine that the second SQL statement needs to be generated, the thread may also generate the second SQL statement through the second sqmap, subsequently execute the first and second SQL statements, if not, determine that the second SQL statement does not need to be generated, and the thread may continue to execute subsequent operations (e.g., execute corresponding operations on the first database according to the first SQL statement).
Continuing with the above example, assume that the application server a determines that thread x that is idle is executing the service request, the application server may write the service identification y of the service request into the thread variable of the thread x, after the thread x generates the first SQL statement through the first sqmap pre-configured in the ORM framework, when the first SQL statement is executed by using the data layer middleware, the service identifier y can be determined by the thread variable of the thread x, further assuming that the service identifier y is included in the service identifier list pre-stored in the application server a, the data tier middleware, upon executing the first SQL statement, may further determine that a second SQL statement needs to be generated, the thread x may also generate a second SQL statement via the second sqmap, and the thread x may execute the first SQL statement and the second SQL statement using the data layer middleware.
S103: and executing corresponding operation on the first relational database through the first SQL statement, and executing corresponding operation on the second relational database through the second SQL statement.
In this embodiment, after the first and second SQL statements are generated, the application server may perform corresponding operations on the first database and the second database by using the data layer middleware through the thread.
Moreover, because the first database and the second database are two independent databases, corresponding transactions can be respectively established for the first SQL statement and the second SQL statement, and further, in order to ensure that the data in the second database can be consistent with the modified data in the first database, the transactions corresponding to the first SQL statement and the second SQL statement respectively can be submitted only when the first SQL statement and the second SQL statement are executed successfully in the present application.
Specifically, the thread uses the data layer middleware to firstly establish a first transaction for executing the first SQL statement in the first database, secondly establish a second transaction for executing the second SQL statement in the second database, thirdly execute an operation corresponding to the first SQL statement in the first database according to the first SQL statement, fourthly execute an operation corresponding to the second SQL statement in the second database according to the second SQL statement, and submit the first transaction and the second transaction when the first SQL statement and the second SQL statement respectively execute successfully, and if the first transaction fails to establish or the second transaction fails to establish or the first SQL statement fails to execute or the second SQL statement fails to execute, the thread returns an error message (also called a throw error) and executes a rollback operation to restore the data in the first database and the second database to the state before executing step S103, as shown in fig. 4.
Fig. 4 is a schematic diagram illustrating a process of executing the first SQL statement and the second SQL statement in the first database and the second database, respectively, according to an embodiment of the present application.
It can be seen that, in the process of executing the corresponding operation on the first database by the first SQL statement and executing the corresponding operation on the second database by the second SQL statement, as long as any step has an error or a failure, the thread will throw the error and execute the rollback operation, and when both the first transaction and the second transaction are successfully submitted, the operation corresponding to the service request is executed.
Through the steps shown in fig. 4, it can be ensured that for the operation corresponding to the service request, the change of the data in the first database and the second database is the same, and the situation that one database is executed successfully and the other database is executed unsuccessfully, resulting in inconsistent data, does not occur.
It should be noted that, as described above, the application server may be a single device or a system composed of multiple devices, so that all execution subjects of the steps of the method provided in the embodiment of the present application shown in fig. 1 may be the same device, or different devices may be used as execution subjects of the method. For example, the execution subject of step S101 and step S102 may be device 1, and the execution subject of step S103 may be device 2; for another example, the execution subject of step S101 may be device 1, and the execution subjects of step S102 and step S103 may be device 2; and so on.
By the data processing method shown in fig. 1, the first and second SQL statements of the operations executed on the first database and the second database respectively can be generated according to the business request received by the application service for the relational database, so that the business request can execute the corresponding operation in the second database through the second SQL statement, so that the data in the second database is not obtained only through the data processing procedure of "asynchronous double write" at the database level, but obtained through the business request of the application, and in the present application, only when the first SQL statement and the second SQL statement are executed successfully, the corresponding first transaction and second transaction will be submitted, so that the first database and second database generate actual data changes, so through the data processing procedure provided by the present application, not only can the actual relation be generated between the operation of the second database and the application, but also the consistency of the data in the first database and the second database (specifically, the change of the data after the preset time point is consistent) can be ensured, so that the second database can directly support the operation of the application when the duration of the period of time is over, the subsequent process of verifying the data in the second database is avoided, and the cost of database replacement is reduced.
In addition, in the present application, since the first SQL statement is already generated in the application server before the second SQL statement is generated, the second sqmap may also be a rule for generating a second SQL statement for operating the second database according to the first SQL statement, and when the thread determines that the second SQL statement needs to be generated, the thread may generate the second SQL statement according to the first SQL statement and the second sqmap.
Further, since it is not necessary for the application program to care whether the application server needs to perform corresponding operations on the first database and the second database according to the service request, the content of the service request may be only for the first database, and the application server may implement the operations of subsequent steps S102 and S103 according to the application request.
In the above description of the embodiment of the present application, the application server is used as the execution subject, that is, the application program running on the application server includes the data layer middleware, but the data layer middleware may exist as a single program or system, so that in the embodiment of the present application, the data layer middleware may execute a part of step S102 and step S103.
Specifically, when the data layer middleware is a single program, the application server may further run an application program (referred to as a surface layer application, for example, for distinction), which faces a user, where the surface layer is applied to receive an operation of the user, determine a service request, generate the first SQL statement according to the preconfigured first sqmap, and send the first SQL statement to the data layer middleware, so that the data layer middleware performs a subsequent operation.
In this embodiment of the application, the application server determines, according to a pre-stored service identifier list and the service identifier carried in the service request, that the second SQL statement that is required to be generated to operate the second relational database is generated, which may specifically be:
after receiving the service request, the surface layer application can generate the first SQL statement according to the first sqmap, write a service identifier corresponding to the service request into a thread for executing the first SQL statement, and send the service identifier to the data layer middleware. The data layer middleware may determine, by running the thread, the service identifier written in the thread variable, and determine, by using a service identifier list stored in advance, whether to generate a second SQL statement, and if necessary, generate the second SQL statement according to the second pre-configured sqmap. Then, the data layer middleware may execute all the processes of step S103, and may specifically be consistent with the processes shown in fig. 4.
Further, since the application server can be divided into the surface application and the data layer middleware, and the two programs can be independent of each other, the pre-configured first and second sqmaps can be accessible to both the surface application and the data layer middleware.
Furthermore, since the surface layer Application and the data layer middleware can be independent from each other, the surface layer Application and the data layer middleware can perform data interaction through an Application Programming Interface (API), when the surface layer Application is connected with the data layer middleware for the first time, the first and the second databases corresponding to the surface layer Application can be declared to the data layer middleware through a declaration file configured in advance in the surface layer Application by a worker, wherein the declaration file includes database information such as database addresses and database configurations respectively corresponding to the first and the second databases.
Of course, the surface layer application and the data layer middleware may also be located in the same application program, which is not particularly limited in this application, and when the data layer middleware is an independent program, the data layer middleware may also be located in one or more independent devices, and the data layer middleware may also be connected to a plurality of surface layer applications and operate a plurality of databases, respectively. Moreover, since the present application relates to a "migration" process of data in a database, only the case where the first and second databases exist has been described, but the method described in the present application is not limited to corresponding to only two databases with the application server, and the application server may also correspond to a plurality of databases as long as the application server is configured with a corresponding sqmap in advance.
In addition, since the data layer middleware can directly call the first and second SQLMAPs configured in advance, in the present application, therefore, the data layer middleware can also request services of the relational database, and generates the first and second SQL statements, and finally executes the first and second SQL statements, it should be noted that, the service request received by the data layer middleware may be different from the aforementioned service request, (for example, for distinction, the following is referred to as a data request) specifically, the service request is sent to the data layer middleware after being processed by the skin application, specifically, the surface application may determine the operations to be performed on the data based on the received service request and based on the service logic, and sending the operation as a data request to the data-layer middleware to cause the data-layer middleware to perform a subsequent operation.
It should be noted that the language used by the application program may be C language, C + + language, etc. other than Java language described in fig. 1, the ORM framework may be Entity EJB, Hibernate, TopLink, OJB, etc. other than ibatis, the types of the first database and the second database may be Oracle, DB2, Microsoft SQL Server, etc., and the application is not limited in particular to the language used by the application program, the ORM framework used by the application Server, and the types of the first and second databases.
Based on the data processing process shown in fig. 1, an embodiment of the present application further provides a data processing apparatus, as shown in fig. 5.
Fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, including:
a receiving module 201, which receives a service request to a relational database;
the generating module 202 is configured to generate a first SQL statement corresponding to the service request according to the service request and a first conversion rule configured in advance, and generate a second SQL statement according to the service request and a second conversion rule configured in advance;
the execution module 203 executes a corresponding operation on the first relational database through the first SQL statement, and executes a corresponding operation on the second relational database through the second SQL statement.
The service request carries a service identifier;
the generating module 202 determines, according to a pre-stored service identifier list and the service identifier carried in the service request, that the second SQL statement that operates on the second relational database needs to be generated.
The receiving module 201 writes the service identifier into a thread variable of a thread executing the first SQL statement;
the generating module 202, when the first SQL statement is executed by the thread, determines the service identifier according to the thread variable, and determines, according to a pre-stored service identifier list and the service identifier, the second SQL statement that needs to be generated to operate the second relational database.
The execution module 203 establishes a first transaction for executing the first SQL statement in the first relational database, establishes a second transaction for executing the second SQL statement in the second relational database, executes a corresponding operation in the first relational database according to the first SQL statement, executes a corresponding operation in the second relational database according to the second SQL statement, submits the first transaction and the second transaction when the first SQL statement and the second SQL statement are both successfully executed, and returns error information and executes a rollback operation when the first SQL statement and/or the second SQL statement are failed to be executed.
The first relational database is an original database, the second relational database is a replacement database, and the corresponding operation comprises at least one of addition, deletion and change.
Specifically, the data processing apparatus shown in fig. 4 may be located in an application server, and the application server may be one device or a system composed of multiple devices.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (11)
1. A method of data processing, comprising:
receiving a service request for a relational database;
generating a first SQL statement corresponding to the service request and used for operating a first database according to the service request and a first conversion rule configured in advance, and generating a second SQL statement used for operating a second database according to the service request and a second conversion rule configured in advance; the first database and the second database are different in database type;
and executing corresponding operation on a first relational database through the first SQL statement, and executing corresponding operation on a second relational database through the second SQL statement.
2. The method of claim 1, wherein the service request carries a service identifier;
before generating a second SQL statement according to the service request and a second conversion rule configured in advance, the method further includes:
and determining the second SQL statement which needs to generate for operating the second relational database according to a pre-stored service identifier list and the service identifier carried in the service request.
3. The method according to claim 2, wherein determining, according to a pre-stored service identifier list and the service identifier carried in the service request, that the second SQL statement that needs to be generated to operate the second relational database specifically includes:
writing the service identification into a thread variable of a thread executing the first SQL statement;
when the thread is operated, the service identifier is determined according to the thread variable;
and determining the second SQL statement needing to be generated for operating the second relational database according to a pre-stored service identification list and the service identification.
4. The method of claim 1, wherein performing the corresponding operation on the first database through the first SQL statement and performing the corresponding operation on the second database through the second SQL statement specifically include:
establishing a first transaction executing the first SQL statement in the first relational database;
establishing a second transaction executing the second SQL statement in the second relational database;
executing corresponding operation on the first relational database according to the first SQL statement through the first transaction;
executing corresponding operation on the second relational database according to the second SQL statement through the second transaction;
submitting the first transaction and the second transaction when the first SQL statement and the second SQL statement are executed successfully;
and when the first SQL statement and/or the second SQL statement fails to execute, returning error information and executing rollback operation.
5. The method of any of claims 1 to 4, wherein the first relational database is an original database, the second relational database is a replacement database, and the corresponding operation comprises at least one of an addition, a deletion, and a change.
6. The method of claim 1, wherein the method further comprises:
the data layer middleware generates a second SQL statement according to the service request and a second conversion rule configured in advance;
and the data layer middleware executes corresponding operation on the first relational database through the first SQL statement and executes corresponding operation on the second relational database through the second SQL statement.
7. An apparatus for data processing, comprising:
the receiving module is used for receiving a service request for the relational database;
the generating module generates a first SQL statement corresponding to the business request and used for operating a first database according to the business request and a first conversion rule configured in advance, and generates a second SQL statement used for operating a second database according to the business request and a second conversion rule configured in advance; the first database and the second database are different in database type;
and the execution module executes corresponding operation on the first relational database through the first SQL statement and executes corresponding operation on the second relational database through the second SQL statement.
8. The apparatus of claim 7, wherein the service request carries a service identifier;
and the generating module determines to generate the second SQL statement for operating the second relational database according to a pre-stored service identifier list and the service identifier carried in the service request.
9. The apparatus of claim 8, wherein the receiving module writes the business identifier into a thread variable of a thread executing the first SQL statement;
and the generating module is used for determining the service identifier according to the thread variable when the thread is operated, and determining the second SQL statement needing to be generated for operating the second relational database according to a pre-stored service identifier list and the service identifier.
10. The apparatus of claim 7, wherein the execution module establishes a first transaction to execute the first SQL statement in the first relational database, establishes a second transaction to execute the second SQL statement in the second relational database, executes a corresponding operation on the first relational database according to the first SQL statement by the first transaction, executes a corresponding operation on the second relational database according to the second SQL statement by the second transaction, commits the first transaction and the second transaction when the first SQL statement and the second SQL statement are both successfully executed, and returns an error message and performs a rollback operation when the first SQL statement and/or the second SQL statement are failed to be executed.
11. The apparatus of any of claims 7 to 10, wherein the first relational database is an original database, the second relational database is a replacement database, and the corresponding operation comprises at least one of an addition, a deletion, and a change.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610882919.7A CN107016016B (en) | 2016-10-10 | 2016-10-10 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610882919.7A CN107016016B (en) | 2016-10-10 | 2016-10-10 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107016016A CN107016016A (en) | 2017-08-04 |
| CN107016016B true CN107016016B (en) | 2021-04-06 |
Family
ID=59439498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610882919.7A Active CN107016016B (en) | 2016-10-10 | 2016-10-10 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107016016B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558303B (en) * | 2017-09-25 | 2023-02-28 | 阿里巴巴集团控股有限公司 | Application behavior analysis method and device and electronic equipment |
| CN110580556B (en) * | 2018-06-08 | 2023-04-07 | 阿里巴巴集团控股有限公司 | Data processing method and system and processor |
| CN109033381A (en) * | 2018-07-27 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of execution method, device and equipment of service request |
| CN113448942B (en) * | 2020-03-27 | 2022-07-22 | 阿里巴巴集团控股有限公司 | Database access method, device, equipment and storage medium |
| CN112732734A (en) * | 2020-12-31 | 2021-04-30 | 上海瑞家信息技术有限公司 | Information processing method and device |
| CN113076331B (en) * | 2021-03-23 | 2023-06-02 | 中国联合网络通信集团有限公司 | Method, device, equipment, storage medium and program product for processing middle-stage data |
| CN113094379B (en) * | 2021-03-23 | 2023-08-01 | 杭州电魂网络科技股份有限公司 | Anchored user data maintenance method, device, equipment and medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629268A (en) * | 2012-03-09 | 2012-08-08 | 华为技术有限公司 | Data synchronization method, system and date access device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030105732A1 (en) * | 2000-11-17 | 2003-06-05 | Kagalwala Raxit A. | Database schema for structure query language (SQL) server |
| CN100437587C (en) * | 2006-12-31 | 2008-11-26 | 华为技术有限公司 | Service processing method and device based on data base |
| CN101369283A (en) * | 2008-09-25 | 2009-02-18 | 中兴通讯股份有限公司 | Data synchronization method and system for internal memory database physical data base |
| CN102708203A (en) * | 2012-05-18 | 2012-10-03 | 成都科泰地理信息技术有限公司 | Database dynamic management method based on XML metadata |
| CN102841927B (en) * | 2012-07-18 | 2015-04-01 | 福建省新泽尔资讯科技有限公司 | Method and device for data interaction among a plurality of databases and distributed data interaction system |
| US9864789B2 (en) * | 2013-04-08 | 2018-01-09 | Oracle International Corporation | Method and system for implementing an on-demand data warehouse |
| CN104462119B (en) * | 2013-09-18 | 2019-11-05 | 腾讯科技（深圳）有限公司 | Data migration method and device |
-
2016
- 2016-10-10 CN CN201610882919.7A patent/CN107016016B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629268A (en) * | 2012-03-09 | 2012-08-08 | 华为技术有限公司 | Data synchronization method, system and date access device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107016016A (en) | 2017-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107016016B (en) | Data processing method and device | |
| US11226847B2 (en) | Implementing an application manifest in a node-specific manner using an intent-based orchestrator | |
| JP6353924B2 (en) | Reduced data volume durability status for block-based storage | |
| US10642694B2 (en) | Monitoring containers in a distributed computing system | |
| US10521268B2 (en) | Job scheduling method, device, and distributed system | |
| US10956374B2 (en) | Data recovery method, apparatus, and system | |
| US10445186B1 (en) | Associating a guest application within a virtual machine to create dependencies in backup/restore policy | |
| CN107016029B (en) | Method, device and system for processing service data | |
| WO2021012932A1 (en) | Transaction rollback method and device, database, system, and computer storage medium | |
| US20200007418A1 (en) | Computerized methods and systems for maintaining and modifying cloud computer services | |
| CN106897345B (en) | Data storage method and device | |
| US11307934B1 (en) | Virtual backup and restore of virtual machines | |
| US9842124B1 (en) | Highly available cluster agent for backup and restore operations | |
| US20150178122A1 (en) | Method and system for providing a high-availability application | |
| CN109829678B (en) | Rollback processing method and device and electronic equipment | |
| CN111142975A (en) | State machine persistence method and state machine persistence system | |
| CN113835625B (en) | Data storage method, device, equipment and storage medium based on sub-path | |
| CN108475211B (en) | Stateless system and system for obtaining resources | |
| CN115391337A (en) | Database partitioning method and device, storage medium and electronic equipment | |
| CN113867776A (en) | Method and device for publishing middle station application, electronic equipment and storage medium | |
| CN113687935A (en) | Cloud native storage scheduling mode based on super-fusion design | |
| CN112699129A (en) | Data processing system, method and device | |
| CN111580938A (en) | Transaction processing method, device, equipment and medium for work unit | |
| CN114691309A (en) | Batch business processing system, method and device | |
| US10339011B1 (en) | Method and system for implementing data lossless synthetic full backups |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1241084 Country of ref document: HK |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |