CN116266185A - Data processing method and device - Google Patents

Abstract

The invention discloses a data processing method and device. The method is: receiving a data manipulation language DML request, the DML request is used to request to operate on the target data, and the DML request has nothing to do with the database type; according to the DML request, generate the first transaction for the OLTP database management system Processing a request; sending the first transaction processing request to the OLTP database management system; obtaining a transaction processing log generated by the OLTP database management system in response to the first transaction processing request; according to the transaction processing log, generating a user A second transaction processing request from the OLAP database management system; sending the second transaction processing request to the OLAP database management system.

Description

A data processing method and device

technical field

The present application relates to the field of databases, in particular to a data processing method and device.

Background technique

At present, some software application services, such as Software-as-a-Service (SaaS), have a wide range of applications. In order to meet comprehensive business requirements, these software application services often deploy an On-Line Transaction Processing (OLTP) database and an On-Line Analytical Processing (OLAP) database at the same time.

However, although these two databases are deployed on the software application service at the same time, the specific application methods of the two databases are independent of each other and need to be synchronized through separate operations, which cannot achieve effective business integration, making the current use of the database relatively inefficient. Low, this is a burning problem.

Contents of the invention

The present application provides a data processing method and device, which are used to realize the effective integration of OLTP database and OLAP database in business, and improve the use efficiency of the current database.

In the first aspect, the embodiment of the present application provides a data processing method, which can be applied to the data processing device provided in the third aspect of the present application, and the method includes:

Receive a data manipulation language DML request, the DML request is used to request an operation on the target data, the DML request has nothing to do with the database type; according to the DML request, generate a first transaction processing request for the OLTP database management system, the The first transaction processing request is used to request to operate on the target data in the OLTP database; send the first transaction processing request to the OLTP database management system; obtain the response from the OLTP database management system to the first A transaction processing log generated by a transaction processing request, the transaction processing log including information that the OLTP database management system operates on the target data; according to the transaction processing log, generate a second transaction processing for the OLAP database management system request, the second transaction processing request is used to request to operate on the target data in the OLAP database; and send the second transaction processing request to the OLAP database management system.

In the above method, since the DML request has nothing to do with the database type, the DML request is used to request to operate on the target data, because the OLTP database is transactional, so the first transaction processing request for the OLTP database management system can be generated, And after obtaining the transaction processing log sent by the OLTP database management system, it can be determined that the first transaction processing request of the OLTP database must be executed successfully, so the second transaction processing request can be generated according to the transaction processing log, and sent to the The OLAP database management system sends the second transaction processing request, so that the OLAP database can be synchronized with the OLTP database, so that the two databases can be effectively integrated in business, and the use efficiency of the database can be improved.

In a possible design, the generating the first transaction processing request for the OLTP database management system according to the DML request includes:

Perform grammatical analysis on the DML request to obtain a first grammatical tree corresponding to the DML request, the first grammatical tree includes at least one node, and any node in the at least one node is used to store the DML instruction keyword and Correspondence between structured query language SQL instruction keywords; generating the first transaction processing request according to the SQL keywords corresponding to the nodes in the first syntax tree.

With this solution, since the nodes in the syntax tree have a definite positional relationship, and any node in the at least one node is used to store the corresponding relationship between the DML instruction keyword and the structured query language SQL instruction keyword, Therefore, the first transaction processing request can be accurately generated according to the SQL keywords corresponding to the nodes in the first syntax tree.

In a possible design, the second transaction processing request is a single hybrid transaction processing request, and the generating the second transaction processing request for the OLAP database management system according to the transaction processing log includes:

According to the transaction processing log, a single mixed transaction processing request for the OLAP database management system is generated, and the mixed type transaction processing request is used to request at least one type of processing of the target data in the OLAP database data manipulation.

With this solution, at least one type of data operation can be requested on the target data with only one mixed transaction processing request, so that the OLAP database can be efficiently operated.

In a possible design, the second transaction processing request is at least one single-type transaction processing request, and generating the second transaction processing request for the OLAP database management system according to the transaction processing log includes:

According to the transaction processing log, generate at least one single-type transaction processing request for the OLAP database management system, any single-type transaction processing request is used to request one type of the target data in the OLAP database data manipulation.

With this solution, any single-type transaction processing request is used to request one type of data operation on the target data, so that the OLAP database can be operated more accurately.

In the second aspect, the embodiment of the present application provides a data processing method, which can be applied to the data processing device provided in the fourth aspect of the present application, and the method includes:

Receive a data query language DQL request, the DQL request is used to request query target data, the DQL request has nothing to do with the database type; according to the DQL request, obtain the corresponding transaction processing request, the transaction processing request includes the OLTP database The first transaction processing request of the management system or the second transaction processing request for the OLAP database management system, the first transaction processing request is used to query the target data stored in the OLTP database, and the second transaction processing request It is used to request to query the target data stored in the OLAP database; and send the transaction processing request to the database management system corresponding to the transaction processing request.

In the above method, the DQL request is used to request query target data and has nothing to do with the database type, then the corresponding transaction processing request can be obtained directly according to the DQL request, and then the transaction processing request is sent to the corresponding database management system , that is, the corresponding database management system can be flexibly selected, which improves the efficiency of data processing.

In a possible design, the acquiring a corresponding transaction processing request according to the DQL request includes:

Obtain an execution information set, the execution information set includes at least one execution information, and any execution information includes a DQL request and a corresponding transaction processing request; if there is first execution information in the execution information set, the first execution information If the received DQL request is included in the received DQL request, then the transaction processing request is obtained from the first execution information.

With this solution, since the execution information set is collected in advance, any execution information includes a DQL request and a corresponding transaction processing request, so that when a DQL request is received, it can be directly searched in the execution information set, whether there is a first execution information, then the transaction processing request corresponding to the DQL request is obtained.

In a possible design, if the first execution information does not exist in the execution information set, further include:

According to the DQL request, generate the first transaction processing request of the OLTP database management system, and the second transaction processing request of the OLAP database management system; send the first transaction processing request to the OLTP database management system A transaction processing request, determining a first performance index of a transaction processing operation execution process corresponding to the first transaction processing request, sending the second transaction processing request to the OLAP database management system, and determining a transaction corresponding to the second transaction processing request Processing the second performance index of the operation execution process; using the transaction processing request corresponding to the highest performance index among the first performance index and the second performance index as the transaction processing request corresponding to the DQL request; according to the DQL request and The transaction processing request corresponding to the DQL request generates execution information of the DQL request.

With this solution, the first performance index and the second performance index are determined through the actual execution of the first transaction processing request and the second transaction processing request, so that the transaction processing request corresponding to the higher performance index of the database management system can be recorded in the execution information.

In a possible design, the first transaction processing request corresponds to a first process, and the first process creates a first cost-optimized CBO request of the OLTP database management system, and the first CBO request is used for Obtain a first CBO result, the first CBO result being the first performance index; the second transaction processing request corresponds to a second process, and the second process creates a second CBO request of the OLAP database management system , the second CBO request is used to obtain a second CBO result, where the second CBO result is the second performance indicator; the first process and the second process are executed in parallel.

With this scheme, the first process and the second process can be executed in parallel, so that the first performance index and the second performance index can be tested at the same time, and the cost-optimized test scheme can be used to evaluate the test performance more accurately. The first performance index and the second performance index are obtained more efficiently and accurately.

In a possible design, if the first execution information does not exist in the execution information set, further include:

Determine the query type of the DQL request; determine the database management system matching the query type of the DQL request, the matching database management system is the OLTP database management system or the OLAP database management system; according to the DQL request , generating a transaction processing request corresponding to the matching database management system, and sending the transaction processing request to the matching database management system.

With this solution, by introducing query types, a database management system that matches the query types can be pre-set. Even if the first execution information does not exist in the execution information set, a more suitable query type based on the query characteristics can be generated. database management system, thereby improving its query efficiency.

In a possible design, the generating a transaction processing request corresponding to the matching database management system according to the target DQL request includes:

Perform grammatical analysis on the target DQL request to obtain a second grammatical tree corresponding to the DQL request, the second grammatical tree includes at least one node, and any node in the at least one node is used to store the DQL instruction keyword Correspondence with structured query language SQL instruction keywords; generate the second transaction processing request according to the SQL keywords corresponding to the nodes in the second syntax tree.

With this scheme, since the nodes in the syntax tree have a definite positional relationship, and any node in the at least one node is used to store the corresponding relationship between the DQL instruction keyword and the structured query language SQL instruction keyword, Therefore, the second transaction processing request can be accurately generated according to the SQL keywords corresponding to the nodes in the second syntax tree.

In a third aspect, an embodiment of the present application provides a data processing device, the device comprising:

An acquisition module, configured to receive a data manipulation language DML request, the DML request is used to request an operation on the target data, and the DML request has nothing to do with the database type;

A processing module, configured to generate a first transaction processing request for an OLTP database management system according to the DML request, where the first transaction processing request is used to request to operate on the target data in the OLTP database;

A sending module, configured to send the first transaction processing request to the OLTP database management system;

The acquisition module is further configured to acquire a transaction processing log generated by the OLTP database management system in response to the first transaction processing request, the transaction processing log including the operation of the target data by the OLTP database management system information;

The processing module is configured to generate a second transaction processing request for the OLAP database management system according to the transaction processing log, and the second transaction processing request is used to request to operate on the target data in the OLAP database;

The sending module is configured to send the second transaction processing request to the OLAP database management system.

In a possible design, the processing module is specifically used for:

Perform grammatical analysis on the DML request to obtain a first grammatical tree corresponding to the DML request, the first grammatical tree includes at least one node, and any node in the at least one node is used to store the DML instruction keyword and Correspondence between structured query language SQL instruction keywords; generating the first transaction processing request according to the SQL keywords corresponding to the nodes in the first syntax tree.

In a possible design, the second transaction processing request is a single mixed transaction processing request, and the processing module is specifically configured to:

According to the transaction processing log, a single mixed transaction processing request for the OLAP database management system is generated, and the mixed type transaction processing request is used to request at least one type of processing of the target data in the OLAP database data manipulation.

In a possible design, the second transaction processing request is at least one single-type transaction processing request, and the processing module is specifically configured to:

According to the transaction processing log, generate at least one single-type transaction processing request for the OLAP database management system, any single-type transaction processing request is used to request one type of the target data in the OLAP database data manipulation.

In a fourth aspect, the embodiment of the present application provides a data processing device, which includes:

An acquisition module, configured to receive a data query language DQL request, the DQL request is used to request query target data, and the DQL request has nothing to do with the database type;

And for obtaining a corresponding transaction processing request according to the DQL request, the transaction processing request includes a first transaction processing request for an OLTP database management system or a second transaction processing request for an OLAP database management system, the The first transaction processing request is used to request to query the target data stored in the OLTP database, and the second transaction processing request is used to request to query the target data stored in the OLAP database;

A sending module, configured to send the transaction processing request to the database management system corresponding to the transaction processing request.

In a possible design, the acquiring module is specifically used for:

Obtain an execution information set, the execution information set includes at least one execution information, and any execution information includes a DQL request and a corresponding transaction processing request; if there is first execution information in the execution information set, the first execution information If the received DQL request is included in the received DQL request, then the transaction processing request is obtained from the first execution information.

In a possible design, the device further includes a processing module, and if the first execution information does not exist in the execution information set, the processing module is specifically configured to:

According to the DQL request, generate the first transaction processing request of the OLTP database management system, and the second transaction processing request of the OLAP database management system; send the first transaction processing request to the OLTP database management system A transaction processing request, determining a first performance index of a transaction processing operation execution process corresponding to the first transaction processing request, sending the second transaction processing request to the OLAP database management system, and determining a transaction corresponding to the second transaction processing request Processing the second performance index of the operation execution process; using the transaction processing request corresponding to the highest performance index among the first performance index and the second performance index as the transaction processing request corresponding to the DQL request; according to the DQL request and The transaction processing request corresponding to the DQL request generates execution information of the DQL request.

In a possible design, the first transaction processing request corresponds to a first process, and the first process creates a first cost-optimized CBO request of the OLTP database management system, and the first CBO request is used for Obtain a first CBO result, the first CBO result being the first performance index; the second transaction processing request corresponds to a second process, and the second process creates a second CBO request of the OLAP database management system , the second CBO request is used to obtain a second CBO result, where the second CBO result is the second performance indicator; the first process and the second process are executed in parallel.

In a possible design, the device further includes a processing module, and if the first execution information does not exist in the execution information set, the processing module is further configured to:

Determine the query type of the DQL request; determine the database management system matching the query type of the DQL request, the matching database management system is the OLTP database management system or the OLAP database management system; according to the DQL request , generating a transaction processing request corresponding to the matching database management system;

The sending module is also used for:

The transaction processing request is sent to the matching database management system.

In a possible design, the processing module is specifically used for:

Perform grammatical analysis on the target DQL request to obtain a second grammatical tree corresponding to the DQL request, the second grammatical tree includes at least one node, and any node in the at least one node is used to store the DQL instruction keyword Correspondence with structured query language SQL instruction keywords; generate the second transaction processing request according to the SQL keywords corresponding to the nodes in the second syntax tree.

In a fifth aspect, an electronic device is provided, and the electronic device includes: one or more processors; one or more memories; wherein, the one or more memories store one or more computer instructions, when the When the one or more computer instructions are executed by the one or more processors, the electronic device is made to execute the method described in any one of the first aspect and the second aspect above.

In a sixth aspect, a computer-readable storage medium is provided, the computer-readable storage medium includes computer instructions, and when the computer instructions are run on a computer, the computer executes any one of the above-mentioned first aspect or second aspect. method described in the item.

For the beneficial effects of the above third aspect to the sixth aspect, please refer to the beneficial effects of the first aspect or the second aspect, and will not be repeated.

Description of drawings

FIG. 1 is a schematic diagram of a system architecture applicable to a data processing method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a server in a system architecture applicable to a data processing method provided by an embodiment of the present application;

FIG. 3 is a schematic flow chart of the steps of a data processing method provided in an embodiment of the present application;

FIG. 4 is a schematic flowchart of specific steps for processing a DML request in a data processing method provided in an embodiment of the present application;

FIG. 5 is a schematic flow chart of specific steps in a situation of processing a DQL request in a data processing method provided by an embodiment of the present application;

FIG. 6 is a schematic flowchart of specific steps in another situation of processing a DQL request in a data processing method provided by an embodiment of the present application;

FIG. 7 is a schematic flowchart of specific steps for generating execution information in a data processing method provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

The terms used in the following examples are for the purpose of describing particular examples only, and are not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also Expressions such as "one or more" are included unless the context clearly dictates otherwise. It should also be understood that in the embodiments of the present application, "one or more" refers to one or more than two (including two); "and/or" describes the association relationship of associated objects, indicating that there may be three relationships; For example, A and/or B may mean: A exists alone, A and B exist simultaneously, and B exists alone, wherein A and B may be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship.

Reference to "one embodiment" or "some embodiments" or the like in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless specifically stated otherwise.

The term "multiple" in the embodiments of the present application refers to two or more than two. In view of this, "multiple" can also be understood as "at least two" in the embodiments of the present application. "At least one" can be understood as One or more, such as being understood as one, two or more. For example, including at least one means including one, two or more, and does not limit which several are included. For example, including A, At least one of B and C, then it may include A, B, C, A and B, A and C, B and C, or A and B and C. Similarly, for "at least one" and other descriptions Understanding is also similar. "And/or" describes the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist at the same time, and B exists alone These three situations. In addition, the character "/", unless otherwise specified, generally indicates that the contextual objects are an "or" relationship.

Unless otherwise specified, ordinal numerals such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the sequence, timing, priority or importance of multiple objects.

Currently commonly used databases are divided into two categories: On-Line Transaction Processing (OLTP) databases and On-Line Analytical Processing (On-Line Analytical Processing, OLAP) databases. Both OLTP database and OLAP database have their own strengths in terms of business characteristics and technical application. In some scenarios, these two databases are often deployed at the same time. For example, in the Software-as-a-Service (SaaS) scenario, when SaaS is applied, the application software can be uniformly deployed by the manufacturer, and users can order SaaS from the SaaS manufacturer according to actual needs to obtain SaaS-related business services. In this way, users no longer need to deploy database infrastructure locally, which can effectively save operation and maintenance costs.

SaaS has developed relatively maturely and can realize multi-tenant sharing. In the case of multi-tenant sharing, in order to meet the business needs of different users, OLTP database and OLAP database can be deployed on SaaS at the same time. The SaaS database stores data records of multiple users in the form of physical tables (Physical Table), and presents them to users in the form of Logical tables. When using the data processing method provided by this application, the user's client can send a data processing request for the logical table to the SaaS server, and the SaaS server can generate a corresponding transaction processing request for the physical table according to the data processing request. Among them, the physical table refers to the data table actually created in the SaaS database and used for data record storage, and the logical table refers to the data table customized by the user according to its own business. The logical table does not exist in the SaaS database. The mapping between tables can realize the conversion of data processing requests to transaction processing requests. Among them, the data processing request can be a Data Manipulation Language (Data Manipulation Language, DML) request or a Data Query Language (Data Query Language, DQL) request, and the DML request or DQL request has nothing to do with the database type, and can be generated by different types of database management systems. The corresponding transaction processing request accesses the physical table, so as to realize the processing of DML request or DQL request.

Specifically, the physical table can be as shown in Table 1-1 below:

Table 1-1: Physical table (table name Data)

Among them, GUID, Tenant, Object, Field1, Field2, Field3 and Field4 are the fields of the data table Data, specifically, GUID is the number; Tenant is the user name, such as Tenant_A is the specific user name; Object is the table object, For example, Customer and Company refer to the table name of the specific logical table of the table object, and Field1, Field2, Field3, and Field4 are untyped fields that can store various types of fields. Users can flexibly customize the fields and map them to Field1~Field4 in the physical table.

It should be noted that the index table (table name Index_Data) of the physical table can also be established according to the type of index in each logical table in the table object, as shown in Table 1-2:

Table 1-2 Index table (table name Index_Data)

Among them, GUID, Tenant, Object, String, and Number are all fields of the index table Index_Data. String is the index of the logical table Company, and Number is the index of the logical table Customer.

In the SaaS database, the basic information of the table object can also be described through the object metadata table, and the basic information of each field in the table object can be described through the field metadata table. The object metadata table can be shown in Table 2-1. The field metadata table can be shown in Table 2-2.

Table 2-1 Object metadata table (table name Objects)

Among them, Name and Tenant are the fields of the object metadata table Objects, Name indicates the table name of the table object, and Tenant indicates the user name.

Table 2-2 Field metadata table (table name Fields)

Among them, Name, Tenant, Object, Type, Field_No and Index are fields of the field metadata table Fields. Name indicates the field name of the field defined in the table object, Tenant indicates the user name corresponding to the field, Object indicates the name of the table object to which the field belongs, Type indicates the specific type of the field, and Field_No indicates the number of the untyped field mapped to the physical table Data. For example, Field_No takes 1 to correspond to Field1, and Index indicates whether the field is an index. Obviously, users can modify the definitions in the field metadata table as needed without affecting other users.

Through Table 1-1 to Table 1-2, and Table 2-1 to Table 2-2, the logic table can be presented to the user, that is, Table 3-1 and Table 3-2. Table 3-1 is the user logic table ( The table name is Customer), and Table 3-2 is the organization logic table (table name Company).

Table 3-1 User logic table (table name Customer)

Among them, name, id, type, and company are the fields defined in the table Customer, which have been defined in Table 2-2. name indicates the user name, id indicates the user number, type indicates the user type, and company indicates the organization name corresponding to the user.

Table 3-2 Organization logic table (table name company)

Among them, name, code, category, and Staff_num are the fields defined in the table company, which have been defined in Table 2-2. name indicates the organization name, code indicates the organization code, category indicates the type of organization, and Staff_num indicates the number of people in the organization.

It should be noted that Table 1-1 to Table 1-2 and Table 2-1 to Table 2-2 are data tables actually defined in the database, while Table 3-1 and Table 3-2 are not actually defined in the database. The defined data table can be presented to the user, which is convenient for the user to operate. In the actual application process, the user's virtual access to Table 3-1~3-2 can be converted into actual access to Table 1-1~Table 1-2, but this process is not visible to the user .

Through the above SaaS example, we can see that for the SaaS multi-tenant sharing scenario, by deploying OLTP database and OLTP database at the same time, it can meet the business needs of different users and process different data processing requests.

A data processing method provided by the present application will be described in detail below. The method can be applied to the system architecture shown in FIG. 1 . The system architecture shown in FIG. It should be understood that the system architecture shown in FIG. 1 is only an example. In some embodiments, there may be more or fewer components than the system architecture shown in FIG. 1, and two or more components may be combined. components, or may have different component configurations. The various components shown in FIG. 1 may be implemented in one or more hardware, software, or a combination of hardware and software. The applicable system architecture of the data processing method may also include only the client 100 and the server 200 , and the functions of the storage 300 may be transferred to the server 200 . The system architecture shown in Figure 1 is specifically as follows:

The client 100 is used to obtain data processing requests, for example, the data processing requests may be generated according to the database operation commands input by the users, or may be received from other devices. The data processing request can be a DML request or a DQL request. The client 100 can also be used to send a data processing request to the server 200 . In this application, there is no limitation on the specific physical device for implementing the client 100, any physical device capable of initiating a data processing request may be used, for example, the client 100 may be a host device.

The server 200 is configured to acquire a data processing request from the client 100 and process the data processing request. It should be noted that during the process of processing the data processing request, the database may be accessed. OLTP database and OLAP database can be deployed at the same time in the system architecture shown in FIG. 1 . The server 200 can process the data processing request according to the OLTP database and/or the OLAP database. In this application, there is no limitation on the specific physical device for realizing the server 200, for example, the server 200 may be a server.

The storage terminal 300 is used for deploying OLTP database and OLAP database, running a database management system (Database Management System, DBMS), and storing log files of the database. It should be noted that, in the example of FIG. 1 , only one storage terminal 300 is shown, and this storage terminal 300 deploys both OLTP database and OLAP database. However, other flexible deployment methods are also possible. For example, two storage terminals 300 may be used, one of which is deployed with an OLTP database, and the other is deployed with an OLAP database. In some other instances, it is also possible to deploy the database on the server without setting the storage terminal 300 . In this application, the specific physical device for implementing the server 200 is not limited. For example, the storage device 300 may be a host device running Random Access Memory (RAM).

In the system architecture shown in FIG. 1 , the specific architecture of the server 200 may also be shown in FIG. 2 . The functional modules in the server 200 may specifically be as follows:

The execution information generator 201 may include a processing request parser 2011 , a processing request translator 2012 and an execution information cache 2013 . Among them, the processing request parser 2011 is used to parse the data processing request, such as parsing the data processing request into a syntax tree, and other parsing methods and parsing content, such as parsing the data processing request into a regular expression, etc. The request can be a processing request for operating the user logic table; the processing request translator 2012 is used to translate the parsed content (taking the syntax tree as an example in FIG. 2 ) to obtain a corresponding transaction processing request, and the transaction processing request can be used for The processing request for operating the user's physical table; the execution information cache 2013 is used to store the execution information of the transaction processing process corresponding to the transaction processing request. The execution information can be in various formats, for example, it can be a data processing request and a transaction processing request corresponding to a database. In some cases, the execution information generator 201 may also include a semantic analyzer 2014 for determining the query type of the DQL request.

The transaction processing executor 202 is used to connect to the database and send a transaction processing request to the corresponding database; the transaction processing executor 202 may include a DML executor 2021 and a DQL executor 2022, and the DML executor is used to send a DML type transaction processing request, DQL Executors are used to send transaction processing requests of DQL type.

The data synchronizer 203 may include a log parser 2031 and an OLAP executor 2032 . The log parser is used to obtain the transaction processing log of the OLTP database management system, and generate a transaction processing request of the OLAP database management system according to the transaction processing log of the OLTP database management system, so that the OLTP database is consistent with the OLAP database.

The execution optimizer 204 is used to select the transaction processing request from the transaction processing request of the OLAP database management system and the transaction processing request of the OLTP database management system. For example, a Cost-Based Optimization (CBO) method can be used to determine two transactions respectively The processing request corresponds to the performance index of the transaction processing process. The execution optimizer 204 may include an OLAP optimizer 2041 and an OLTP optimizer 2042, which are respectively configured to send corresponding transaction processing requests and CBO requests, and determine the performance indicators of the transaction processing processes corresponding to the transaction processing requests of the two databases.

The server 200 shown in FIG. 2 can be used to execute a data processing method provided in this application. Referring to FIG. 3 , a data processing method provided by the present application will be described in detail below.

Step 301: Receive a data processing request.

If the data processing request is a DML request, execute the processing method for the DML request, namely steps 302 to 306; DML requests are used to request operations on target data, and DML requests have nothing to do with database types; DQL requests are used to request query target data, and DQL requests have nothing to do with database types.

Step 302: Generate a first transaction processing request for the OLTP database management system according to the DML request.

Step 303: Send the first transaction processing request to the OLTP database management system.

Step 304: Obtain a transaction processing log generated by the OLTP database management system in response to the first transaction processing request.

Wherein, the transaction processing log includes information about the operation of the OLTP database management system on the target data.

Step 305: Generate a second transaction processing request for the OLAP database management system according to the transaction processing log.

Step 306: Send a second transaction processing request to the OLAP database management system.

Step 307: Obtain a corresponding transaction processing request according to the DQL request.

The transaction request includes a first transaction request for an OLTP database management system or a second transaction request for an OLAP database management system.

Step 308: Send the transaction processing request to the database management system corresponding to the transaction processing request.

It should be noted that, in steps 302 to 306, the first transaction processing request is used to request to operate on the target data in the OLTP database, and the second transaction processing request is used to request to operate on the target data in the OLAP database; In steps 307 to 308, the first transaction processing request is used to request to query the target data stored in the OLTP database, and the second transaction processing request is used to request to query the target data stored in the OLAP database.

Steps 302 to 306 are the processing methods of the DML request, and steps 307 to 308 are the processing methods of the DQL request. The relevant steps will be described in detail below.

DML request processing method:

In step 302, a possible implementation manner may be as follows:

Perform grammatical analysis on the DML request to obtain a first grammatical tree corresponding to the DML request, and generate a first transaction processing request according to a structured query language (Structured Query Language, SQL) keyword corresponding to a node in the first grammatical tree.

The first syntax tree includes at least one node, and any node in the at least one node is used to store the corresponding relationship between DML instruction keywords and structured query language SQL instruction keywords.

For example, for the examples in Table 1-1 to Table 3-2, the specific DML request is:

INSERT INTO Customer(name,id,type,company) VALUES('Mary',10022,'VIP1','CompanyA');

Since the DML request is a processing request for the user logical table, and the user logical table is not actually created in the database, the DML request needs to be converted into the first transaction processing request before the physical table Data can be operated. The first transaction The processing request is specifically:

INSERT INTO Data(GUID,Tenant,Objects,Filed1,Filed2,Filed3,Filed4) VALUES(GUID,'Tenant_A',Customer,'Mary',10022,'VIP1','CompanyA');

Both the DML request and the first transaction processing request are composed of multiple keywords, and the nodes of the first syntax tree store the corresponding relationship between the DML instruction keywords and the SQL instruction keywords, and the tree structure of the first syntax tree can also indicate The order of the keywords of the SQL instruction is displayed, so that the conversion of the DML request to the first transaction processing request can be realized through the first syntax tree. For example, the nodes in the first syntax tree store the corresponding relationship between Filed1 and name, and the Filed1 is converted to name. In the present application, the form of the first syntax tree is not limited, and the first transaction processing request may be generated, and any form from the implementation of the DML request to the first transaction processing request may be used.

In step 304, in a possible implementation, the transaction log of the OLTP database management system can be monitored, and the change log in the transaction log of the OLTP database management system after step 303 is executed is obtained, and the change log is used as the The transaction processing log generated by the OLTP database management system in response to the first transaction processing request; it is also possible that the OLTP database management system receives the corresponding transaction processing log sent by the OLTP database management system after successfully responding to the first transaction processing request. This is not limited.

Because the OLTP database management system satisfies transactionality, that is, atomicity, consistency, isolation and persistence (Aomicity/Consistency/Isolation/Durability, ACID). Therefore, if step 304 is executed successfully, it means that the OLTP database management system must have executed successfully, that is, the OLTP database has performed data operations as expected in the first transaction processing request. Even if the execution of step 304 fails, the OLTP database can be rolled back to the state of step 303 and executed again without causing data pollution. Therefore, sending the first transaction processing request first, and then sending the second transaction processing request can increase the fault tolerance of the data operation.

In a possible implementation manner of step 305, the second transaction processing request is a single hybrid transaction processing request, and step 305 can be specifically performed in the following manner:

From the transaction log, generate a single mixed transaction request for the OLAP database management system.

The mixed transaction processing request is used to request at least one type of data operation on the target data in the OLAP database. For example, at least one type of data operation can be: data insertion operation, data deletion operation, and data modification operation at least one of the .

In another possible implementation manner of step 305, the second transaction processing request is at least one single-type transaction processing request, and step 305 may be specifically performed in the following manner:

Based on the transaction log, at least one single-type transaction processing request for the OLAP database management system is generated.

Among them, any single-type transaction processing request is used to request a type of data operation on the target data in the OLAP database, and the data operation of any single-type transaction processing request can be: data insertion operation, data deletion operation and data modification any of the operations.

If the system architecture shown in FIG. 1 is adopted, and the server 200 adopts the architecture shown in FIG. 2 , the specific interaction process for processing a DML request may be shown in FIG. 4 .

Step 401 : the client 100 sends a DML request to the processing request parser 2011 of the server 200 .

Step 402: The processing request parser 2011 of the server 200 parses the DML request to obtain a first syntax tree corresponding to the DML request.

Step 403 : the processing request parser 2011 of the server 200 sends the first syntax tree to the processing request translator 2012 of the server 200 .

Step 404: The processing request translator 2012 of the server 200 generates a first transaction processing request according to the SQL keywords corresponding to the nodes in the first syntax tree.

Step 405 : the processing request translator 2012 of the server 200 sends the first transaction processing request to the transaction executor 202 of the server 200 .

Step 406 : the transaction executor 202 of the server 200 sends the first transaction request to the OLTP database management system in the storage 300 .

Step 407: The OLTP database management system executes the first transaction processing request.

Step 408 : The OLTP database management system sends the execution result of the first transaction processing request to the transaction processing executor 202 of the server 200 .

Step 409: The transaction executor 202 of the server 200 determines whether the first transaction request is executed successfully according to the execution result of the first transaction request.

If the execution is unsuccessful, go to step 410; otherwise, go to step 411.

Step 410: The transaction executor 202 of the server 200 returns an execution failure message to the user 100.

Step 411 : the transaction executor 202 of the server 200 returns the execution result of the first transaction request to the user 100 .

Step 412: The OLTP database management system generates a transaction processing log corresponding to the first transaction processing request according to the execution result of the first transaction processing request.

Step 413 : the OLTP database management system sends the transaction processing log corresponding to the first transaction processing request to the data synchronizer 203 of the server 200 .

Step 414: The data synchronizer 203 of the server 200 generates a second transaction processing request for the OLAP database management system according to the transaction processing log.

Step 415 : the data synchronizer 203 of the server 200 sends the second transaction processing request to the transaction executor 202 of the server 200 .

Step 416: The transaction executor 202 of the server 200 sends a second transaction request to the OLAP database management system.

Step 417: The OLAP database management system executes the second transaction processing request.

The processing method of DQL request:

A possible implementation manner of step 307 may be as follows:

An execution information set is acquired, and if first execution information exists in the execution information set, a transaction processing request is acquired from the first execution information.

The execution information set includes at least one execution information, any execution information includes a DQL request and a corresponding transaction processing request, and the first execution information includes the received DQL request.

If the first execution information exists in the execution information set, when the system architecture shown in FIG. 1 is adopted and the server 200 adopts the architecture shown in FIG. 2, the DQL request can be processed according to the interaction process shown in FIG. 5:

Step 501 : the user 100 sends a DQL request to the processing request resolver 2011 of the server 200 .

Step 502: The processing request parser 2011 of the server 200 determines that the first execution information exists in the execution information set.

Step 503 : the processing request parser 2011 of the server 200 sends the transaction processing request in the first execution information to the transaction processing executor 202 of the server 200 .

If the transaction processing request corresponding to the DQL request in the first execution information is the first transaction processing request, execute steps 504 to 506; otherwise, if the transaction processing request corresponding to the DQL request in the first execution information is the second transaction processing request, Execute steps 507 to 509.

In the processing method of the DQL request, the first transaction processing request is used to request to query the target data stored in the OLTP database, and the second transaction processing request is used to request to query the target data stored in the OLAP database.

Step 504: The transaction executor 202 of the server 200 sends a first transaction request to the OLTP database management system.

Step 505: The OLTP database management system executes the first transaction processing request.

Step 506 : The OLTP database management system sends the execution result of the first transaction processing request to the transaction processing executor 202 of the server 200 .

Step 507: The transaction executor 202 of the server 200 sends a second transaction request to the OLAP database management system.

Step 508: the OLAP database management system executes the second transaction processing request.

Step 509 : The OLAP database management system sends the execution result of the second transaction processing request to the transaction executor 202 of the server 200 .

If the first execution information does not exist in the execution information set, the DQL request may also be processed in the following manner:

Determine the query type of the DQL request; determine the database management system that matches the query type of the DQL request; generate a transaction processing request corresponding to the matching database management system according to the DQL request, and send the transaction processing request to the matching database management system.

For example, the query type of the DQL request may include joint query, conditional query and group query, etc., and the database management system corresponding to each query type may be preset according to the characteristics of the database management system, such as group query corresponding to OLAP database management System, this is because in the group query, the columns of a large number of data records in the data table will be counted. Group the query result set, and the OLAP database management system contains many indexes and adopts parallel technology, then using the OLAP database management system to execute the corresponding transaction processing request can make the performance of the transaction processing execution process higher.

In a possible implementation manner, the step of generating the transaction processing request corresponding to the matching database management system may specifically be as follows:

Perform grammatical analysis on the DQL request to obtain a second grammatical tree corresponding to the DQL request, the second grammatical tree includes at least one node, and any node in the at least one node is used to store the DQL instruction keyword and Correspondence between structured query language SQL instruction keywords; generating the second transaction processing request according to the SQL keywords corresponding to the nodes in the second syntax tree.

The above process of generating the second transaction processing request is substantially the same as the process of generating the first transaction processing request, and reference may be made to the description of generating the first transaction processing request, which will not be repeated here.

For the situation where the first execution information does not exist in the execution information set, there may be multiple implementations for processing the DQL request. For example, the database management system corresponding to the transaction processing request can be preset as an OLAP database management system, and randomly selected A database management system, etc.

For the situation where the first execution information does not exist in the execution information set, if the system architecture shown in FIG. 1 is adopted, and the server 200 shown in FIG. 2 is used, the server 200 shown in FIG. 2 can also be provided with a semantic analyzer 2014 , the DQL request can be processed according to the interaction process shown in Figure 6:

Step 601 : the user 100 sends a DQL request to the processing request resolver 2011 of the server 200 .

Step 602: The processing request resolver 2011 of the server 200 determines that the first execution information does not exist in the execution information set.

Step 603 : the processing request parser 2011 of the server 200 sends the DQL request to the semantic analyzer 2014 of the server 200 .

Step 604: The semantic analyzer 2014 of the server 200 determines the query type of the DQL request, and determines the database management system that matches the query type of the DQL request.

If the matched database management system is an OLTP database management system, execute steps 605 to 609; if the matched database management system is an OLAP database management system, execute steps 610 to 614.

Step 605: The processing request translator 2012 of the server 200 generates a first transaction processing request corresponding to the OLTP database management system according to the DQL request.

Step 606 : the processing request translator 2012 of the server 200 sends the first transaction request to the transaction executor 202 of the server 200 .

Step 607: The transaction executor 202 of the server 200 sends the first transaction request to the OLTP database management system.

Step 608: The OLTP database management system executes the first transaction processing request.

Step 609 : The OLTP database management system sends the execution result of the first transaction processing request to the transaction processing executor 202 of the server 200 .

Step 610: The processing request translator 2012 of the server 200 generates a second transaction processing request corresponding to the OLAP database management system according to the DQL request.

Step 611 : the processing request translator 2012 of the server 200 sends the second transaction request to the transaction executor 202 of the server 200 .

Step 612: The transaction executor 202 of the server 200 sends a second transaction request to the OLAP database management system.

Step 613: the OLAP database management system executes the second transaction processing request.

Step 614 : The OLAP database management system sends the execution result of the second transaction processing request to the transaction processing executor 202 of the server 200 .

It should be noted that, for the situation that the first execution information does not exist in the execution information set, the execution information of the DQL request may also be generated according to the following steps:

Step (1): Generate the first transaction processing request of the OLTP database management system and the second transaction processing request of the OLAP database management system according to the DQL request.

Step (2): Send the first transaction processing request to the OLTP database management system, determine the first performance index of the execution process of the transaction processing operation corresponding to the first transaction processing request, and send the first transaction processing request to the OLAP database management system The second transaction processing request is used to determine a second performance indicator of a transaction processing operation execution process corresponding to the second transaction processing request.

Step (3): taking the transaction processing request corresponding to the highest performance index among the first performance index and the second performance index as the transaction processing request corresponding to the DQL request.

Step (4): Generate execution information of the DQL request according to the DQL request and the transaction processing request corresponding to the DQL request.

The performance indicators in steps (1) to (4) are not limited here. For example, it may be the execution time, memory usage, etc. of the execution process of the transaction processing operation.

The first performance index and the second performance index can be obtained in the following manner:

Create a first process and a second process; execute the first process and the second process.

Wherein, the first transaction processing request corresponds to a first process, and the first process creates a first cost-based optimization (Cost-Based Optimization, CBO) request of the OLTP database management system, and the first CBO request is used to obtain The first CBO result, the first CBO result is the first performance indicator; the second transaction processing request corresponds to a second process, and the second process creates a second CBO request of the OLAP database management system, The second CBO request is used to obtain a second CBO result, where the second CBO result is the second performance indicator.

It should be noted that, the execution of the first process and the second process has no sequence constraint relationship, and may be executed in parallel or asynchronously. When the first process and the second process are executed in parallel, the test durations of the two performance indicators overlap, reducing the total test duration of the performance indicators.

For the situation where the first execution information does not exist in the execution information set, if the system architecture shown in FIG. 1 is adopted, the server 200 adopts the architecture shown in FIG. 2 , and the DQL can be obtained according to the interaction process shown in FIG. 7 Requested execution information:

Step 701 : the user 100 sends a DQL request to the processing request resolver 2011 of the server 200 .

Step 702 : the processing request parser 2011 of the server 200 sends the DQL request to the processing request translator 2012 of the server 200 .

Step 703: The processing request translator 2012 of the server 200 generates a first transaction processing request corresponding to the OLTP database management system and a second transaction processing request corresponding to the OLAP database management system according to the DQL request.

Step 704 : the processing request translator 2012 of the server 200 sends the first transaction processing request and the second transaction processing request to the execution optimizer 204 of the server 200 .

Step 705: The execution optimizer 204 of the server 200 creates a first CBO request of the first transaction processing request, and a second CBO request of the second transaction processing request.

Step 706: The execution optimizer 204 of the server 200 sends the first transaction processing request and the first CBO request to the OLTP database management system.

Step 707: The OLTP database management system executes the first transaction processing request and the first CBO request, and obtains the first CBO result.

Step 708 : The OLTP database management system sends the first CBO result to the execution optimizer 204 of the server 200 .

Step 709: The execution optimizer 204 of the server 200 sends the second transaction processing request and the second CBO request to the OLAP database management system.

Step 710: The OLAP database management system executes the second transaction processing request and the second CBO request, and obtains the second CBO result.

Step 711 : The OLTP database management system sends the second CBO result to the execution optimizer 204 of the server 200 .

It should be noted that there is no restriction on the execution order of steps 706 to 708 and steps 709 to 711 , and steps 706 to 708 may also be executed in parallel with steps 709 to 711 .

Step 712: The execution optimizer 204 of the server 200 takes the transaction processing request corresponding to the highest performance CBO result among the first CBO result and the second CBO result as the transaction processing request corresponding to the DQL request, and generates execution information of the DQL request.

Step 713 : the execution optimizer 204 of the server 200 sends the execution information of the DQL request to the execution information cache 2013 of the server 200 .

As shown in FIG. 8 , an embodiment of the present application provides a data processing device. When the device shown in Figure 8 is used to process the DML request, each module of the device is specifically used for:

An acquisition module 801, configured to receive a data manipulation language DML request, the DML request is used to request an operation on the target data, and the DML request has nothing to do with the database type;

A processing module 802, configured to generate a first transaction processing request for an OLTP database management system according to the DML request, where the first transaction processing request is used to request to operate on the target data in the OLTP database;

A sending module 803, configured to send the first transaction processing request to the OLTP database management system;

The obtaining module 801 is further configured to obtain a transaction processing log generated by the OLTP database management system in response to the first transaction processing request, and the transaction processing log includes operations performed by the OLTP database management system on the target data Information;

The processing module 802 is configured to generate a second transaction processing request for the OLAP database management system according to the transaction processing log, and the second transaction processing request is used to request to operate on the target data in the OLAP database ;

The sending module 803 is configured to send the second transaction processing request to the OLAP database management system.

In a possible design, the processing module 802 is specifically configured to:

Perform grammatical analysis on the DML request to obtain a first grammatical tree corresponding to the DML request, the first grammatical tree includes at least one node, and any node in the at least one node is used to store the DML instruction keyword and Correspondence between structured query language SQL instruction keywords; generating the first transaction processing request according to the SQL keywords corresponding to the nodes in the first syntax tree.

In a possible design, the second transaction processing request is a single mixed transaction processing request, and the processing module 802 is specifically configured to:

According to the transaction processing log, a single mixed transaction processing request for the OLAP database management system is generated, and the mixed type transaction processing request is used to request at least one type of processing of the target data in the OLAP database data manipulation.

In a possible design, the second transaction processing request is at least one single-type transaction processing request, and the processing module 802 is specifically configured to:

According to the transaction processing log, generate at least one single-type transaction processing request for the OLAP database management system, any single-type transaction processing request is used to request one type of the target data in the OLAP database data manipulation.

When the device shown in Figure 8 is used for processing DQL requests, each module of the device is specifically used for:

The acquisition module 801 is configured to receive a data query language DQL request, the DQL request is used to request query target data, and the DQL request has nothing to do with the database type;

And for obtaining a corresponding transaction processing request according to the DQL request, the transaction processing request includes a first transaction processing request for an OLTP database management system or a second transaction processing request for an OLAP database management system, the The first transaction processing request is used to request to query the target data stored in the OLTP database, and the second transaction processing request is used to request to query the target data stored in the OLAP database;

The sending module 803 is configured to send the transaction processing request to the database management system corresponding to the transaction processing request.

In a possible design, the acquiring module 801 is specifically configured to:

Obtain an execution information set, the execution information set includes at least one execution information, and any execution information includes a DQL request and a corresponding transaction processing request; if there is first execution information in the execution information set, the first execution information If the received DQL request is included in the received DQL request, then the transaction processing request is obtained from the first execution information.

In a possible design, the device further includes a processing module 802. If the first execution information does not exist in the execution information set, the processing module 802 is specifically configured to:

According to the DQL request, generate the first transaction processing request of the OLTP database management system, and the second transaction processing request of the OLAP database management system; send the first transaction processing request to the OLTP database management system A transaction processing request, determining a first performance index of a transaction processing operation execution process corresponding to the first transaction processing request, sending the second transaction processing request to the OLAP database management system, and determining a transaction corresponding to the second transaction processing request Processing the second performance index of the operation execution process; using the transaction processing request corresponding to the highest performance index among the first performance index and the second performance index as the transaction processing request corresponding to the DQL request; according to the DQL request and The transaction processing request corresponding to the DQL request generates execution information of the DQL request.

In a possible design, the first transaction processing request corresponds to a first process, and the first process creates a first cost-optimized CBO request of the OLTP database management system, and the first CBO request is used for Obtain a first CBO result, the first CBO result being the first performance index; the second transaction processing request corresponds to a second process, and the second process creates a second CBO request of the OLAP database management system , the second CBO request is used to obtain a second CBO result, where the second CBO result is the second performance indicator; the first process and the second process are executed in parallel.

In a possible design, the device further includes a processing module 802, and if the first execution information does not exist in the execution information set, the processing module 802 is further configured to:

Determine the query type of the DQL request; determine the database management system matching the query type of the DQL request, the matching database management system is the OLTP database management system or the OLAP database management system; according to the DQL request , generating a transaction processing request corresponding to the matching database management system;

The sending module 803 is also used for:

The transaction processing request is sent to the matching database management system.

In a possible design, the processing module 802 is specifically configured to:

Perform grammatical analysis on the target DQL request to obtain a second grammatical tree corresponding to the DQL request, the second grammatical tree includes at least one node, and any node in the at least one node is used to store the DQL instruction keyword Correspondence with structured query language SQL instruction keywords; generate the second transaction processing request according to the SQL keywords corresponding to the nodes in the second syntax tree.

The embodiment of the present application also provides an electronic device. The electronic device may have the structure shown in FIG. 9 . The electronic device may be a computer device, or may be a chip or a chip system capable of supporting the computer device to implement the above method.

The electronic device shown in FIG. 9 may include at least one processor 901, and the at least one processor 901 is used to be coupled with a memory, read and execute instructions in the memory to implement the data processing provided by the embodiment of the present application. step. Optionally, the electronic device may further include a communication interface 902, configured to support the electronic device in receiving or sending signaling or data. The communication interface 902 in the electronic device can be used to realize interaction with other electronic devices. The processor 901 may be used to implement the steps of electronic equipment performing data processing. Optionally, the electronic device may also include a memory 903, in which computer instructions are stored, and the memory 903 may be coupled with the processor 901 and/or the communication interface 902, and is used to support the processor 901 to call the computer instructions in the memory 903 to implement Steps of data processing; in addition, the memory 903 can also be used to store the data involved in the method embodiment of the present application, for example, to store the data and instructions necessary to support the communication interface 902 to realize interaction, and/or to store the electronic Configuration information necessary for the device to execute the method described in the embodiment of the present application.

The embodiment of the present application also provides a computer-readable storage medium. Computer instructions are stored on the computer-readable storage medium. When these computer instructions are called and executed by a computer, the computer can complete any one of the above-mentioned method embodiments and method embodiments. methods involved in a possible design. In this embodiment of the present application, the computer-readable storage medium is not limited, for example, it may be RAM (random-access memory, random access memory), ROM (read-only memory, read-only memory), and the like.

The present application also provides a chip, which may include a processor and an interface circuit, for completing the methods involved in any of the above method embodiments and any possible implementation of the method embodiments, wherein "coupling" means Two parts are directly or indirectly coupled to each other, and this coupling can be fixed or movable.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of computer instructions. When the computer instructions are loaded and executed on the computer, the procedures or functions according to the embodiments of the present invention will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, a solid state disk (Solid State Disk, SSD)) and the like.

The steps of the method or algorithm described in the embodiments of the present application may be directly embedded in hardware, a software unit executed by a processor, or a combination of both. The software unit may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other storage medium in the art. Exemplarily, the storage medium can be connected to the processor, so that the processor can read information from the storage medium, and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be set in the ASIC, and the ASIC can be set in the terminal device. Optionally, the processor and the storage medium may also be disposed in different components in the terminal device.

These computer instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device Steps are provided for implementing the functions specified in the flow chart or flow charts and/or block diagram block or blocks.

Although the invention has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations can be made thereto without departing from the scope of the invention. Accordingly, the specification and drawings are merely illustrative of the invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the invention. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (14)

1. A method of data processing, comprising:

receiving a data operation language (DML) request, wherein the DML request is used for requesting to operate on target data, and is irrelevant to a database type;

generating a first transaction request for an OLTP database management system according to the DML request, wherein the first transaction request is used for requesting to operate on the target data in an OLTP database;

Sending the first transaction request to the OLTP database management system;

acquiring a transaction log generated by the OLTP database management system in response to the first transaction request, wherein the transaction log comprises information of operations of the OLTP database management system on the target data;

generating a second transaction request for the OLAP database management system according to the transaction log, wherein the second transaction request is used for requesting to operate on the target data in the OLAP database;

and sending the second transaction request to the OLAP database management system.

2. The method of claim 1, wherein generating a first transaction request for an OLTP database management system based on the DML request comprises:

carrying out grammar analysis on the DML request to obtain a first grammar tree corresponding to the DML request, wherein the first grammar tree comprises at least one node, and any one node in the at least one node is used for storing the corresponding relation between a DML instruction keyword and a structured query language SQL instruction keyword;

and generating the first transaction processing request according to the SQL keywords corresponding to the nodes in the first grammar tree.

3. The method of claim 1 or 2, wherein the second transaction request is a single hybrid transaction request, the generating a second transaction request for an OLAP database management system from the transaction log comprising:

a single hybrid transaction request for the OLAP database management system is generated from the transaction log, the hybrid transaction request requesting at least one type of data operation on the target data in the OLAP database.

4. The method of claim 1 or 2, wherein the second transaction request is at least one single type of transaction request, the generating a second transaction request for an OLAP database management system from the transaction log comprising:

generating at least one single type transaction request for the OLAP database management system based on the transaction log, any single type transaction request being used to request one type of data operation on the target data in the OLAP database.

5. A method of data processing, comprising:

Receiving a data query language, DQL, request for requesting query target data, the DQL request being independent of database type;

acquiring corresponding transaction requests according to the DQL request, wherein the transaction requests comprise a first transaction request for an OLTP database management system or a second transaction request for an OLAP database management system, the first transaction request is used for requesting to query the target data stored in the OLTP database, and the second transaction request is used for requesting to query the target data stored in the OLAP database;

and sending the transaction request to a database management system corresponding to the transaction request.

6. The method of claim 5, wherein the obtaining the corresponding transaction request from the DQL request comprises:

acquiring an execution information set, wherein the execution information set comprises at least one piece of execution information, and any piece of execution information comprises a DQL request and a corresponding transaction processing request;

and if the first execution information exists in the execution information set, the first execution information comprises the received DQL request, and a transaction processing request is acquired from the first execution information.

7. The method of claim 6, wherein if the first execution information is not present in the execution information set, further comprising:

generating the first transaction request of the OLTP database management system and the second transaction request of the OLAP database management system from the DQL request;

sending the first transaction request to the OLTP database management system, determining a first performance index of an execution process of a corresponding transaction operation of the first transaction request, sending the second transaction request to the OLAP database management system, and determining a second performance index of an execution process of a corresponding transaction operation of the second transaction request;

taking a transaction request corresponding to the highest performance index in the first performance index and the second performance index as a transaction request corresponding to the DQL request;

and generating the execution information of the DQL request according to the DQL request and the transaction request corresponding to the DQL request.

8. The method of claim 7, wherein,

the first transaction processing request corresponds to a first process, the first process creates a first cost-optimized CBO request of the OLTP database management system, the first CBO request is used for obtaining a first CBO result, and the first CBO result is the first performance index;

The second transaction request corresponds to a second process, the second process creates a second CBO request of the OLAP database management system, the second CBO request is used to obtain a second CBO result, and the second CBO result is the second performance index;

the first process is executed in parallel with the second process.

9. The method of any of claims 6 to 8, further comprising, if the first execution information is not present in the set of execution information:

determining a query type of the DQL request;

determining a database management system with matched query types of the DQL requests, wherein the matched database management system is the OLTP database management system or the OLAP database management system;

and generating a transaction request corresponding to the matched database management system according to the DQL request, and sending the transaction request to the matched database management system.

10. The method of claim 9, wherein generating the transaction request corresponding to the matching database management system from the target DQL request comprises:

the DQL request is subjected to grammar parsing to obtain a second grammar tree corresponding to the DQL request, wherein the second grammar tree comprises at least one node, and any one node in the at least one node is used for storing the corresponding relation between the DQL instruction keywords and the structured query language SQL instruction keywords;

And generating the second transaction processing request according to the SQL keywords corresponding to the nodes in the second grammar tree.

11. A data processing apparatus, comprising:

the acquisition module is used for receiving a data operation language (DML) request, wherein the DML request is used for requesting to operate on target data, and the DML request is irrelevant to the type of a database;

the processing module is used for generating a first transaction processing request for the OLTP database management system according to the DML request, wherein the first transaction processing request is used for requesting to operate on the target data in the OLTP database;

a sending module, configured to send the first transaction processing request to the OLTP database management system;

the obtaining module is further configured to obtain a transaction log generated by the OLTP database management system in response to the first transaction request, where the transaction log includes information that the OLTP database management system operates on the target data;

the processing module is used for generating a second transaction request for the OLAP database management system according to the transaction log, wherein the second transaction request is used for requesting to operate on the target data in the OLAP database;

And the sending module is used for sending the second transaction processing request to the OLAP database management system.

12. A data processing apparatus, comprising:

an acquisition module for receiving a data query language, DQL, request for requesting query of target data, the DQL request being independent of database type;

and obtaining a corresponding transaction request according to the DQL request, the transaction request including a first transaction request for an OLAP database management system or a second transaction request for an OLAP database management system, the first transaction request being for requesting to query the target data stored in the OLAP database, the second transaction request being for requesting to query the target data stored in the OLAP database;

and the sending module is used for sending the transaction request to a database management system corresponding to the transaction request.

13. An electronic device, the electronic device comprising: one or more processors; one or more memories; wherein the one or more memories store one or more computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-4 or 5-10.

14. A computer readable storage medium comprising computer instructions which, when run on a computer, cause the computer to perform the method of any of claims 1-4 or 5-10.

Images