CN117235117A

CN117235117A - Data processing method, device, electronic equipment and storage medium

Info

Publication number: CN117235117A
Application number: CN202311278232.9A
Authority: CN
Inventors: 曹建
Original assignee: Asiainfo Technologies China Inc
Current assignee: Asiainfo Technologies China Inc
Priority date: 2023-09-28
Filing date: 2023-09-28
Publication date: 2023-12-15

Abstract

The embodiment of the application provides a data processing method, a data processing device, electronic equipment and a storage medium, and relates to the technical field of databases. The method comprises the following steps: acquiring a Structured Query Language (SQL) request; if no connection exists in the context information of the transaction corresponding to the SQL request is detected, judging the operation type of the SQL request; if the operation type is the write operation type, any write library connection in the connection pool is used as a target connection corresponding to the SQL request, and the target connection is recorded into the context information of the transaction corresponding to the SQL request; if the operation type is the read operation type, any connection in the connection pool is used as a target connection corresponding to the SQL request; and connecting the corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request. The embodiment of the application can more effectively realize read-write separation, lighten the data access pressure of the write library, improve the data processing efficiency of the write library and improve the overall resource utilization rate of the system.

Description

Data processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of database technologies, and in particular, to a data processing method, apparatus, electronic device, and storage medium.

Background

Read-write separation refers to deploying a master library that provides write services as a write library and at least one slave library that provides read-only services as a read library. By uniformly distributing the write/read operation requests on a plurality of databases, the overall access performance of the system is improved.

The existing read-write separation method determines which library to route the SQL (Structured Query Language ) request to by taking the method name as a judging condition, and can be allocated with a connection according to the method name of the method, so that a plurality of SQL requests in the method access corresponding databases through the connection. For example, all SQL requests in a save-headed method are routed to the write library and all SQL requests in a query-headed method are routed to the read library.

The existing read-write separation method is used for routing SQL requests based on method names, and a plurality of SQL requests in one method can only be routed to one database, so that all SQL requests are concentrated in a single database, the data access pressure is high, and the processing efficiency is low.

Disclosure of Invention

The embodiment of the application provides a data processing method, a device, electronic equipment and a storage medium, which can solve the problems of higher data access pressure and lower processing efficiency in the prior art.

The technical scheme is as follows:

according to an aspect of an embodiment of the present application, there is provided a data processing method, including:

acquiring a Structured Query Language (SQL) request;

if no connection exists in the context information of the transaction corresponding to the SQL request is detected, judging the operation type of the SQL request;

if the operation type is a write operation type, any write library connection in a connection pool is used as a target connection corresponding to the SQL request, and the target connection is recorded into context information of a transaction corresponding to the SQL request;

if the operation type is a read operation type, any one of the connection pools is used as a target connection corresponding to the SQL request;

and connecting a corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request.

Optionally, the method further comprises:

and if the connection exists in the context information of the transaction corresponding to the SQL request is detected, the connection in the context information is used as the target connection corresponding to the SQL request.

Optionally, the method further comprises:

if a commit operation or a rollback operation for the transaction is detected and a connection exists in the context information, the commit operation or the rollback operation is performed and the connection in the context information is released.

Optionally, the target connection includes a read library connection, and the database executes the SQL request, and then further includes:

releasing the target connection.

Optionally, the method further comprises:

initializing the connection pool;

determining, for each connection in the connection pool, a connection type of the connection based on connection information of the connection;

the connection type comprises a write library connection and a read library connection, wherein the write library connection is used for connecting a write library, and the read library connection is used for connecting a read library.

Optionally, the connection pool comprises a read library connection pool and a write library connection pool;

the method further comprises the steps of:

dividing at least one connection with a connection type of a read library connection into the read library connection pool, and dividing at least one connection with a connection type of a write library connection into the write library connection pool.

Optionally, the connection pool includes at least one write library connection;

the step of using any connection in the connection pool as the target connection corresponding to the SQL request includes:

for the SQL request, if the connection pool comprises at least one read library connection, selecting any read library connection from the connection pool, and taking the selected read library connection as a target connection corresponding to the SQL request;

Otherwise, any write library connection is selected from the connection pool, and the selected write library connection is used as a target connection corresponding to the SQL request.

According to another aspect of an embodiment of the present application, there is provided a data processing apparatus including:

the request acquisition module is used for acquiring a Structured Query Language (SQL) request;

the judging module is used for judging the operation type of the SQL request if no connection exists in the context information of the transaction corresponding to the SQL request;

the first target connection determining module is used for taking any write library connection in the connection pool as a target connection corresponding to the SQL request if the operation type is a write operation type, and recording the target connection into the context information of the transaction corresponding to the SQL request;

the second target connection determining module is used for taking any connection in the connection pool as a target connection corresponding to the SQL request if the operation type is a read operation type;

and the connection module is used for connecting the corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request.

According to another aspect of an embodiment of the present application, there is provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any one of the data processing methods described above when executing the program.

According to a further aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the data processing methods described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that:

under the condition that no connection exists in the context information of the transaction corresponding to the SQL request, determining the target connection corresponding to the SQL request according to the operation type of the SQL request, namely, performing read-write separation based on the operation type of the SQL request, wherein compared with the read-write separation based on a method name in the prior art, the read-write separation based on finer granularity is determined, and the read-write separation can be more effectively realized, so that the read library can share the SQL request of the query class, thereby reducing the data access pressure of the write library and improving the data processing efficiency of the write library; and each database is fully utilized, the overall resource utilization rate of the system is improved, and the data processing efficiency is further improved.

In addition, by deploying the SQL request routing module between the database and the service application, proxy is carried out on the connection of the database through the SQL request routing module, and the read-write separation routing logic is embedded in the SQL request routing module, so that the code of the service application is prevented from being modified, and the development period and the online period of the service application are shortened; meanwhile, no mandatory convention is imposed on service codes, and development cost is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present application;

FIG. 2 is a schematic diagram of another system architecture according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application;

FIG. 4 is a flowchart illustrating another data processing method according to an embodiment of the present application;

fig. 5 is a schematic flow chart of connection release according to an embodiment of the present application;

fig. 6 is a schematic flow chart of another connection release according to an embodiment of the present application;

FIG. 7 is a schematic diagram of yet another system architecture according to an embodiment of the present application;

FIG. 8 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below with reference to the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and the technical solutions of the embodiments of the present application are not limited.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and "comprising," when used in this specification, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, all of which may be included in the present specification. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates that at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

In the field of database access, if only one database provides read-write service, when the data volume reaches a certain scale, the read-write performance and throughput of the database can be reduced. For the above cases, a read-write separation technology may be generally adopted, where read-write separation refers to deploying a master library and at least one slave library, where the master library provides a write service as a write library, and the slave library provides a read-only service as a read library. By uniformly distributing the write/read operation requests on a plurality of databases, the overall access performance of the system is improved.

The existing read-write separation method determines which library to route the SQL (Structured Query Language ) request to by taking the method name as a judging condition, and can allocate a connection to the SQL (Structured Query Language ) request according to the method name of the method, so that a plurality of SQL requests in the method access corresponding databases through the connection to ensure the consistency of the transaction, wherein one method can be used as one transaction. For example, all SQL requests in a save-headed method are routed to the write library and all SQL requests in a query-headed method are routed to the read library.

The existing read-write separation method is used for routing SQL requests based on method names, and although read-write separation can be realized to a certain extent, a plurality of SQL requests in one method can only be routed to one database, so that all SQL requests are concentrated in a single database, the data access pressure is higher, and the processing efficiency is lower.

In addition, the existing read-write separation method comprises mandatory conventions, for example, the method name needs to be guaranteed to comprise save or query, so that the codes developed later need to adhere to the conventions, and development and drinking maintenance costs are increased.

The application provides a data processing method, a data processing device, electronic equipment and a storage medium, and aims to solve the technical problems in the prior art.

The technical solutions of the embodiments of the present application and technical effects produced by the technical solutions of the present application are described below by describing several exemplary embodiments. It should be noted that the following embodiments may be referred to, or combined with each other, and the description will not be repeated for the same terms, similar features, similar implementation steps, and the like in different embodiments.

Fig. 1 is a schematic diagram of a system architecture provided in an embodiment of the present application, where, as shown in fig. 1, the system architecture includes a service application, and a read library and a write library connected to the service application, where the service application may include a service logic module and an SQL request routing module, and the SQL request routing module may include a connection pool.

Wherein, the business application refers to a software application program which is developed based on specific business requirements and scenes and is used for supporting and solving actual business problems; the business logic module is used for executing specific logic of business; the SQL request routing module is used for routing the SQL request to the corresponding database, and the business logic module and the SQL request routing module can be deployed in one process.

A read library (i.e., slave library) refers to a copy or instance of a database used to handle read operations, typically implemented by copying data from a master library (i.e., write library) and maintaining synchronization with the master library; a write library refers to a database instance that is used to handle write operations.

The service logic module sends the SQL request to the SQL request routing module, the SQL request routing module acquires the SQL request, the context information of the transaction corresponding to the SQL request is detected, and if no connection exists in the context information of the transaction corresponding to the SQL request, the operation type of the SQL request is judged; if the operation type is the write operation type, any write library connection in the connection pool is used as a target connection corresponding to the SQL request, and the target connection is recorded into the context information of the transaction corresponding to the SQL request; if the operation type is the read operation type, any connection in the connection pool is used as a target connection corresponding to the SQL request; and connecting the corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request.

Fig. 2 is a schematic diagram of a system architecture provided in an embodiment of the present application, as shown in fig. 2, in the system architecture, an SQL request routing module may be deployed in a service form, where a service application includes only a service logic module and does not include the SQL request routing module.

Those skilled in the art will appreciate that the above examples do not constitute a limitation on the system architecture of the data processing method of the present application.

Fig. 3 is a flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 3, where the method includes:

step S110, a structured query language SQL request is obtained.

Specifically, the execution main body of the data processing method provided by the embodiment of the application can be an SQL request routing module, and the SQL request routing module can be used for proxy of the connection of the database, namely, the SQL request routing module is used as an intermediate layer between the business logic module and the database.

The service logic module can send SQL requests to the database according to service requirements, and the SQL request routing module can acquire the SQL requests by intercepting the SQL requests sent by the service logic module.

Step S120, if no connection exists in the context information of the transaction corresponding to the SQL request is detected, judging the operation type of the SQL request.

In particular, the transactions in the database may be logical units consisting of at least one SQL request, the SQL requests in the transactions being executed in a particular order. The transaction may end by commit (commit) or rollback (rollback), where commit indicates that all SQL requests in the transaction were successfully executed and the results are permanently saved in the database; rollback means that any SQL request in the transaction is retracted and the database state returns to the state before the transaction began. By transactions, it can be ensured that the data of the database remains consistent and intact in concurrent operations.

For a transaction, the context information of the transaction refers to data and state information related to the transaction that needs to be saved and managed when the database transaction is performed.

After the SQL request is intercepted, the context information of the transaction corresponding to the SQL request can be detected, when no connection exists in the context information, the operation type of the SQL request needs to be further judged, and according to the operation type of the SQL request, the database to which the SQL request is routed is decided.

Step S130, if the operation type is the write operation type, any write library connection in the connection pool is used as a target connection corresponding to the SQL request, and the target connection is recorded in the context information of the transaction corresponding to the SQL request;

in step S140, if the operation type is the read operation type, any connection in the connection pool is used as the target connection corresponding to the SQL request.

Specifically, the operation type of the SQL request may be a specific operation type executed on the database, the operation type of the SQL request may include a read operation type and a write operation type, the read operation type may be an unmodified operation such as querying data in the database, and the read operation type may include a query request; the write operation type may be an operation of performing a modification operation such as insertion, update, or deletion of data in the database, and may include an insertion request, an update request, a deletion request, and the like.

The SQL request routing module can comprise a connection pool, wherein the connection pool can comprise a plurality of connections which are preconfigured, and the connection pool can comprise at least one write library connection.

When the operation type of the SQL request is the write operation type, only the write library can process the SQL request, any write library connection in the connection pool can be allocated to the SQL request as a target connection corresponding to the SQL request, and the target connection is recorded in the context information of the transaction corresponding to the SQL request.

When the operation type of the SQL request is the read operation type, the writing library or the reading library can process the SQL request, and any connection in the connection pool can be allocated to the SQL request as a target connection corresponding to the SQL request. The connection may be a write library connection or a read library connection.

In the practical application scene of database access, the query request often occupies a larger proportion, namely the read operation is more frequent than the write operation, and in the embodiment of the application, the SQL request is decided to be routed to which database according to the operation type of the SQL request, so that the read library can share the SQL request of the query class, thereby reducing the data access pressure of the write library and improving the data processing efficiency of the write library; meanwhile, each database is fully utilized, the overall resource utilization rate of the system is improved, and the data processing efficiency is further improved.

Step S150, based on the target connection corresponding to the SQL request, connecting the corresponding database to allow the database to execute the SQL request.

Specifically, after determining the target connection corresponding to the SQL request, connecting the corresponding database according to the target connection, and when the target connection is a write library connection, routing the SQL request to the write library through the write library connection; when the target connection is a read library connection, the SQL request may be routed to the read library through the read library connection.

After receiving the SQL request, the corresponding database can execute the SQL request, so that the operations of inquiring, modifying, adding or deleting the data in the database are realized.

According to the method provided by the embodiment of the application, under the condition that no connection exists in the context information of the transaction corresponding to the SQL request, the operation type of the SQL request is judged, and the target connection corresponding to the SQL request is determined according to the operation type of the SQL request, namely, the read-write separation is carried out based on the operation type of the SQL request, compared with the read-write separation carried out based on the method name in the prior art, the granularity of the read-write separation basis is more fine, the read-write separation can be more effectively realized, so that the read library can share the SQL request of the query class, the data access pressure of the write library is reduced, and the data processing efficiency of the write library is improved; and each database is fully utilized, the overall resource utilization rate of the system is improved, and the data processing efficiency is further improved.

As an alternative embodiment, fig. 4 is a schematic flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 4, where the method further includes:

Specifically, when it is detected that a connection exists in context information of a transaction corresponding to an SQL request, it indicates that an SQL request with an operation type of a write operation type exists before the SQL request, and the connection in the context information can be used as a target connection corresponding to the SQL request, so as to ensure the consistency of the transaction.

For example, a transaction includes an insertion request and an update request which are sequentially arranged, the business logic module sends the insertion request to the SQL request acquisition module, the SQL request acquisition module acquires the insertion request, and then takes a write library connection from the connection pool as a target connection of the insertion request, records the target connection in context information of the transaction, and routes the insertion request to the write library based on the target connection for the write library to perform data insertion operation based on the insertion request; the business logic module sends the update request to the SQL request acquisition module, after the SQL request acquisition module acquires the update request, at the moment, a write library connection exists in the context information of the transaction, the write library connection is used as a target connection of the update request, and the update voice is routed to the write library based on the target connection so as to enable the write library to perform data update operation based on the update request.

In the above example, if the update request is changed for the data inserted by the previous insertion request, the two requests use the same connection by recording the target connection corresponding to the insertion request in the context information and using the recorded connection as the target connection of the subsequent SQL request, so as to ensure that the update operation corresponding to the update request can access the data inserted by the insertion request, avoid the situation that the subsequent request cannot access the inserted data when the transaction is not finished, and ensure the consistency of the transaction.

In addition, if the update request is not changed for the data inserted by the previous insertion request, the occupation of the write library connection in the connection pool can be reduced by multiplexing the target connection of the insertion request, so that the stability, the reliability and the concurrent processing capability of the system are improved.

In the embodiment of the application, when the SQL request with the first operation type of writing operation type in the transaction is obtained, a writing library connection is taken out from the connection pool as the target connection of the SQL request, and the target connection is recorded in the context information of the transaction so that the subsequent SQL request in the transaction can be routed based on the target connection, thereby ensuring the consistency of the transaction, reducing the connection occupation and improving the stability, the reliability and the concurrent processing capability of the system.

As an alternative embodiment, the method further comprises:

if a commit or rollback operation for the transaction is detected and a connection exists in the context information, then the commit or rollback operation is performed and the connection in the context information is released.

Specifically, when a commit operation or a rollback operation for a transaction is detected, which indicates that the transaction has ended, in this case, a connection in context information of the transaction may be detected, and if the context information has a connection, the connection in the context information may be released after the commit operation or the rollback operation is performed.

Note that, since the write library connection allocated to the SQL request is recorded in the context information of the transaction only if the operation type of the SQL request is the write operation type, the connections recorded in the context information are all write library connections.

Fig. 5 is a schematic flow chart of connection release provided in an embodiment of the present application, as shown in fig. 5, after the SQL request routing module intercepts a commit operation or a rollback operation for a transaction, it may determine whether a connection exists in context information of the transaction, if the connection exists, the connection in the context information is released after the commit operation or the rollback operation is performed, and the context information is cleaned, otherwise, the connection is directly ended.

In the embodiment of the application, the connection in the context information of the transaction is released after the transaction is finished, and the occupation of the connection of the write library is reduced on the premise of ensuring the consistency of the transaction, so that the requirement on the number of the connection of the write library of the connection pool is reduced, each database connection needs to occupy certain system resources, and when the number of the connections in the connection pool is excessive, a large amount of system resources are consumed, and the system load is possibly caused to be overlarge, so that the performance is reduced and even crashes are caused. By reducing the occupation of the write library connection, the resource consumption can be effectively saved, and the stability and the reliability of the system are improved.

Further, database connections are often limited resources, and when a write library connection in a connection pool is too busy, other requests may not be able to acquire an available connection and be blocked or delayed. By reducing occupation of write library connection, write library connection in a connection pool can be ensured to better meet the requirement of concurrent request, and concurrent processing capacity of a system is improved.

As an alternative embodiment, the target connection comprises a read library connection, and the database executes an SQL request, and then further comprises:

releasing the target connection.

Specifically, after the SQL request is obtained, if no connection exists in the context information of the transaction corresponding to the SQL request and the operation type of the SQL request is the read operation type, one connection can be taken out from the connection pool as the target connection of the SQL request.

If the target connection includes a read library connection, i.e., one read library connection is fetched from the connection pool and allocated to the SQL request, the read library connection may be released immediately after the database has executed the SQL request, thereby reducing the overhead of the read library connection.

Fig. 6 is a schematic flow chart of connection release provided in an embodiment of the present application, as shown in fig. 6, after the SQL request is executed, whether the target connection includes a read library connection may be determined, if yes, the target connection is released immediately, otherwise, the connection is ended directly.

In the embodiment of the application, the target connection comprising the read library connection is released immediately after the SQL request is executed, so that the occupation of the read library connection is reduced, the requirement on the number of the read library connections of the connection pool is reduced, each database connection needs to occupy certain system resources, and when the number of the connections in the connection pool is excessive, the system resources are consumed in a large amount, and the system load is possibly caused to be excessive, so that the performance is reduced and even crashed. By reducing the occupation of the read library connection, the resource consumption can be effectively saved, and the stability and the reliability of the system are improved.

Further, database connections are often limited resources, and when a read library connection in a connection pool is too busy, other requests may not be able to acquire an available connection and be blocked or delayed. The occupation of the read library connection is reduced, so that the read library connection in the connection pool can be ensured to better meet the requirement of concurrent requests, and the concurrent processing capacity of the system is improved.

As an alternative embodiment, the method further comprises:

initializing a connection pool;

determining a connection type of the connection based on connection information of the connection for each connection in the connection pool;

Specifically, prior to obtaining the SQL request, the SQL request routing module may initialize a connection pool by creating a plurality of pre-established connections and saving the connections in the connection pool. For each connection in the connection pool, a connection type of the connection may be identified based on connection information of the connection.

The connection information of the connection may include parameters and configuration information required for connection to the database, and the connection type may include a write library connection for connecting to the write library and a read library connection for connecting to the read library.

In addition, for each connection of the connection pool, the connection can be also set to be automatically submitted to be opened or closed, when the automatic submission is opened, each SQL statement is automatically submitted to the database after being executed, the submitting operation is not required to be manually executed, namely, after each updating operation is executed, the operation is immediately effective and is persisted to the database, and one SQL request can be understood as one transaction; when the auto commit closes, commit operations need to be performed manually to commit a transaction or a set of related operations, which are permanently saved to the database, which does not apply or save any executed SQL statements to maintain data consistency and integrity.

Optionally, fig. 7 is a schematic diagram of a system architecture provided in an embodiment of the present application, where, as shown in fig. 7, the connection pool includes a read pool connection pool and a write pool connection pool;

the method further comprises the steps of:

at least one connection with a connection type of a read library connection is divided into a read library connection pool, and at least one connection with a connection type of a write library connection is divided into a write library connection pool.

Specifically, the connection pool may further include a read library connection pool and a write library connection pool, the read library connection pool may be used to store read library connections, and the write library connection pool is used to store write library connections.

After identifying the connection type of the connection in the connection pool, at least one connection of which the connection type is a read library connection may be partitioned into the read library connection pool and at least one connection of which the connection type is a write library connection may be partitioned into the write library connection pool such that the read library connection and the write library connection are stored separately.

As an alternative embodiment, the connection pool includes at least one write library connection;

taking any one of the connection pools as a target connection corresponding to the SQL request, wherein the method comprises the following steps:

Specifically, in the connection pool after initialization, the connection pool includes at least one write library connection. If no connection exists in the context information of the transaction corresponding to the SQL request, and the operation type of the SQL request is a read operation type, judging whether a read library connection exists in a connection pool, if the connection pool comprises at least one read library connection, selecting any read library connection from the connection pool, and taking the selected read library connection as a target connection corresponding to the SQL request; otherwise, any write library connection is selected from the connection pool, and the selected write library connection is used as a target connection corresponding to the SQL request.

Fig. 8 is a flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 8, where the method includes:

(1) Intercepting an SQL request;

(2) Judging whether the automatic submission of the connection is started or not;

(3) If the connection is automatically submitted and started, identifying the operation type of the SQL request; if the operation type of the SQL request is the write operation type, taking out a write library connection from the connection pool as a target connection of the SQL request, and recording the write library connection in the context information of the transaction; if the operation type of the SQL request is a read operation type and the read library connection exists in the connection pool, taking out one read library connection from the connection pool as a target connection of the SQL request; otherwise, taking out a write library connection from the connection pool as a target connection of the SQL request;

(4) If the connection is automatically submitted and closed, judging whether the connection exists in the context information of the transaction corresponding to the SQL request;

(5) When the connection exists in the context information, the connection in the context is used as the target connection of the SQL request; returning to the step (2) when no connection exists in the context information;

(6) After determining the target connection, connecting a corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request;

(7) After the SQL request is executed, judging whether the target connection comprises a library reading connection or not, and if so, releasing the target connection; otherwise, directly ending.

According to the data processing method provided by the embodiment of the application, the SQL request routing module is used for proxy to database connection, and the SQL request is intercepted and is routed to the read library or the write library according to the operation type of the SQL request by the read-write separation strategy, so that the read-write separation can be more effectively realized on the premise of ensuring the consistency of the transaction, the read library can share the SQL request of the query class, the data access pressure of the write library is reduced, and the data processing efficiency of the write library is improved; and each database is fully utilized, the overall resource utilization rate of the system is improved, and the data processing efficiency is further improved. Meanwhile, the code of the service application is prevented from being modified, and the development period and the online period of the service application are shortened; meanwhile, no mandatory convention is imposed on service codes, and development cost is reduced.

Fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, as shown in fig. 9, where the apparatus includes:

a request acquisition module 210, configured to acquire a structured query language SQL request;

a judging module 220, configured to judge an operation type of the SQL request if no connection exists in the context information of the transaction corresponding to the SQL request;

the first target connection determining module 230 is configured to, if the operation type is a write operation type, connect any write library in the connection pool as a target connection corresponding to the SQL request, and record the target connection into context information of a transaction corresponding to the SQL request;

a second target connection determining module 240, configured to, if the operation type is a read operation type, use any connection in the connection pool as a target connection corresponding to the SQL request;

and the connection module 250 is used for connecting the corresponding database based on the target connection corresponding to the SQL request so as to enable the database to execute the SQL request.

As an alternative embodiment, the apparatus further comprises:

and the third target connection determining module is used for taking the connection in the context information as the target connection corresponding to the SQL request if the connection exists in the context information of the transaction corresponding to the SQL request.

As an alternative embodiment, the apparatus further comprises:

and the first connection release module is used for executing the commit operation or the rollback operation and releasing the connection in the context information if the commit operation or the rollback operation for the transaction is detected and the connection exists in the context information.

As an alternative embodiment, the target connection comprises a read library connection, and the apparatus further comprises:

and the second connection release module is used for releasing the target connection.

As an alternative embodiment, the device also comprises a connection pool configuration module for

Initializing the connection pool;

As an alternative embodiment, the connection pool includes a read library connection pool and a write library connection pool;

the device also comprises a connection dividing module for:

As an alternative embodiment, the connection pool comprises at least one write library connection;

the second target connection determining module is specifically configured to:

The device of the embodiment of the present application may perform the method provided by the embodiment of the present application, and its implementation principle is similar, and actions performed by each module in the device of the embodiment of the present application correspond to steps in the method of the embodiment of the present application, and detailed functional descriptions of each module of the device may be referred to the descriptions in the corresponding methods shown in the foregoing, which are not repeated herein.

The embodiment of the application provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to realize the steps of the data processing method, and compared with the related technology, the steps of the data processing method can be realized: under the condition that no connection exists in the context information of the transaction corresponding to the SQL request, determining the target connection corresponding to the SQL request according to the operation type of the SQL request, namely, performing read-write separation based on the operation type of the SQL request, wherein compared with the read-write separation based on a method name in the prior art, the read-write separation based on finer granularity is determined, and the read-write separation can be more effectively realized, so that the read library can share the SQL request of the query class, thereby reducing the data access pressure of the write library and improving the data processing efficiency of the write library; and each database is fully utilized, the overall resource utilization rate of the system is improved, and the data processing efficiency is further improved. In addition, by deploying the SQL request routing module between the database and the service application, proxy is carried out on the connection of the database through the SQL request routing module, and the read-write separation routing logic is embedded in the SQL request routing module, so that the code of the service application is prevented from being modified, and the development period and the online period of the service application are shortened; meanwhile, no mandatory convention is imposed on service codes, and development cost is reduced.

In an alternative embodiment, there is provided an electronic device, as shown in fig. 10, the electronic device 4000 shown in fig. 10 includes: a processor 4001 and a memory 4003. Wherein the processor 4001 is coupled to the memory 4003, such as via a bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

Bus 4002 may include a path to transfer information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 10, but not only one bus or one type of bus.

Memory 4003 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (Electrically Erasable Programmable Read Only Memory ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer.

The memory 4003 is used for storing a computer program for executing an embodiment of the present application, and is controlled to be executed by the processor 4001. The processor 4001 is configured to execute a computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiment.

Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the foregoing method embodiments and corresponding content.

The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate, such that the embodiments of the application described herein may be implemented in other sequences than those illustrated or otherwise described.

It should be understood that, although various operation steps are indicated by arrows in the flowcharts of the embodiments of the present application, the order in which these steps are implemented is not limited to the order indicated by the arrows. In some implementations of embodiments of the application, the implementation steps in the flowcharts may be performed in other orders as desired, unless explicitly stated herein. Furthermore, some or all of the steps in the flowcharts may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or phases may be performed at the same time, or each of these sub-steps or phases may be performed at different times, respectively. In the case of different execution time, the execution sequence of the sub-steps or stages can be flexibly configured according to the requirement, which is not limited by the embodiment of the present application.

The foregoing is merely an optional implementation manner of some of the implementation scenarios of the present application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the present application are adopted without departing from the technical ideas of the scheme of the present application, and the implementation manner is also within the protection scope of the embodiments of the present application.

Claims

1. A method of data processing, comprising:

acquiring a Structured Query Language (SQL) request;

2. The data processing method of claim 1, wherein the method further comprises:

3. The data processing method of claim 1, wherein the method further comprises:

4. The data processing method of claim 1, wherein the target connection comprises a read library connection, the database executing the SQL request, and further comprising thereafter:

releasing the target connection.

5. The data processing method of claim 1, wherein the method further comprises:

initializing the connection pool;

6. The data processing method of claim 5, wherein the connection pool comprises a read library connection pool and a write library connection pool;

The method further comprises the steps of:

7. The data processing method of claim 5, wherein the connection pool comprises at least one write library connection;

8. A data processing apparatus, comprising:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the method according to any one of claims 1 to 7.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 7.