CN115809301A

CN115809301A - Database processing method and device, electronic equipment and readable storage medium

Info

Publication number: CN115809301A
Application number: CN202310054559.1A
Authority: CN
Inventors: 陈羽飞; 温文鎏; 李振达; 姬永飞; 邓祺; 吕图
Original assignee: Tianyi Cloud Technology Co Ltd
Current assignee: Tianyi Cloud Technology Co Ltd
Priority date: 2023-02-03
Filing date: 2023-02-03
Publication date: 2023-03-17
Anticipated expiration: 2043-02-03
Also published as: CN115809301B

Abstract

The embodiment of the application provides a database processing method, a database processing device, electronic equipment and a readable storage medium, and belongs to the technical field of computers, wherein the database processing method is applied to a distributed server, a distributed database is arranged on the distributed server, the distributed server comprises a plurality of data sub-servers, and the method comprises the following steps: receiving a data processing request aiming at the distributed database, and determining a lock resource parameter according to the lock resource information associated with the target data in the data processing request; acquiring target lock resources in a lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data; and under the condition of successfully acquiring the target lock resources in the lock managers of all the data sub-servers, performing specified processing on the target data according to the operation indicated by the data processing request. The problem that target data of each data sub-server cannot be synchronously processed is avoided, and data consistency of the distributed database can be ensured.

Description

Database processing method and device, electronic equipment and readable storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a database processing method and apparatus, an electronic device, and a readable storage medium.

Background

The distributed database is a unified database which stores data on different data sub-servers of the distributed server logically, and the client can access the distributed database through network connection to perform operations such as insertion, update, deletion and the like on the data in the distributed database.

In the prior art, a data sub-server of a distributed server receives a data processing request sent by a client, finds target data requested to be processed by the client by directly querying a distributed database, and performs a processing flow in the data processing request to operate the target data.

After the target data in the distributed database is processed, the target data on different data sub-servers needs to be synchronously processed based on the lock resources corresponding to the target data in the data sub-servers. In the prior art, after data processing, there may be a problem that target data on different data sub-servers cannot be synchronously processed, thereby causing data inconsistency of a distributed database.

Disclosure of Invention

The application provides a database processing method, a database processing device, an electronic device and a readable storage medium, so as to solve the problem of inconsistent distributed database data caused by the fact that synchronous processing cannot be performed on data.

In order to solve the technical problem, the present application is implemented as follows:

in a first aspect, the present application provides a database processing method, which is applied to a distributed server, where the distributed server is provided with a distributed database, and includes multiple data sub-servers, where the method includes:

receiving a data processing request aiming at the distributed database, and determining a lock resource parameter according to the lock resource information associated with target data in the data processing request; the target data is data requested to be processed by the data processing request;

acquiring target lock resources in the lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data;

and under the condition of successfully acquiring target lock resources in the lock managers of all the data sub-servers, performing specified processing on the target data according to the operation indicated by the data processing request.

Optionally, the data processing request includes a data processing code; the receiving a data processing request for the distributed database, and determining a lock resource parameter according to lock resource information associated with target data in the data processing request includes:

receiving, by a target data sub-server in the distributed servers, a data processing request for the distributed database;

performing tangent plane processing on the data processing code based on the target data sub-server to separate a lock resource sub-code from the data processing code;

and determining a lock resource parameter according to the lock resource information associated with the target data in the lock resource subcode.

Optionally, the obtaining, according to the lock resource parameter, a target lock resource in a lock manager of each data sub-server includes:

acquiring target lock resources in a lock manager of the target data sub-server according to the lock resource parameters;

synchronizing the lock resource parameter to other data sub-servers except the target data sub-server in the plurality of data sub-servers based on a registration node corresponding to each data sub-server in a preset coordination service assembly;

and for any one of the other data sub-servers, acquiring target lock resources in the lock managers of the other data sub-servers through the other data sub-servers according to the lock resource parameters.

Optionally, the synchronizing the lock resource parameter to the other data subservers except the target data subservers in the multiple data subservers based on the registration node corresponding to each data subserver in the preset coordination service component includes:

establishing a first connection between the target data sub-server and a registration node corresponding to the target data sub-server in a preset coordination service assembly;

sending the lock resource parameter to the coordinating service component over the first connection based on the target data sub-server;

and for any other data sub-server, sending the lock resource parameter to the other data sub-server based on the registration node corresponding to the other data sub-server in the coordination service component.

Optionally, before obtaining the target lock resource in the lock manager of each data sub-server according to the lock resource parameter, the method further includes:

responding to the data processing request, and establishing a second connection between the client and the distributed database through the target data sub-server; the client is a device for sending the data processing request to the target data sub-server;

the performing specified processing on the target data according to the operation indicated by the data processing request comprises:

executing the data processing code based on the target data sub-server to acquire data processing information from the client through the second connection, and performing specified processing on the target data in the distributed database according to the data processing information; the data processing code is for implementing the operation indicated by the data processing request.

Optionally, the method further includes:

and under the condition that the target lock resources in the lock managers of all the data sub-servers are not successfully acquired, disconnecting the second connection so as to establish connection between other clients and the distributed database and process the data in the distributed database.

Optionally, the distributed server further includes a backup sub-server, and the method further includes:

under the condition that the timing parameters in the backup sub-servers meet the preset time requirement, obtaining operation logs of the data sub-servers through the backup sub-servers, and storing the obtained operation logs in the backup sub-servers;

and when an error log exists in the operation log stored by the backup sub-server, correcting the error of the data in the data sub-server corresponding to the error log based on the backup sub-server.

Optionally, the error correction of the data in the data sub-server corresponding to the error log based on the backup sub-server includes:

determining error data according to the error log based on the backup sub-server, and determining a data sub-server storing the error data as a first data sub-server;

selecting one data sub-server storing correct data corresponding to the error data as a second data sub-server according to the error data;

and acquiring the correct data in the second data sub-server based on the backup sub-server, and replacing the error data in the first data sub-server with the correct data.

In a second aspect, the present application provides a database processing apparatus, which is applied to a distributed server, where a distributed database is disposed on the distributed server, the distributed server includes a plurality of data sub-servers, and the apparatus includes:

the determining module is used for receiving a data processing request aiming at the distributed database and determining a lock resource parameter according to the lock resource information associated with the target data in the data processing request; the target data is data requested to be processed by the data processing request;

the acquisition module is used for acquiring target lock resources in the lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data;

and the processing module is used for carrying out appointed processing on the target data according to the operation indicated by the data processing request under the condition of successfully acquiring the target lock resources in the lock managers of all the data sub-servers.

Optionally, the data processing request includes a data processing code; the determining module is specifically configured to:

receiving, by a target data sub-server in the distributed server, a data processing request for the distributed database;

Optionally, the obtaining module is specifically configured to:

synchronizing the lock resource parameters to other data sub-servers in the plurality of data sub-servers except the target data sub-server based on a registration node corresponding to each data sub-server in a preset coordination service assembly;

Optionally, the obtaining module is further specifically configured to:

Optionally, the apparatus further comprises:

the connection module is used for responding to the data processing request before the acquisition module acquires the target lock resource in the lock manager of each data sub-server according to the lock resource parameter and establishing second connection between the client and the distributed database through the target data sub-server; the client is a device for sending the data processing request to the target data sub-server;

the processing module is specifically configured to: executing the data processing code based on the target data sub-server to acquire data processing information from the client through the second connection, and performing specified processing on the target data in the distributed database according to the data processing information; the data processing code is used for realizing the operation indicated by the data processing request.

Optionally, the apparatus further comprises:

and the disconnection module is used for disconnecting the second connection under the condition that the target lock resources in the lock managers of all the data sub-servers are not successfully acquired so as to establish connection between other clients and the distributed database and process the data in the distributed database.

Optionally, the distributed server further includes a backup sub-server, and the apparatus further includes:

the storage module is used for acquiring the operation logs of each data sub-server through the backup sub-server under the condition that the timing parameters in the backup sub-server meet the preset time requirement, and storing the acquired operation logs in the backup sub-server;

and the error correction module is used for correcting the error of the data in the data sub-server corresponding to the error log based on the backup sub-server under the condition that the error log exists in the operation log stored by the backup sub-server.

Optionally, the error correction module is specifically configured to:

In a third aspect, the present application provides an electronic device, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the above-mentioned database processing method when executing the program.

In a fourth aspect, the present application provides a readable storage medium, wherein instructions of the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the above-mentioned database processing method.

In the embodiment of the application, a data processing request for a distributed database is received, and a lock resource parameter is determined according to lock resource information associated with target data in the data processing request; the target data is data requested to be processed by the data processing request; acquiring target lock resources in a lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data; and under the condition of successfully acquiring the target lock resources in the lock managers of all the data sub-servers, performing specified processing on the target data according to the operation indicated by the data processing request. For a distributed database, there may be differences in lock resources of different data sub-servers due to network errors or system errors, for example, some data sub-servers have lock resources in the lock manager that include a target lock resource, and some data sub-servers have lock resources in the lock manager that do not include a target lock resource. When the target data on different data sub-servers are synchronously processed, target lock resources corresponding to the target data on each data sub-server are needed, and under the condition that the lock resources in the lock managers of some data sub-servers include the target lock resources and the lock resources in the lock managers of some data sub-servers do not include the target lock resources, the target data on different data sub-servers cannot be synchronously processed, so that the problem of inconsistent data of the distributed database is caused. In the embodiment of the application, only when the target lock resources in the lock managers of all the data sub-servers are successfully acquired, the target data is subjected to the designated processing according to the operation indicated by the data processing request. Therefore, the problem that target data on different data sub-servers cannot be synchronously processed due to the fact that lock resources of different data sub-servers are different when subsequent synchronous processing is carried out can be avoided, namely, target lock resources exist in lock managers of some data sub-servers, and target lock resources do not exist in lock managers of some data sub-servers, and therefore data of the distributed database are guaranteed to be consistent.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a flowchart illustrating steps of a database processing method according to an embodiment of the present application;

fig. 2 is a schematic view of an application scenario of a database processing method according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating steps of another database processing method according to an embodiment of the present application;

fig. 4 is a structural diagram of a database processing apparatus according to an embodiment of the present application;

fig. 5 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

Fig. 1 is a flowchart of steps of a database processing method provided in an embodiment of the present application, and as shown in fig. 1, the method is applied to a distributed server, where a distributed database is disposed on the distributed server, and the distributed server includes a plurality of data sub-servers, where the method includes:

step 101, receiving a data processing request for the distributed database, and determining a lock resource parameter according to lock resource information associated with target data in the data processing request; the target data is data requested to be processed by the data processing request.

In the embodiment of the present application, a distributed server refers to a server device cluster including a plurality of sub servers distributed in different geographic locations. The data sub-server refers to a sub-server for storing data of the distributed database in a plurality of sub-servers included in the distributed server. The data processing request may be a network data packet sent by a computer device accessing the distributed database, the network data packet including lock resource information associated with the target data. The computer device accessing the distributed database can be a client or a proxy server for forwarding a data processing request sent by the client, and the proxy server is connected with the distributed server and used as an intermediate layer between the client and the distributed server for receiving and forwarding the data processing request sent by the client to the distributed server.

In this embodiment of the present application, the lock resource parameter may be, for example, a database primary key corresponding to a lock resource in a distributed database, a database name and a table name of the database, and the lock resource information may include information, such as the database primary key related to the lock resource corresponding to the target data, the database name and the table name of the database, which are only examples, and this is not limited in this embodiment of the present application. The database primary key refers to one column or a combination of multiple columns of identification values corresponding to each row of a data table in the distributed database, and the database name and the table name of the database refer to an identification of each sub-database of the distributed database and an identification of the data table in the sub-database respectively.

In this embodiment, the distributed server may receive a data processing request sent by a computer device, such as a client or a proxy server, to one of the data sub-servers of the distributed server. The data processing request may be generated by the computer device based on its own processing requirement for the data in the distributed database, for example, the computer device needs to perform query, update, and delete operations on some data in the distributed database, or needs to insert data in the distributed database, and the data processing request is generated according to the data to be processed, the data processing operation that needs to be executed, and the relevant information of the lock resource corresponding to the data in the distributed database. The embodiments are merely illustrative, and the present disclosure is not limited thereto.

In the embodiment of the application, after receiving the data processing request, the distributed server may parse the network data packet corresponding to the data processing request, acquire a part of data related to the lock resource in the network data packet, that is, lock resource information associated with the target data, for example, information such as a database primary key, a database name and a table name of the database, and then take an actual numerical value corresponding to the lock resource information as a lock resource parameter.

102, acquiring target lock resources in the lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data.

In the embodiment of the present application, all lock resources corresponding to all data in the distributed database may be aggregated into a whole and placed in the distributed lock manager for management, where the distributed lock manager stores an image of the distributed lock manager on each data sub-server of the distributed server, and the image is used as a lock manager of the data sub-server. For any data sub-server, the program code corresponding to the lock manager may be stored in the memory of the data sub-server, and the Central Processing Unit (CPU) of the data sub-server reads the program code corresponding to the lock manager in the memory to call the lock manager.

In the embodiment of the present application, for the lock manager of any data sub-server, the lock resource in the lock manager matching with the lock resource parameter can be searched according to the lock resource parameter, and the lock resource with the actual parameter consistent with the lock resource parameter is obtained, so as to obtain the target lock resource. Specifically, the lock resource in which the target keyword in the lock manager matches the searched keyword can be searched through the keyword in the lock resource parameter, and whether the actual value of the lock resource is consistent with the value corresponding to the lock resource parameter is compared, and if the target keyword is matched with the searched keyword and the value is consistent, the found lock resource is the target lock resource corresponding to the target data.

And 103, under the condition that the target lock resources in the lock managers of all the data sub-servers are successfully acquired, performing specified processing on the target data according to the operation indicated by the data processing request.

In the embodiment of the present application, the lock manager is an image of the distributed lock manager stored in any data sub-server, and the lock manager includes lock resources corresponding to all data of the distributed database. For any target data, the lock manager of each data sub-server has a target lock resource corresponding to the target data, even if the target data is not stored on all the data sub-servers, only stored on one or more of the data sub-servers. Therefore, the condition that the target lock resources in the lock managers of all the data sub-servers are successfully acquired represents that all the lock resources of all the data sub-servers storing the target data are acquired, and the target lock resources corresponding to the target data on all the data sub-servers are ensured to be acquired synchronously.

In the embodiment of the present application, when the target lock resources in the lock managers of all the data sub-servers are successfully acquired, corresponding codes such as insertion, update, and deletion may be executed by the data sub-servers storing the target data in the distributed servers according to the operation indicated by the data processing request, and specified processing such as insertion, update, and deletion may be performed on data in the sub-databases of the distributed databases set on the data sub-servers storing the target data, respectively, so as to complete the operation indicated by the data processing request.

Specifically, based on the code of the data subserver receiving the data processing request in the distributed server for executing the data processing operation, a data processing instruction is sent to the data subserver storing the target data through the communication network of each data subserver in the distributed server to instruct each data subserver storing the target data to perform specified processing such as insertion, update, deletion and the like on the target data in each data subserver according to the data processing instruction, so that the target data in the distributed databases are processed synchronously, and the problem that the target data on any data subserver storing the target data is inconsistent with the target data processed on other data subservers storing the target data can be avoided. For example, if the data processing request indicates to update the number of a certain device in the device ledger stored in the distributed database, the distributed server may execute a database Language (SQL) corresponding to a data update operation through the data sub-server receiving the data processing request, and update the device number value on the data sub-server storing the certain device number at the same time, or may sequentially update the device number value on the data sub-server storing the certain device number, which is only an example and is not limited in this application.

In the embodiment of the application, a data processing request for a distributed database is received, and a lock resource parameter is determined according to lock resource information associated with target data in the data processing request; the target data is data requested to be processed by the data processing request; acquiring target lock resources in a lock manager of each data sub-server according to the lock resource parameters; the target lock resource is a lock resource corresponding to the target data; and under the condition of successfully acquiring the target lock resources in the lock managers of all the data sub-servers, performing specified processing on the target data according to the operation indicated by the data processing request. For a distributed database, there may be differences in lock resources of different data sub-servers due to network errors or system errors, for example, some data sub-servers have lock resources in the lock manager that include a target lock resource, and some data sub-servers have lock resources in the lock manager that do not include a target lock resource. When the target data on different data sub-servers are synchronously processed, target lock resources corresponding to the target data on each data sub-server are needed, and under the condition that the lock resources in the lock managers of some data sub-servers include the target lock resources and the lock resources in the lock managers of some data sub-servers do not include the target lock resources, the target data on different data sub-servers cannot be synchronously processed, so that the problem of inconsistent data of the distributed database is caused. In the embodiment of the application, only when the target lock resources in the lock managers of all the data sub-servers are successfully acquired, the target data are subjected to designated processing according to the operation indicated by the data processing request. Therefore, the problem that target data on different data sub-servers cannot be synchronously processed due to the fact that lock resources of different data sub-servers are different when subsequent synchronous processing is carried out can be avoided, namely, target lock resources exist in lock managers of some data sub-servers, and target lock resources do not exist in lock managers of some data sub-servers, and therefore data of the distributed database are guaranteed to be consistent.

Optionally, the data processing request includes a data processing code; step 101 may comprise the steps of:

and step 1011, receiving a data processing request aiming at the distributed database through a target data sub-server in the distributed server.

In the embodiment of the application, the distributed server may include a plurality of data sub-servers distributed in different geographic locations, and the distributed server may receive, through each data sub-server included in the distributed server, a data processing request for the distributed database sent by the computer device. The computer device can directly send the data processing request to the data sub-server of the distributed server, and the data sub-server receiving the data processing request is the target data sub-server. The distributed server may receive, through the target data sub-server, a data processing request for the distributed database sent by the computer device, where the target data sub-server becomes a connection node for the computer device to access the distributed database this time, and the distributed server may perform access control on the distributed database provided on each data sub-server based on the connection node.

Step 1012, performing tangent plane processing on the data processing code based on the target data sub-server to separate the lock resource sub-code from the data processing code.

In this embodiment of the present application, the data processing code may be a database programming language related to data processing operations in a network data packet corresponding to the data processing request, and the data processing code includes a lock resource acquisition part code and a target data processing part code corresponding to the target data. The target data processing part code can be, for example, SQL statements of insert, update, delete corresponding to the insert, update, delete operation. The lock resource acquisition part code, i.e. the lock resource subcode, may include lock resource information as well as lock conflicts, lock pending processing logic, etc.

In the embodiment of the application, the tangent plane processing refers to cutting the packaged code, and adding some new codes into the original code to perform enhancement processing on the original code. The target data sub-server cuts the data processing code by running a tangent plane program language in the target data sub-server, adds a separation code in the data processing code, searches for a lock resource sub-code in the data processing code through the separation code, and extracts the lock resource sub-code from the data processing code, thereby realizing the operation of separating the lock resource sub-code from the data processing code. Specifically, the data processing code may be subjected to tangent plane processing by a tangent plane (Aspect) module of an open source application tool on the Java development platform, so as to separate the lock resource subcode from the data processing code. The embodiments are merely illustrative, and the present disclosure is not limited thereto.

Step 1013, determining the lock resource parameters according to the lock resource information associated with the target data in the lock resource subcode.

In the embodiment of the application, the target data sub-server may analyze the lock resource sub-code, obtain a code statement related to the lock resource corresponding to the obtained target data in the lock resource sub-code, and determine the lock resource information included in the code statement, that is, the lock resource information related to the target data, for example, information such as a database main key, a database name and a table name of the database, and then determine an actual numerical value corresponding to the lock resource information as the lock resource parameter.

It should be noted that the lock resources corresponding to the distributed database have different lock levels, also called different lock granularities, and the data processing efficiency of the distributed database is different when different lock granularities are obtained, and after the lock resources with smaller granularity are obtained, the query Per Second rate (QPS) and transaction Per Second number (TPS) corresponding to each data sub-server in the distributed server where the distributed database is located are obviously improved, thereby improving the data processing efficiency of the distributed database.

In the embodiment of the application, by performing section processing on the data processing code and separating the lock resource subcode from the data processing code, the lock resource parameter corresponding to the lock resource with smaller granularity can be determined according to the lock resource information associated with the target data in the lock resource subcode.

In the embodiment of the application, a data processing request aiming at a distributed database is received through a target data sub-server in a distributed server; performing tangent plane processing on the data processing code based on the target data sub-server to separate a lock resource sub-code from the data processing code; and determining the lock resource parameters according to the lock resource information associated with the target data in the lock resource subcode. Therefore, a data processing request aiming at the distributed database can be conveniently received on the basis of the target data sub-server, the response speed of the distributed database is improved, and the lock resource sub-code is separated after the data processing code is subjected to tangent plane processing, so that the lock resource information associated with the target data in the lock resource sub-code is more detailed, the lock resource granularity corresponding to the lock resource parameter determined according to the lock resource information is smaller, further, the target lock resource obtained by the distributed server is more accurate, and the lock resource obtaining efficiency of the distributed server is improved.

Optionally, step 102 may include the steps of:

step 1021, according to the lock resource parameter, obtaining the target lock resource in the lock manager of the target data sub-server.

In the embodiment of the application, the distributed server can receive a data processing request aiming at the distributed database through the target data sub-server, directly perform tangent plane processing on the data processing code in the target data sub-server to separate the lock resource sub-code, and determine the lock resource parameter according to the lock resource information associated with the target data in the lock resource sub-code, so that the target data sub-server can obtain the lock resource parameter. The target data servant can directly call the lock manager according to the lock resource parameter obtained by the target data servant and obtain the target lock resource in the lock manager, and the specific obtaining process may refer to the related description in step 102, which is not described herein again.

Step 1022, synchronizing the lock resource parameter to other data sub-servers in the plurality of data sub-servers except the target data sub-server based on the registered node corresponding to each data sub-server in the preset coordination service component.

In the embodiment of the application, the coordination service component refers to software for providing a consistency coordination service for the distributed database, the coordination service component encapsulates the coordination service into a simple interface to be provided for a user, and the user registers for each data sub-server of the distributed server in the coordination service component, so that each data sub-server in the distributed server can be coordinated and managed through the coordination service component. The registration node is a dedicated node created when each data sub-server registers in the coordination service component, and the dedicated node stores an Internet Protocol Address (IP Address), a port, a calling method (Protocol, serialization method), and the like corresponding to the data sub-server.

In the embodiment of the application, the target data sub-server establishes connection between the target data sub-server and the coordination service component through a corresponding registration node of the target data sub-server in the coordination service component, then sends the lock resource parameter in the target data sub-server to the coordination service component through the established connection, the coordination service component establishes connection with other data sub-servers through respective corresponding registration nodes of other data sub-servers, and then sends the lock resource parameter to other data sub-servers, so that the lock resource parameter is synchronized to other data sub-servers except the target data sub-server in the plurality of data sub-servers.

And 1023, for any one of the other data sub-servers, acquiring target lock resources in the lock managers of the other data sub-servers through the other data sub-servers according to the lock resource parameters.

In this embodiment, for any other data sub-server, the data sub-server may search, according to the lock resource parameter, a lock resource in the lock manager of the data sub-server that matches the lock resource parameter, and obtain a lock resource whose actual parameter is consistent with the lock resource parameter, to obtain a target lock resource in the lock manager of the data sub-server. For the specific searching and obtaining process, reference may be made to the related description of step 102, which is not described herein again.

In the embodiment of the application, target lock resources in a lock manager of a target data sub-server are obtained according to the lock resource parameters; synchronizing the lock resource parameters to other data sub-servers in the plurality of data sub-servers except the target data sub-server based on the registration node corresponding to each data sub-server in the preset coordination service assembly; and for any other data sub-server, acquiring target lock resources in the lock managers of other data sub-servers through other data sub-servers according to the lock resource parameters. Therefore, the target lock resource in the lock manager of the target data sub-server can be conveniently and directly obtained according to the lock resource parameter in the target data sub-server, the lock resource parameter is conveniently synchronized to other data sub-servers through the corresponding registration node of each data sub-server in the coordination service assembly, so that other data sub-servers can also obtain the target lock resource in the lock manager of the other data sub-servers according to the lock resource parameter, the operation of realizing the lock resource parameter synchronization through the coordination service assembly can accelerate the speed of obtaining the target lock resource in the lock manager of each data sub-server, and the obtaining efficiency of the target lock resource is improved.

Optionally, step 1022 may include the steps of:

step 1022a, establishing a first connection between the target data sub-server and a registered node corresponding to the target data sub-server in a preset coordination service component.

In this embodiment of the present application, the registration node corresponding to the target data servlet is a dedicated node created when the target data servlet registers in the coordination service component, and the dedicated node stores an Internet Protocol Address (IP Address), a port, a calling mode (Protocol, serialization mode), and the like corresponding to the target data servlet. The target data servant can call the target port corresponding to the target data servant according to the calling mode stored in the corresponding registration node, so that the connection between the target data servant and the corresponding registration node in the coordination service component is established through the target port, and the established connection is used as the first connection.

Step 1022b, the sub-server sends the lock resource parameter to the orchestration service component through the first connection based on the target data.

In the embodiment of the application, the target data sub-server may send the lock resource parameter in the target data sub-server to a target port corresponding to a target data sub-server in the coordination service component based on the first connection between the target data sub-server and a corresponding registered node in the coordination service component, and then the coordination service component receives the lock resource parameter through the target port, so that the lock resource parameter is sent to the coordination service component through the first connection.

Step 1022c, for any one of the other data sub-servers, based on the registration node corresponding to the other data sub-server in the coordination service component, sending the lock resource parameter to the other data sub-server.

In this embodiment, referring to the relevant description in step 1022, for any other data sub-server, the coordinating service component is provided with a registration node, which is a dedicated node created when the data sub-server is registered, and the registration node stores an Internet Protocol Address (IP Address), a port, a calling mode (Protocol, serialization mode), and the like corresponding to the data sub-server. The target data sub-server can call a target port corresponding to the data sub-server according to a call mode stored in a register node corresponding to the data sub-server, connection between the data sub-server and a corresponding register node in the coordination service assembly is established through the target port, and then the lock resource parameter is sent to the data sub-server from the coordination service assembly through the established connection, so that the lock resource parameter is sent to other data sub-servers based on register nodes corresponding to other data sub-servers in the coordination service assembly.

In the embodiment of the application, a first connection between a target data sub-server and a registration node corresponding to the target data sub-server in a preset coordination service component is established; the sub-server sends the lock resource parameter to the coordination service component through the first connection based on the target data; and for any other data sub-server, sending the lock resource parameter to other data sub-servers based on the corresponding registered nodes of other data sub-servers in the coordination service component. Therefore, the lock resource parameter can be conveniently acquired from the target data sub-server through the first connection between the target data sub-server and the corresponding registration node in the coordination service assembly, and the lock resource parameter can be conveniently synchronized to other data sub-servers based on the respective registration nodes corresponding to other data sub-servers in the coordination service assembly, so that the synchronization efficiency of the lock resource parameter can be improved through the registration node corresponding to each data sub-server in the coordination service assembly.

Optionally, before step 102, the method further includes:

step 201, responding to the data processing request, establishing a second connection between the client and the distributed database through the target data sub-server; the client is a device that sends the data processing request to the target data sub-server.

In the embodiment of the application, a management tool corresponding to the distributed database can be pre-installed in the client, the management tool stores the IP addresses of the data sub-servers of the distributed server, and the client can send a data processing request to the target data sub-server according to the IP address of the target data sub-server. After receiving the data processing request, the target data sub-server responds to the data processing request, firstly establishes communication connection between the client and the target data sub-server, then inquires the distributed database for verification according to a user name and a password sent by the client in a network data packet corresponding to the data processing request, if the verification is passed, the target data sub-server is connected with the distributed database through a communication network with other data sub-servers in the distributed server, connection between the client and the distributed database is established through the target data sub-server, and the established connection is used as a second connection.

Optionally, step 103 may include the steps of:

step 1031, executing the data processing code based on the target data sub-server to acquire data processing information from the client through the second connection, and performing specified processing on the target data in the distributed database according to the data processing information; the data processing code is for implementing the operation indicated by the data processing request.

In the embodiment of the present application, the data processing information may be operation information and data information required when the target data is subjected to the specified processing. For example, the data processing request indicates to perform specified processing such as insertion, update, deletion, etc. on the target data, the data processing information may include data that needs to be inserted into the distributed database, update data corresponding to an update operation performed on the target data, or information of the target data in the distributed database that needs to be deleted, etc. The examples are given here by way of illustration only and are not intended to be limiting.

In the embodiment of the application, based on the target data servant running data processing code, an information acquisition instruction is first sent to the client to instruct the client to send data processing information, and then a data processing instruction is sent to the data servant in which the target data is stored through a communication network of each data servant in the distributed server, where the data processing instruction may include data processing information. And each data sub-server storing the target data receives the data processing instruction, and in response to the data processing instruction, the data sub-servers respectively perform specified processing on the target data in the sub-databases arranged on the data sub-servers according to the operation information and the data information in the data processing information.

In the embodiment of the application, a second connection between the client and the distributed database is established through the target data sub-server by responding to the data processing request; the client is a device for sending a data processing request to the target data sub-server; executing data processing codes based on the target data sub-server to acquire data processing information from the client through the second connection and perform designated processing on target data in the distributed database according to the data processing information; the data processing code is for implementing the operation indicated by the data processing request. Therefore, the data processing information can be conveniently acquired through the second connection between the client and the distributed database based on the target data sub-server, and further, the data processing code is executed through the target data sub-server to accelerate the processing speed of the target data, so that the data processing efficiency of the distributed database is improved.

Optionally, the method further includes:

step 301, under the condition that the target lock resources in the lock managers of all the data sub-servers are not successfully acquired, disconnecting the second connection for establishing connection between other clients and the distributed database and processing the data in the distributed database.

In the embodiment of the present application, the reason for the situation that target lock resources in lock managers of all data child servers are not successfully acquired may include: first, a target lock resource corresponding to target data is already occupied by data processing requests sent by other clients, and at this time, the target data is being processed by an operation indicated by the data processing requests sent by other clients, so the target lock resource cannot be acquired. Secondly, due to a network error or a system error, a lock resource corresponding to target data on one or more data sub-servers is abnormal, that is, inconsistent with a target lock resource in a lock manager of another normal data sub-server, so that the lock manager of the data sub-server with the abnormal lock resource does not have the target lock resource, and the target lock resource in the lock manager of the data sub-server with the abnormal lock resource is not obtained.

In the embodiment of the application, in order to ensure that no logic conflict occurs in the data processing process and the data consistency of the distributed database, the target data is subjected to the specified processing according to the operation indicated by the data processing request under the condition that the target lock resources in the lock managers of all the data sub-servers are successfully acquired. And under the condition that the target lock resources in the lock managers of all the data sub-servers are not successfully acquired, the target data sub-server can disconnect the second connection, and after waiting for a certain time according to the processing logic of the lock waiting part in the data processing code, the second connection between the client and the distributed database is established again through the target data sub-server, so that the target data is subjected to specified processing according to the operation indicated by the data processing request. After the second connection is disconnected, the distributed server may receive a data processing request of the other client for the distributed data, and establish a connection between the other client and the distributed database to process the data in the distributed database.

In the embodiment of the application, the second connection is disconnected under the condition that the target lock resources in the lock managers of all the data sub-servers are not successfully acquired, so that the connection between other clients and the distributed database can be conveniently established, the data in the distributed database is processed, and the rationality and the practicability of the database processing method are improved.

step 401, under the condition that the timing parameter in the backup sub-server meets the preset time requirement, obtaining the operation log of each data sub-server through the backup sub-server, and storing the obtained operation log in the backup sub-server.

In the embodiment of the application, the backup sub-server is used for regularly backing up the operation logs corresponding to the data sub-servers and executing a data recovery task according to the backed-up operation logs so as to correct errors of error data in the distributed database. The operation log of the data sub-server records a server identifier of the data sub-server, such as an IP address, and an operation record for performing an operation such as inserting, updating, or deleting certain data. The timing parameter may be a time variable for timing the zeroing for timing the periodic backup tasks of the backup subservers. The preset time requirement may be determined based on the time periodicity requirement for the periodic backup task of the backup sub-server, for example, the preset time requirement may be one day, one week, or one month. The examples are given here by way of illustration only and are not intended to be limiting.

In the embodiment of the application, the condition that the timing parameter in the backup sub-server meets the preset time requirement means that the current parameter value of the timing parameter is consistent with the preset value of the preset time requirement, and the condition indicates that the current time meets the triggering condition of the periodic backup task of the backup sub-server.

Step 402, when there is an error log in the operation log stored in the backup sub-server, performing error correction on data in the data sub-server corresponding to the error log based on the backup sub-server.

In this embodiment of the present application, the error log may be an operation log in which the recorded operation record represents that data of the data sub-server corresponding to the error log is inconsistent with data of other data sub-servers. The operation record causing the data inconsistency may include: first, the target data on the data sub-server corresponding to the error log is not consistent with the target data on the other data sub-servers after being modified. Or, secondly, the operation log records of other data sub-servers storing the target data have operations on the target data, and the error log has no corresponding operation record, that is, the target data on the data sub-server corresponding to the error log is not processed synchronously.

In the embodiment of the present application, when an error log exists in an operation log stored by the backup sub-server, a data sub-server corresponding to the error log may be found based on the server identifier of the data sub-server recorded by the backup sub-server according to the error log, and for the first case, an operation on target data on the data sub-server is cancelled, and data before data processing is recovered, so that target data on the data sub-server corresponding to the error log and target data on other data sub-servers are kept consistent, that is, data recovery is performed. In the second case, according to the operation on the target data recorded by the operation logs of the other data sub-servers storing the target data, the target data on the data sub-server corresponding to the error log is synchronized, so that the target data on the data sub-server corresponding to the error log is consistent with the target data on the other data sub-servers, that is, the data is synchronized. And realizing error correction of the data in the data sub-server corresponding to the error log based on the backup sub-server through a data recovery or data synchronization process.

In the embodiment of the application, under the condition that the timing parameters in the backup sub-servers meet the preset time requirement, the operation logs of the data sub-servers are obtained through the backup sub-servers, and the obtained operation logs are stored in the backup sub-servers; and when the operation logs stored by the backup sub-server include error logs, correcting the data in the data sub-server corresponding to the error logs on the basis of the backup sub-server. Therefore, the operation logs of the data sub-servers can be conveniently backed up through the backup sub-service according to the timing parameters and the preset time requirement, and the backup sub-server corrects the data in the data sub-server corresponding to the error logs, so that the error data in the distributed database can be reduced, and the data consistency of the distributed database is improved.

Optionally, step 402 may include the steps of:

step 4021, based on the backup sub-server, determining error data according to the error log, and determining a data sub-server storing the error data as a first data sub-server.

In the embodiment of the application, the error log may be analyzed based on the backup sub-server, and it is determined that an error operation record or a missing operation record in the error log is caused when the data sub-server corresponding to the error log is inconsistent with target data of other data sub-servers in the error log. The relevant description in step 402 may be referred to for the erroneous operation record or the missing operation record, and details are not repeated here. Then, data inconsistent with data stored on other data sub-servers is found according to the wrong operation record or the missing operation record in the error log, and the found data is determined as error data. The data sub-server storing the error data can be found according to the server identification of the data sub-server recorded by the error log, and the found data sub-server is determined as the first data sub-server.

Step 4022, selecting one data sub-server storing correct data corresponding to the error data as a second data sub-server according to the error data.

In this embodiment of the present application, the error data refers to data stored in the first data sub-server and corresponding to an erroneous operation record or a missing operation record in the error log, and the operation log corresponding to a correct operation record may be determined according to the erroneous operation record or the missing operation record corresponding to the error data, and the data sub-server storing the correct data corresponding to the erroneous data may be determined according to the server identifier recorded in the correct operation log. One data sub-server may be selected from the data sub-servers storing correct data as the second data sub-server, or a data sub-server having an address position closest to the first data sub-server in the data sub-servers storing correct data is selected as the second data sub-server, which is not limited in this embodiment of the present application.

Step 4023, acquiring the correct data in the second data sub-server based on the backup sub-server, and replacing the error data in the first data sub-server with the correct data.

In the embodiment of the application, the communication network of each data sub-server in the distributed server may be connected based on the backup sub-server, correct data may be obtained from a sub-database set on the second data sub-server through the communication network and sent to the first data sub-server, and then, correct data may be replaced with incorrect data in the sub-database set on the first data sub-server.

Optionally, a recovery queue may be set in the backup sub-server, and the backup sub-server may store the error log into the recovery queue when finding that the error log exists in the stored operation log. And the distributed server sequentially takes out the error logs from the recovery queue through the recovery threads and analyzes and processes the error logs so as to correct the error data in the distributed database according to the error logs.

In the embodiment of the application, error data are determined according to the error log based on the backup sub-server, and a data sub-server storing the error data is determined as a first data sub-server; selecting one data sub-server which stores correct data corresponding to the error data as a second data sub-server according to the error data; and acquiring correct data in the second data sub-server based on the backup sub-server, and replacing error data in the first data sub-server with correct data. Therefore, the data in the data sub-server storing the error data can be conveniently corrected based on the backup sub-server, the data correction efficiency is improved, and the data consistency of the distributed database is further improved.

Fig. 2 is a schematic view of an application scenario of the database processing method provided in the embodiment of the present application, and as shown in fig. 2, each data sub-server and each backup sub-server in the distributed server may serve as a physical node with different functions corresponding to the distributed database according to their respective functions. The target data sub-server receiving the data processing request may serve as a computing node of the distributed database, the data sub-server storing the target data may serve as a storage node of the distributed database, and the backup sub-server may serve as a backup node. In addition, the coordination service component can be used as a coordination service node of the distributed database, and the coordination service node can be a virtual node and is arranged on a distributed server. The client and the distributed server can be connected through a proxy server, and the proxy server can be a proxy node which is used as an intermediate layer between the client and the distributed server and is used for receiving and forwarding data processing requests sent by the client to the computing nodes of the distributed server. The distributed database system is formed by a client, an agent node, a computing node, a storage node, a backup node and a coordination service node, all nodes in the system are in bidirectional communication connection with one another, and can transmit query instructions and data, so that the database processing method of the embodiment of the application is realized.

Fig. 3 is a flowchart of steps of another database processing method provided in an embodiment of the present application, and as shown in fig. 3, the method includes: s1, performing tangent plane processing on a data processing code connected with a distributed database to acquire a lock resource parameter related to a target lock resource. And S2, acquiring target lock resources in a lock manager of the target data sub-server based on the target data sub-server. And S3, acquiring target lock resources in the lock managers of other data sub-servers through the coordination service assembly. And S4, executing data processing codes or disconnecting the connection between the client and the distributed database according to whether the target lock resources in the lock managers of all the data sub-servers are successfully acquired. And S5, obtaining the operation logs of the data sub-servers based on the backup sub-server to backup. And S6, under the condition that the backup sub-server finds the error log, correcting error of the error data stored in the data sub-server according to the error log.

Fig. 4 is a structural diagram of a database processing apparatus provided in an embodiment of the present application, where the database processing apparatus is applied to a distributed server on which a distributed database is disposed, and the distributed server includes a plurality of data sub-servers, where the apparatus 50 may include:

a determining module 501, configured to receive a data processing request for the distributed database, and determine a lock resource parameter according to lock resource information associated with target data in the data processing request; the target data is data requested to be processed by the data processing request;

an obtaining module 502, configured to obtain, according to the lock resource parameter, a target lock resource in a lock manager of each data sub-server; the target lock resource is a lock resource corresponding to the target data;

and a processing module 503, configured to perform specified processing on the target data according to the operation indicated by the data processing request when the target lock resources in the lock managers of all the data sub-servers are successfully acquired.

Optionally, the data processing request includes a data processing code; the determining module 501 is specifically configured to:

and determining the lock resource parameters according to the lock resource information associated with the target data in the lock resource subcode.

Optionally, the obtaining module 502 is specifically configured to:

Optionally, the obtaining module 502 is further specifically configured to:

sending the lock resource parameter to the orchestration service component over the first connection based on the target data sub-server;

Optionally, the apparatus 50 further comprises:

a connection module, configured to, before the obtaining module 502 obtains the target lock resource in the lock manager of each data sub-server according to the lock resource parameter, respond to the data processing request, and establish a second connection between the client and the distributed database through the target data sub-server; the client is a device for sending the data processing request to the target data sub-server;

the processing module 503 is specifically configured to: executing the data processing code based on the target data sub-server to acquire data processing information from the client through the second connection, and performing specified processing on the target data in the distributed database according to the data processing information; the data processing code is used for realizing the operation indicated by the data processing request.

Optionally, the apparatus 50 further includes:

Optionally, the distributed server further includes a backup sub-server, and the apparatus 50 further includes:

the storage module is used for acquiring the operation logs of the data sub-servers through the backup sub-servers under the condition that the timing parameters in the backup sub-servers meet the preset time requirement and storing the acquired operation logs in the backup sub-servers;

Optionally, the error correction module is specifically configured to:

For the apparatus embodiment, since it is basically similar to the method embodiment, it is described relatively simply, and for the relevant points, refer to the partial description of the method embodiment.

The advantages of the database processing apparatus and the database processing method described above are the same as those of the prior art, and are not described herein again.

The present application also provides an electronic device 60, see fig. 5, comprising: a processor 601, a memory 602, and a computer program 6021 stored on the memory 602 and executable on the processor 601, the processor 601 implementing the database processing method of the foregoing embodiment when executing the program.

The present application also provides a readable storage medium, wherein when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the database processing method of the foregoing embodiment.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. The required structure for constructing such a system is apparent from the description above. In addition, this application is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best mode of use of the present application.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

Those skilled in the art will appreciate that the modules in the devices in an embodiment may be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore, may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a sequencing device according to the present application. The present application may also be embodied as an apparatus or device program for carrying out a part or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

It should be noted that, in the embodiments of the present application, the various data related processes are performed under the premise of complying with the data protection regulation policy corresponding to the country of the location, and obtaining the authorization given by the owner of the corresponding device.

Claims

1. A database processing method is applied to a distributed server, a distributed database is arranged on the distributed server, the distributed server comprises a plurality of data sub-servers, and the method comprises the following steps:

2. The method of claim 1, wherein the data processing request includes data processing code; the receiving a data processing request for the distributed database, and determining a lock resource parameter according to lock resource information associated with target data in the data processing request includes:

3. The method according to claim 2, wherein the obtaining a target lock resource in the lock manager of each data sub-server according to the lock resource parameter comprises:

4. The method of claim 3, wherein the synchronizing the lock resource parameter to the other data subservers of the plurality of data subservers other than the target data subservers based on a registered node corresponding to each of the data subservers in a preset coordination service component comprises:

5. The method according to claim 2, wherein before the acquiring, according to the lock resource parameter, a target lock resource in a lock manager of each of the data sub-servers, the method further comprises:

6. The method of claim 5, further comprising:

7. The method of claim 1, wherein the distributed server further comprises a backup sub-server, the method further comprising:

and when the operation log stored by the backup sub-server has an error log, correcting the data in the data sub-server corresponding to the error log based on the backup sub-server.

8. The method of claim 7, wherein the correcting the error of the data in the data sub-server corresponding to the error log based on the backup sub-server comprises:

selecting a data sub-server which stores correct data corresponding to the error data as a second data sub-server according to the error data;

9. A database processing apparatus, applied to a distributed server on which a distributed database is provided, the distributed server including a plurality of data sub-servers, the apparatus comprising:

10. The apparatus of claim 9, wherein the data processing request comprises data processing code; the determining module is specifically configured to:

11. The apparatus of claim 10, wherein the obtaining module is specifically configured to:

12. The apparatus of claim 11, wherein the obtaining module is further specifically configured to:

13. The apparatus of claim 10, further comprising:

14. An electronic device, comprising: processor, memory and computer program stored on the memory and executable on the processor, which when executed by the processor implements the database processing method of any of claims 1-8.

15. A readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the database processing method of any of claims 1-8.