CN114422567A

CN114422567A - Data request processing method, device, system, computer equipment and medium

Info

Publication number: CN114422567A
Application number: CN202111499493.4A
Authority: CN
Inventors: 肖冰; 孟庆义; 沈春辉; 杨成虎
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2021-12-09
Filing date: 2021-12-09
Publication date: 2022-04-29

Abstract

One or more embodiments of the present specification provide a method, an apparatus, a system, a computer device, and a computer-readable storage medium for processing a data request, where the method is applied to a client and is used to: creating a first connection session for maintaining a connection with a master cluster and creating a second connection session for maintaining a connection with a slave cluster, the master cluster and the slave cluster being distributed database clusters; acquiring configuration information; if the configuration information indicates that the client preferentially accesses a main cluster, preferentially sending a data request to the main cluster for processing based on the first connection session; and if the main cluster does not normally return a processing result, the data request is sent to the standby cluster based on a second connection session, so that the standby cluster processes the data request.

Description

Data request processing method, device, system, computer equipment and medium

Technical Field

Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a method, an apparatus, a system, a computer device, and a computer-readable storage medium for processing a data request.

Background

Service continuity availability is the lifeline of business and is one of the core values of distributed storage systems. For online services, the availability of the services is more sensitive, and the requirement on data reliability is higher, so that scenarios such as online services often adopt dual clusters or multiple clusters. In a double-cluster or multi-cluster scenario, in some related technologies, a user uses a client to connect a master cluster, and if the user finds that the master cluster has a problem, the user needs to manually update the connection configuration of the client to connect the client to a backup cluster, and the client needs to be restarted after the update, and then the client is switched to access the backup cluster after the restart. The switching process needs to go from the condition that a user finds a cluster problem and manually updates to the condition that a client restarts, so that the switching speed is slow and the high availability of the service cannot be ensured.

Disclosure of Invention

To overcome the problems in the related art, the present specification provides a method, an apparatus, a system, a computer device, and a computer-readable storage medium for processing a data request.

According to a first aspect of embodiments of the present specification, there is provided a method for processing a data request, the method being applied to a client, the method including:

creating a first connection session for maintaining a connection with a master cluster and creating a second connection session for maintaining a connection with a slave cluster, the master cluster and the slave cluster being distributed database clusters;

acquiring configuration information;

if the configuration information indicates that the client preferentially accesses a main cluster, preferentially sending a data request to the main cluster for processing based on the first connection session;

and if the main cluster does not normally return a processing result, the data request is sent to the standby cluster based on a second connection session, so that the standby cluster processes the data request.

According to a second aspect of embodiments herein, there is provided an apparatus for processing a data request, the apparatus comprising:

the system comprises a multi-cluster access module and an automatic perception module; wherein the content of the first and second substances,

the multi-cluster access module is to: creating a first connection session for maintaining a connection with a master cluster and a second connection session for maintaining a connection with a slave cluster, the master cluster and the slave cluster being distributed database clusters; acquiring configuration information; if the configuration information indicates that the client preferentially accesses a main cluster, preferentially sending a data request to the main cluster for processing based on the first connection session; if the main cluster does not normally return a processing result, the data request is sent to the standby cluster based on a second connection session, so that the standby cluster processes the data request;

the automatic perception module is used for: and collecting the response result of the main cluster and/or the standby cluster to the data request, and judging whether the main cluster and/or the standby cluster do not return a normal processing result.

According to a third aspect of embodiments of the present specification, a system for processing a data request is provided, where the system includes a master cluster and a slave cluster, and both the master cluster and the slave cluster are distributed database clusters; the system further comprises at least one client; the client is configured to:

creating a first connection session for maintaining a connection with the master cluster and a second connection session for maintaining a connection with the standby cluster;

acquiring configuration information;

According to a fourth aspect of embodiments herein, there is provided a computer apparatus comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor executes the executable instructions to implement an embodiment of the data request processing method according to the first aspect.

According to a fifth aspect of embodiments herein, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps in the embodiments of the method for processing a data request according to the first aspect.

The technical scheme provided by the embodiment of the specification can have the following beneficial effects:

in the embodiment of the present specification, a client creates two connection sessions, where a first connection session is used to maintain connection with a master cluster, and a second connection session is used to maintain connection with a slave cluster, so that the client can maintain a connection state with the master cluster and the slave cluster, and the client obtains configuration information, and a data request of the client can be sent according to an instruction of the configuration information; if the configuration information indicates that the client preferentially accesses a main cluster, preferentially sending a data request to the main cluster for processing based on the first connection session; if the main cluster does not normally return a processing result, the data request is sent to the standby cluster based on a second connection session, so that the standby cluster processes the data request; therefore, the client can automatically and quickly access another cluster according to the processing condition of the cluster on the data request, user operation is not needed in the switching process, high availability of service can be guaranteed, and the data request can be quickly processed.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present specification and together with the description, serve to explain the principles of the specification.

Fig. 1A is an application scenario diagram illustrating a method for processing a data request according to an exemplary embodiment of the present disclosure.

FIG. 1B is a flow chart illustrating a method of processing a data request according to an exemplary embodiment of the present description.

Fig. 2A is a schematic diagram of a multi-cluster access module shown in accordance with an exemplary embodiment of the present description.

Fig. 2B is a schematic processing procedure diagram of a cluster switching module according to an exemplary embodiment shown in this specification.

Fig. 2C is a diagram illustrating an auto-perception process of an auto-perception module according to an exemplary embodiment of the present disclosure.

Fig. 3 is a hardware block diagram of a computer device in which a data request processing apparatus according to an exemplary embodiment is shown in the present specification.

FIG. 4 is a block diagram illustrating a data request processing device according to an example embodiment of the present specification.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.

The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

Service continuity availability is the lifeline of business and is one of the core values of distributed storage systems. For online services, the availability of the services is more sensitive, and the requirement on data reliability is higher, so that scenarios such as online services often adopt dual clusters or multiple clusters. Through dual-cluster or multi-cluster disaster recovery, better service stability can be provided for online service. Disaster recovery means that when the primary cluster is detected to be unavailable, the traffic can be switched to the backup cluster, so that the service availability is improved.

Fig. 1A is a schematic diagram of an application scenario of a data request processing method according to an exemplary embodiment shown in this specification, where fig. 1A illustrates a main cluster and a standby cluster by taking two clusters as an example, where a cluster has at least two servers; in practical applications, more standby clusters may be included according to needs, which is not limited in this embodiment. Fig. 1A also includes a client, where the client refers to a client that needs to access the cluster to obtain the service provided by the cluster, and the client includes one or more clients.

As shown in fig. 1A, in a scenario including dual clusters, if a main cluster cannot meet a high availability requirement of a user, a traffic may be switched to a standby cluster, and a problem exists in practical application that a switching speed is slow and the high availability requirement cannot be met.

Taking an HBase cluster as an example, the HBase cluster is a distributed data storage cluster, which is a storage cluster constructed by a NoSQL database and a storage system in Hadoop ecology and is widely used in online systems such as search, commodity orders, consumption records, monitoring systems or chat push.

In the existing HBase, there is a synchronization (Replication) mechanism, i.e. a data synchronization mechanism between clusters. The Replication can synchronize data in the main HBase cluster to the standby HBase cluster in real time. Therefore, when the main HBase cluster fails, the user can still access the standby HBase cluster, and the purpose of high availability is achieved.

However, the clients of the HBase cluster only support access to a single HBase cluster, that is, the clients of the HBase cluster only support configuration of an address of one HBase cluster, and therefore, a request can only be sent to the address of the configured HBase cluster throughout the lifetime. If the configured HBase cluster fails and cannot be served, a user needs to operate the client, the process of the HBase cluster is stopped, the configuration of the HBase cluster is modified, the access address of the HBase cluster points to a new available HBase cluster, the client is restarted, and the client can be connected with the available HBase cluster to start accessing the available HBase cluster.

Therefore, the switching process needs user operation, the time consumption is more than minutes, the client needs to be restarted, and the client cannot work in the restarting process, so that the service is damaged in the restarting process, and the service cannot be normally performed; moreover, the switching decision depends on the judgment of the cluster fault by manpower, the decision time is not fixed, and the fault influence time cannot be estimated. In addition, when a plurality of clients exist, the clients all need to operate one by one. Therefore, the switching method is slow, which results in low processing efficiency for data request and failure to achieve high availability.

Based on this, this specification provides an embodiment of a data request processing method, where a client in this embodiment runs at least two connection tasks, each of the connection tasks is respectively used for maintaining a connection state with a corresponding cluster, as shown in fig. 1B, where fig. 1B is a schematic flowchart of a data request processing method shown in this specification according to an exemplary embodiment, and includes the following steps:

in step 120, creating a first connection session for maintaining a connection with a master cluster and creating a second connection session for maintaining a connection with a slave cluster, the master cluster and the slave cluster being distributed database clusters;

in step 122, configuration information is obtained;

in step 124, if the configuration information indicates that the client preferentially accesses the master cluster, preferentially sending a data request to the master cluster for processing based on the first connection session;

in step 126, if the primary cluster does not return the processing result normally, the data request is concurrently sent to the standby cluster based on the second connection session, so that the standby cluster processes the data request.

The number of clusters in this embodiment may be two or more, that is, one master cluster and at least one slave cluster. In the operation of the client in this embodiment, at least two connection sessions may be created, where the at least two connection sessions include a first connection session and a second connection session, the first connection session is kept connected with the primary cluster, and the second connection session is kept connected with the standby cluster; optionally, in a case that there are multiple standby clusters, the second connection session may be one, that is, only one standby cluster remains connected, and the second connection sessions may also be two or more, and each second connection session remains connected to one standby cluster. That is, the number of clusters in this embodiment is at least two, and the number of connection tasks running in the corresponding client may be more than two. The number of connection tasks may be the same as or less than the number of clusters.

In some examples, the connection session may specifically be implemented by using threads, that is, a client creates at least two connection threads, for example, a first connection thread and a second connection thread run in the client, where the first connection thread is used to create a first connection session that is connected to a master cluster, and the second connection thread is used to create a second connection session that is connected to a slave cluster, that is, each connection thread is used to maintain a connection state with a corresponding cluster.

In some examples, the lifecycles of the first connection thread and the second connection thread may be flexibly configured according to needs, for example, each connection thread may have a different lifecycle, for example, there may be a different creation opportunity and/or a different release opportunity. In other scenarios, optionally, the first connection thread and the second connection thread of this embodiment may keep the same lifecycle, that is, may be created and released simultaneously, for example, if a user needs a function of connecting a cluster, the first connection thread and the second connection thread may be created simultaneously, and when the user does not need to connect the cluster, the first connection thread and the second connection thread may be released simultaneously, so that in a case where the user needs to connect the cluster, high availability of a service may be ensured by creating the first connection thread and the second connection thread simultaneously, and in a case where the user does not need to connect the cluster, the first connection thread and the second connection thread may be released simultaneously, thereby reducing occupation of resources by a connection process.

In this embodiment, the client operates the first connection session and the second connection session, so that the client can quickly and automatically switch to access different clusters, where the client of this embodiment may obtain configuration information, where the configuration information is used to indicate a cluster that the client has a priority to access, and the configuration information may be input by a user through the client or may be automatically generated by the client according to the operation condition of the cluster.

In this embodiment, the client maintains a connection state with the active/standby cluster, and the data request of the client may be sent according to the indication of the configuration information. Taking the case that configuration information is used for indicating a client to preferentially access a master cluster as an example, the configuration information indicates that the client preferentially accesses the master cluster, and preferentially sends a data request to the master cluster for processing based on the first connection session; and the main cluster processes the data request after receiving the data request, and if the main cluster operates normally, the client can receive a normal processing result in time. If the main cluster does not return the processing result normally for some reasons, for example, network jitter or the condition that the main cluster does not work normally, the present embodiment may concurrently send the data request to the standby cluster based on the second connection session, so that the standby cluster processes the data request.

As an example, in some scenarios, after sending the data request to the master cluster, if the master cluster returns a result normally, the data request processing is ended; if an error message returned by the master cluster is received, or a return result of the master cluster to the data request is not received within a set time, it may be determined that the master cluster does not normally return a processing result. Therefore, the data request of this embodiment may be sent to another cluster for processing when the cluster with the priority access fails to return the normal processing result in time. Based on this, the client may receive the normal processing results for the data request respectively returned by the two clusters, in this embodiment, if the normal processing results for the same data request respectively returned by the main cluster and the standby cluster are received, the normal processing result with the later return time is discarded, that is, the fastest returned normal processing result is selected, and the other normal processing result with the slower return time may be discarded, thereby implementing a fast response to the data request.

The condition that the data request needs to be sent to the standby cluster through the connection session corresponding to the standby cluster when the processing result is not normally returned by the main cluster can be flexibly configured as required, and this embodiment does not limit this. As an example, the condition may be set based on a response time of the data request, for example, a set time after the data request is sent does not receive a normal processing result, or may also be determined based on an error message fed back by the cluster, for example, an error message fed back by the cluster to the data request is received; the number of times may also be determined, for example, the number of consecutively received error messages or the number of times of consecutive timeout exceeds a set number of times, and the like, which is not limited in this embodiment.

In some examples, different processing modes may be set according to different feedback of the current cluster. As an example, if the primary cluster does not normally return the processing result, the sending the data request to the standby cluster through a second connection session, so that the standby cluster processes the data request includes:

if the processing of the main cluster is overtime according to the response result of the main cluster to the data request, the data request is sent to the standby cluster through a second connection session, so that the standby cluster processes the data request;

and if the main cluster is determined to have a fault according to the response result, updating the configuration information to that the client accesses the standby cluster preferentially, switching the data request to be sent to the standby cluster preferentially for processing based on the second connection session, and sending the data request to the main cluster for processing through the first connection session.

In this embodiment, after the data request is sent to the master cluster, a response result of the master cluster to the data request may be collected. The present embodiment sets two processing modes for different feedback situations. If the processing of the data request by the main cluster is overtime, the data request can be sent to the standby cluster through the connection task corresponding to the standby cluster, so that the standby cluster processes the data request. Therefore, when the main cluster fails to return a normal processing result in time, the client can quickly send the data request to the standby cluster because the main cluster is connected with the standby cluster, so that the standby cluster processes the data request. The determination of the processing timeout of the data request by the master cluster may be flexibly configured as required, for example, an timeout duration may be set as required, the client performs timing after sending the data request, and the processing timeout of the data request by the current master cluster may be determined if a normal processing result is not received within the set timeout duration.

In another processing mode, the failure of the cluster can be dealt with, and the traffic can be switched from the main cluster to the priority access standby cluster. And if the main cluster is determined to have a fault according to the response result of the main cluster to the data request, updating the configuration information to a client side to preferentially access the standby cluster so as to trigger the execution of the processing flow. The configuration information is updated to enable the client to preferentially access the standby cluster and trigger execution of the processing flow, so that the standby cluster is taken as a cluster with preferential access in the processing flow, and the data request is preferentially sent to the standby cluster, thereby realizing automatic identification of the fault of the cluster and ensuring high availability of the service. The determination of the failure of the main cluster may be flexibly configured according to needs, for example, the number of continuously received error messages or the number of continuously timed-out times exceeds a set number, and the like, which is not limited in this embodiment.

For the above fault scenario, optionally, according to needs, when the master cluster fails and is switched to the standby cluster, whether the master cluster is recovered to be normal or not may be determined, so as to switch back to the standby cluster after the master cluster is recovered to be normal. As an example, after the switching the data request to be preferentially sent to the standby cluster for processing based on the second connection session, the method further includes:

and if the main cluster is determined to be recovered to be normal according to the response result, updating the configuration information to that the client accesses the main cluster preferentially, and switching the data request back to be preferentially sent to the main cluster for processing based on the first connection session.

As an example, when the main cluster fails, the present embodiment switches the traffic to the standby cluster, that is, preferentially sends the data request to the standby cluster; meanwhile, in order to implement the switchback operation, the scheme of this embodiment further sends the data request to the master cluster, so as to determine the time when the master cluster recovers to normal according to the processing condition of the master cluster on the data request. At this time, the data is preferentially sent to the standby cluster and is concurrently sent to the main cluster, namely, the data is sent to the standby cluster and then to the main cluster. If the main cluster fails, the transmitted data request cannot receive the normal processing result. If the master cluster is recovered to be normal, the master cluster can process the sent data request and return a normal processing result, based on the result, after the fact that the master cluster is recovered to be normal can be determined according to a response result of the master cluster to the data request, the configuration information is updated again, the configuration information is updated to enable the client to access the master cluster preferentially, the updated configuration information indicates that the client accesses the master cluster preferentially, and the processing of preferentially sending the data request to the master cluster is triggered to be executed, so that the purpose of switching from the slave cluster to the master cluster is achieved, and the back switching after the fault of the master cluster is recovered is achieved.

In practical application, the cluster may provide services for a plurality of clients of a user, and each client can identify that a cluster which is accessed preferentially at present has a fault by adopting the scheme, so that cluster switching is automatically realized, and the high availability of a data request is improved. In order to further improve the processing efficiency of the data request, in this embodiment, when a certain client finds that a cluster which is currently accessed preferentially fails and needs to be switched, in this embodiment, because the client and another cluster are also in a connected state, other clients can also execute cluster switching, so that under the condition that other clients do not find cluster failures temporarily, cluster switching can be executed in advance, and thus the processing efficiency of the data request is improved. As an example, the method further comprises: and after determining that the main cluster has a fault, sending a cluster switching message to the standby cluster through the second connection session, so that other clients connected to the standby cluster update configuration information to the client to preferentially access the standby cluster after monitoring the cluster switching message issued by the standby cluster.

Taking a cluster which is accessed preferentially at present as a main cluster and a cluster to be switched as a standby cluster as an example, in the scheme of the embodiment, a client sends a cluster switching message to the standby cluster after finding that the main cluster is in fault and needs to be switched, and in the embodiment, the standby cluster can issue the cluster switching message after receiving the cluster switching message of the client; based on the design of the client side in the scheme of the application, a plurality of other client sides are also connected with the standby cluster, the other client sides can monitor the information issued by the standby cluster, and when the information of cluster switching issued by the standby cluster is monitored, each other client side can determine that the main cluster has a fault, so that the other client sides can be switched to the standby cluster in time.

In practical applications, a situation that a plurality of clients simultaneously recognize a failure of a master cluster and send a cluster switching message to a backup cluster may also occur, and in order to prevent a situation that a plurality of clients simultaneously send a cluster switching instruction to a backup cluster, in this embodiment, the sending a cluster switching message to the backup cluster may include: and sending a holding request for an automatic switching lock to the standby cluster, and sending the cluster switching message to the standby cluster after holding the automatic switching lock returned by the standby cluster based on the holding request. That is, the client sends the cluster switching message to the standby cluster after receiving the automatic switching lock returned by the standby cluster.

In this embodiment, after finding that a failure of the master cluster needs to be switched, the client may request the backup cluster to hold an automatic switching lock, where the automatic switching lock can be held by only one client at any time. If the automatic switching lock of the current standby cluster is not held by the client, after a certain client requests to hold the automatic switching lock, the standby cluster can return the automatic switching lock to the client, and the standby cluster can determine that the client needs to be switched to the cluster due to the fault of other clusters based on the holding request of the client to the automatic switching lock, so that the standby cluster can issue cluster switching information, and after monitoring by other clients, the configuration information is updated to the client to preferentially access the standby cluster. The automatic switching lock can be held by only one client at any time, so that the automatic switching lock can be returned to only one client, and the client receiving the automatic switching lock can update the configuration information of the client; and other clients which do not obtain the automatic switching lock can receive the cluster switching message issued by the standby cluster and update the configuration information of the clients according to the cluster switching message issued by the standby cluster.

In the foregoing embodiment, the process of the client actively executing the switching is described, and in this embodiment, the client may also receive a cluster switching instruction triggered by a user. As an example, a cluster generally runs a coordination service, and a user managing the cluster may trigger a cluster switching message through the coordination service and/or a client, and according to the scheme of this embodiment, the user only needs to operate the cluster switching message once, which may trigger other clients connected to the cluster to perform cluster switching. The client of this embodiment may be connected to the coordination service of each cluster, so that it may monitor whether each cluster generates the cluster switching message. In order to ensure high availability of switching, the client and the primary and secondary clusters are both kept in a connected state, so that cluster switching messages on the primary and secondary clusters can be monitored simultaneously. In general, different clusters are respectively deployed in different machine rooms, and by monitoring cluster switching messages of a main cluster and a standby cluster at the same time, it can be guaranteed that the cluster switching messages can still be monitored normally under the condition of single machine room failure. The cluster switching message is used for indicating the cluster which the client preferentially accesses, and after monitoring the cluster switching message, the client can update the configuration information according to the indication of the cluster switching message, so that the subsequent processing flow is triggered according to the configuration information, and the flow is switched to the cluster indicated in the cluster switching message.

In some scenarios, a user may send multiple trunking handover messages in sequence, and due to uncertainty of network communication, a situation that the user monitors after sending the trunking handover message may occur, and for such a situation, the trunking handover message of this embodiment carries a timestamp. Based on this, the cluster switching instruction carrying the latest timestamp can be used as a reference, and whether cluster switching needs to be executed or not is determined according to the timestamp of the cluster switching message.

As an example, the obtaining configuration information includes:

monitoring cluster switching messages which are issued by the main cluster and/or the standby cluster and carry timestamps, wherein the cluster switching messages are used for indicating clusters which are accessed by a client preferentially;

if the timestamp of the cluster switching message is larger than the updating time of the configuration information, updating the configuration information by using the cluster switching message;

and if the timestamp of the cluster switching message is less than the updating time of the configuration information, discarding the cluster switching message.

In this embodiment, after receiving the cluster switching instruction, the client may determine whether to update the configuration information according to the timestamp of the cluster switching instruction, so as to ensure the accuracy of the switching action.

As an example, after receiving the switching message carrying the first timestamp, the client may determine that the first timestamp is newer by comparing the first timestamp with 0 according to the setting that is initially 0 because the switching message is received for the first time, and switch to the second cluster according to the indication of the cluster switching message. And then, the client receives the switching message carrying the second timestamp again, compares the second timestamp with the first timestamp, and because the time of the second timestamp is before the first timestamp, the switching message carrying the second timestamp can be discarded and not executed, and the latest switching message carrying the first timestamp is taken as the reference, so that the switching action is executed according to the latest switching message, and the accuracy of the switching action is guaranteed.

Next, a data request processing method according to an embodiment of the present specification will be described.

The main-standby cluster switching core mainly relates to two problems, namely how to switch and how to make a decision, the key of switching is to establish new connection, and the key of making a decision is to judge whether the main cluster fails.

In this embodiment, two active and standby clusters are taken as an example for explanation, and in practical application, there may be one or more standby clusters as needed. As an example, the client of the present embodiment may include the following three modules: the system comprises a multi-cluster access module, a cluster switching module and an automatic perception module.

A multiple cluster access module (MultiConnection + multiple executor) capable of supporting access to both the primary and backup clusters. The traditional client only supports accessing a single cluster, the link configuration of the client needs to be modified in a fault scene, then the client is restarted to complete switching among the clusters, the process can be completed in tens of minutes, time consumption is positively correlated with the number of the clients, and meanwhile, services cannot be normally accessed in the switching process, so that the services are damaged. The multi-cluster access module of the embodiment mainly solves the problem, supports simultaneous access of the main and standby clusters, switches fault scenes in second level, and has no interruption to transparent services of upper-layer services.

Cluster switching module (SwitchTracker): the cluster switching module mainly solves three difficulties of cluster switching: how to perceive the switching operation, how to ensure the high availability of the switching under the network disconnection scene, and how to recognize the switching information with error expiration so as to ensure the accuracy of the switching action.

Auto-sense module (AutoSwitch): the core function is to accurately judge the fault of the main cluster, then send a switching command to automatically switch the main cluster and the standby cluster, and automatically switch back after the fault is recovered.

The following describes each module in detail:

multiple cluster access module (MultiConnection + multiple executor):

fig. 2A is a schematic diagram of a multi-cluster access module according to an exemplary embodiment shown in this specification, where a client access cluster needs to establish a corresponding Connection, and a plurality of Connection MultiConnection are created in this embodiment; a user view MultiConnection is equal to a Connection, and the connections to the main cluster and the standby cluster are simultaneously and respectively established in the MultiConnection, so that a client can be in a Connection state with both the two clusters, wherein the two connections refer to two processes; optionally, the life cycles of the two connections are identical, i.e. created and released together.

When a client accesses the cluster to acquire data, an actual request is executed through a MultiExecutor when a user accesses the cluster through a MultiConnection;

wherein, the execution process of the MultiExecutor is as follows:

according to the setting, if the current setting is to access the main cluster, the request is preferentially sent to the main cluster; if the main cluster is normal, the client can normally receive the return result of the main cluster, that is, the execution result of the main cluster to the request can be received within normal time.

If the main cluster is not in a normal state, the client cannot normally receive a return result of the main cluster; for example, the client may wait for a certain time to fail to receive the return result of the master cluster, or the master cluster returns an error message, and so on. Based on this, the multifxecutor may access the standby cluster concurrently or switch to the standby cluster.

As an example, in some scenarios, after sending the data request to the master cluster, if the master cluster returns a result normally, the data request processing is ended; if an error message returned by the master cluster is received, or a return result of the master cluster to the data request is not received within a set time, it may be determined that the master cluster cannot process the data request normally. Therefore, when the main cluster cannot return the processing result of the data request in time, the client of this embodiment is also in a connected state with the standby cluster, and thus the data request can be sent to the standby cluster for processing. In this embodiment, after the data request is preferentially sent to the master cluster, the client side also sends the data request to the slave cluster concurrently because the normal feedback of the master cluster cannot be received in time; based on this, after the client side sends the data request to the standby cluster, the client side may receive normal processing results returned by both the main cluster and the standby cluster, and if the normal processing results are received by both the main cluster and the standby cluster, the client side may take any one of the normal processing results as a processing result of the data request; if not, the first received data request may be taken as the processing result of the data request.

If it is determined that the primary cluster fails, the scheme of this embodiment has multi-cluster access capability, and the client is also in a connected state with the secondary cluster in a failure scene, so that the client can automatically and quickly complete switching of cluster access, that is, after it is determined that the primary cluster fails, all data requests are sent to the secondary cluster through Connection of the secondary cluster. Therefore, through the design of the double Connection, in a fault scene, the client of the embodiment can complete cluster switching only by exchanging the roles of the double Connection, the whole process is completed in the second level, and the method is transparent to users, has no perception of service, and does not need user operation; meanwhile, the implementation mode has zero intrusion on the native client codes, and can flexibly support a self-built and public cloud HBase cluster mixed deployment mode by removing the dependence on the version.

As shown in fig. 2B, the processing procedure of the cluster switching module (SwitchTracker) in this embodiment is schematically illustrated, where the cluster switching module in this embodiment is configured to monitor whether a cluster switching message actively sent by a cluster-side user is received; as an example, the cluster of this embodiment is configured with a coordination service, taking an HBase cluster as an example, the coordination service is a Zookeeper, the Zookeeper is a distributed, open-source distributed application program coordination service, that is, an application program that provides a consistency service for a distributed application, the Zookeeper can be considered as a manager of the cluster, and can monitor states of each node in the cluster to perform a next reasonable operation according to feedback submitted by the node, and the provided functions include: configuration maintenance, domain name service, distributed synchronization, group service, etc.

In this embodiment, a user may input a cluster switching instruction through the Zookeeper, and the cluster switching instruction in this embodiment may be received by other clusters after the Zookeeper of any one cluster receives the cluster switching instruction. The cluster switching module may monitor whether the Zookeeper of the master cluster and the Zookeeper of the standby cluster acquire the cluster switching message to sense the switching action. In order to ensure high availability of switching, the cluster switching module may monitor the cluster switching message on the active and standby HBase clusters Zookeeper at the same time because the client maintains a connection state with both the active and standby clusters. In general, the master cluster and the slave cluster are respectively deployed in different machine rooms, and the cluster switching module of this embodiment can ensure that the cluster switching message can still be normally monitored under the condition of a single machine room failure by monitoring the cluster switching messages of the master cluster and the slave cluster at the same time. After monitoring the cluster switching message, the cluster switching module may send the cluster switching message to the multi-cluster access module according to the indication of the cluster switching message, and the multi-cluster access module updates the configuration information according to the cluster switching message, thereby controlling the multi-cluster access module to switch the flow to the cluster indicated in the cluster switching message.

As an example, a user may send multiple trunking handover messages one after another, and due to uncertainty of network communication, a situation may occur that the handover message sent first is heard later, for which case the handover message of this embodiment carries a time stamp. Based on this, the switching message carrying the latest timestamp can be used as a reference, and the timestamp cluster switching module can determine whether the multi-cluster access module is required to perform the switching operation according to the timestamp of the switching message. As an example, after receiving the switching message carrying the first timestamp, the cluster switching module may determine that the first timestamp is newer by comparing the first timestamp with 0 according to the setting that is initially 0 because the switching message is received for the first time, and switch from the standby cluster to the master cluster according to the indication of the switching message. And then, the cluster switching module receives the switching message carrying the second timestamp again, compares the second timestamp with the first timestamp, and because the time of the second timestamp is before the first timestamp, the switching message carrying the second timestamp can be discarded and not executed, and the latest switching message carrying the first timestamp is taken as the reference, so that the switching action is executed according to the latest switching message, and the accuracy of the switching action is guaranteed.

Fig. 2C is a schematic diagram of an auto-perception process of an auto-perception module according to an exemplary embodiment, where the auto-perception module is used to automatically perceive whether cluster switching is required to be performed automatically. As an example, the following situations may occur in a cluster failure scenario: dimension faults of the machine room, such as network disconnection, power failure and the like, cannot be normally linked to the cluster, and the cluster throws all data requests wrongly; or, a full cluster downtime due to the software BUG; or cluster access timeout due to slow, bad disks of servers in the cluster. These fault scenarios are either an access miss of the cluster from the user perspective or an access timeout of the cluster. The automatic sensing module of this embodiment collects the execution results of the data request returned by the MultiExecutor from the currently accessed cluster, and may determine that the currently accessed cluster has a fault and needs to be switched according to a set fault condition (for example, continuously received error messages or continuously overtime times exceed a set number of times).

Taking the case that the main cluster fails and needs to be switched to the standby cluster as an example, after the automatic sensing module determines that the currently accessed cluster has a failure and needs to be switched, the automatic switching lock on the standby cluster Zookeeper can be acquired, the operation can prevent a plurality of clients from sending a plurality of switching messages at the same time, and after the lock is acquired, the automatic sensing module sends the automatic switching message to the multi-cluster access module, so that the multi-cluster access module performs the main-standby cluster switching. After the switching is finished, the automatic sensing module can enter a recovery state, and under the state, the execution result of the MultiExecutor is continuously collected; if the main cluster is recovered to be normal, the original data request sent to the main cluster may be processed, that is, the MultiExecutor may receive the normal execution result returned by the main cluster, and may determine that the main cluster has recovered to be normal according to the set condition (for example, the number of times of continuously receiving the normal execution result exceeds the set number of times, etc.) through the collected normal execution result, at this time, a back-switching operation may be performed, that is, the main cluster is switched back from the currently accessed standby cluster, and the switching operation is as described above, and after the switching is completed, the automatic sensing module may recover to be normal from the recovery state.

As an example, as shown in fig. 2C, the working process of the auto-perception module may include: judging whether the master cluster is in a recovery state or not, and judging whether the master cluster throws errors or overtime if the master cluster is not in the recovery state; if the main cluster is wrongly thrown or overtime, further monitoring to judge whether continuous wrongly throwing/overtime exists, if the continuous wrongly throwing/overtime is larger than the specified times, acquiring a switching lock of the standby cluster can be requested, after the switching lock is acquired, sending a switching message to the multi-cluster access module, enabling the multi-cluster access module to switch from accessing the main cluster to accessing the standby cluster under the indication of the switching message, namely, the data request preferentially accesses the standby cluster, and after the switching is finished, the automatic sensing module can enter a switching state.

The automatic sensing module is in a recovery state, and in the state, the data request preferentially accesses the standby cluster and is also concurrently sent to the main cluster which is judged to be in fault; the automatic sensing module continues to collect the processing result of the main cluster to the data request, if the main cluster starts to recover to normal, the automatic sensing module can start to collect the normal processing result returned by the main cluster, and through the collected normal execution result, according to the set conditions (for example, the number of times of continuously receiving the normal execution result exceeds the set number of times), it can be determined that the main cluster has recovered to normal, at this time, a switch-back operation can be executed, the switch-back operation is consistent with the switch-back operation, a request for acquiring the switch lock of the main cluster can be made, after the switching lock is obtained, a switching instruction is sent to the multi-cluster access module, so that the multi-cluster access module is switched from the standby access cluster to the main access cluster under the instruction of the switching message, namely, the data request accesses the standby cluster preferentially, namely, the standby cluster accessed currently is switched back to the main cluster, and the automatic sensing module can be recovered to a normal state from a recovery state after the switching is completed.

In the scheme of the embodiment, the multi-cluster access module, the cluster switching module and the automatic sensing module are matched with each other, so that the client can sense automatically and perform cluster switching under the fault condition; the method solves the defects that the cluster switching of the client side is time-consuming and is harmful to the service in the original scheme, can realize automatic switching at a second level, has no perception of upper-layer service, ensures the continuous availability of the service to the maximum extent, and enables the availability of the cluster to be a new step.

The scheme of the embodiment can be realized in a plug-in mode, so that zero intrusion of codes of an original client can be realized, a mixed cloud deployment mode can be supported, the client can actively sense to automatically perform cluster switching in a fault scene, multi-cluster access and switching sensing are innovatively utilized, the main/standby switching capability which is not required to be restarted, cannot sense services and can be completed in a second level is realized, the automatic sensing of faults is realized on the basis, and the automatic switching decision capability is provided to minimize the influence of the faults on the availability.

Corresponding to the foregoing embodiments of the data request processing method, embodiments of the present specification also provide embodiments of a data request processing apparatus and a computer device applied thereto.

The embodiments of the data request processing apparatus in the present specification can be applied to a computer device, such as a server or a terminal device. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for operation through the processor in which the file processing is located. From a hardware aspect, as shown in fig. 3, which is a hardware structure diagram of a computer device in which a processing apparatus for data request is located in the present specification, except for the processor 310, the memory 330, the network interface 320, and the nonvolatile memory 340 shown in fig. 3, in an embodiment, a computer device in which the processing apparatus 331 for data request is located may also include other hardware according to an actual function of the computer device, which is not described again.

As shown in fig. 4, fig. 4 is a block diagram of a data request processing device shown in the present specification according to an exemplary embodiment, the device including:

a multi-cluster access module 41 and an auto-aware module 42; wherein the content of the first and second substances,

the multi-cluster access module 41 is configured to: creating a first connection session for maintaining a connection with a master cluster and a second connection session for maintaining a connection with a slave cluster, the master cluster and the slave cluster being distributed database clusters; acquiring configuration information; if the configuration information indicates that the client preferentially accesses a main cluster, preferentially sending a data request to the main cluster for processing based on the first connection session; if the main cluster does not normally return a processing result, the data request is sent to the standby cluster based on a second connection session, so that the standby cluster processes the data request;

the auto-perception module 42 is configured to: and collecting the response result of the main cluster and/or the standby cluster to the data request, and judging whether the main cluster and/or the standby cluster do not return a normal processing result.

In some examples, the client runs a first connection thread and a second connection thread, the first connection thread is used for creating a first connection session connected with the main cluster, and the second connection thread is used for creating a second connection session connected with the standby cluster.

In some examples, the first connection thread and the second connection thread are created simultaneously and released simultaneously.

In some examples, the method further comprises:

and if normal processing results of the same data request, which are respectively returned by the main cluster and the standby cluster, are received, discarding the normal processing results with later return time.

In some examples, if the primary cluster does not normally return the processing result, the sending the data request to the standby cluster through a second connection session concurrently, so that the standby cluster processes the data request, including:

In some examples, after the switching the data request to be preferentially sent to the standby cluster for processing based on the second connection session, the method further comprises:

In some examples, the method further comprises:

and after determining that the main cluster has a fault, sending a cluster switching message to the standby cluster through the second connection session, so that other clients connected to the standby cluster update configuration information to the client to preferentially access the standby cluster after monitoring the cluster switching message issued by the standby cluster.

In some examples, the sending the cluster switching message to the standby cluster includes:

and sending a holding request for an automatic switching lock to the standby cluster, and sending the cluster switching message to the standby cluster after holding the automatic switching lock returned by the standby cluster based on the holding request.

In some examples, the obtaining the configuration information includes:

In some examples, the cluster comprises an HBase cluster.

Correspondingly, the embodiment of the present specification further provides a system for processing a data request, where the system includes at least two clusters and at least one client; at least two connection tasks run in the client, and each connection task is respectively used for keeping a connection state with a corresponding cluster;

the client is configured to:

acquiring configuration information, wherein the configuration information is used for indicating a client to preferentially access a first cluster;

executing the following processing flow according to the configuration information:

taking the cluster preferentially accessed by the client indicated in the configuration information as a target cluster, and preferentially sending a data request to the target cluster for processing through a connection task corresponding to the target cluster;

and if the target cluster does not return a normal processing result, sending the data request to a second cluster through a connection task corresponding to the second cluster so that the second cluster processes the data request.

Correspondingly, the embodiment of the present specification further provides a computer device, including: a processor; a memory for storing processor-executable instructions; wherein the processor executes the executable instructions to implement an embodiment of the data request processing method according to the first aspect.

Accordingly, embodiments of the present specification further provide a computer readable storage medium, on which computer instructions are stored, and the instructions, when executed by a processor, implement the steps in the embodiments of the method for processing a data request according to the foregoing first aspect.

The implementation process of the function and the action of each module in the data request processing apparatus is specifically described in the implementation process of the corresponding step in the data request processing method, and is not described herein again.

For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, wherein the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement it without inventive effort.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.

Other embodiments of the present description will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the specification following, in general, the principles of the specification and including such departures from the present disclosure as come within known or customary practice within the art to which the specification pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It will be understood that the present description is not limited to the precise arrangements described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims

1. A method for processing data requests, which is applied to a client, comprises the following steps:

acquiring configuration information;

2. The method of claim 1, wherein a first connection thread and a second connection thread run in the client, the first connection thread is used for creating a first connection session connected with a main cluster, and the second connection thread is used for creating a second connection session connected with a standby cluster.

3. The method of claim 2, the first connection thread and the second connection thread being created and released simultaneously.

4. The method of claim 1, further comprising:

5. The method according to claim 1, wherein if the primary cluster does not normally return the processing result, the data request is concurrently sent to the standby cluster through a second connection session, so that the standby cluster processes the data request, including:

6. The method of claim 5, after the switching the data request to be preferentially sent to the standby cluster for processing based on the second connection session, further comprising:

7. The method of claim 1, further comprising:

8. The method of claim 7, the sending a cluster switch message to the standby cluster, comprising:

9. The method of claim 1, the obtaining configuration information, comprising:

10. The method of claim 1, the cluster comprising an HBase cluster.

11. An apparatus for processing a data request, the apparatus being applied to a client, the apparatus comprising:

12. A data request processing system comprises a main cluster and a standby cluster, wherein the main cluster and the standby cluster are distributed database clusters; the system further comprises at least one client; the client is configured to:

acquiring configuration information;

13. A computer device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor implements the method of any one of claims 1 to 10 by executing the executable instructions.

14. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 10.