CN114625489A

CN114625489A - Access request response method and device and distributed system

Info

Publication number: CN114625489A
Application number: CN202210296635.5A
Authority: CN
Inventors: 罗剑明; 严祥光; 安凯歌
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-03-24
Filing date: 2022-03-24
Publication date: 2022-06-14

Abstract

The embodiment of the specification provides an access request response method, an access request response device and a distributed system, wherein the access request response method comprises the following steps: when disconnection with a main node is detected, stopping writing data in a data log into an application queue, wherein the data log records data to be subjected to node consistency judgment, and the application queue records data meeting the node consistency judgment condition; matching the access request in a waiting queue with the data in the application queue, and keeping the connection with a first client corresponding to a first access request which is successfully matched, wherein the waiting queue records the access request to be responded; and responding to the first access request, and returning a response result to the first client. The scheme can ensure that the usability of the distributed system is higher when the distributed system is disconnected with the main node.

Description

Access request response method and device and distributed system

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to an access request response method.

Background

With the increasing demands for processing efficiency, data disaster tolerance, reliability, and the like, distributed systems are widely used.

In the related art, a user can send an access request through a client connected with a slave node of a distributed system, a master node in the distributed system initiates consistency processing for target data corresponding to the access request by using a consistency protocol, and the target data is stored under the condition that the target data meets the consistency condition, so that the consistency of the data in the distributed system is ensured. In a particular application, the master node is likely to switch: the current master node is unavailable due to factors such as downtime of equipment or version update, and a new master node needs to be elected. At this time, in order to prevent consistency abnormality caused by unavailability of the master node, each slave node disconnects all connected clients and clears the pending access request, so that the client retransmits the access request to the distributed system.

However, after the disconnection, in order to reconnect to send the access request, the client needs to send a large number of reconnection requests to the distributed system, which easily causes the problem that the performance of the distributed system is greatly reduced, and even the distributed system is not available after breakdown. Therefore, there is a need to provide a more highly available solution.

Disclosure of Invention

In view of this, the present specification provides an access request response method. One or more embodiments of the present specification also relate to an access request responding apparatus, a distributed system, a computing device, a computer-readable storage medium, and a computer program, so as to solve the technical deficiencies of the prior art.

According to a first aspect of the embodiments of the present specification, there is provided an access request response method applied to a slave node of a distributed system, including:

when disconnection with a main node is detected, stopping writing data in a data log into an application queue, wherein the data log records data to be subjected to node consistency judgment, and the application queue records data meeting the node consistency judgment condition;

matching the access request in the waiting queue with the data in the application queue, and keeping the connection with the first client corresponding to the first access request which is successfully matched, wherein the waiting queue records the access request to be responded;

and responding to the first access request, and returning a response result to the first client.

According to a second aspect of the embodiments of the present specification, there is provided an access request responding apparatus applied to a slave node of a distributed system, including:

the system comprises a stopping module and an application queue, wherein the stopping module is configured to stop writing data in a data log into the application queue when disconnection with a main node is detected, the data log records data to be subjected to node consistency judgment, and the application queue records data meeting a node consistency judgment condition;

the matching module is configured to match the access requests in the waiting queue with the data in the application queue and maintain the connection of a first client corresponding to a first access request successfully matched, wherein the waiting queue records the access requests to be responded;

a response module configured to respond to the first access request and return a response result to the first client.

According to a third aspect of embodiments herein, there is provided a distributed system comprising:

the system comprises slave nodes, a master node and a client;

the slave node is configured to stop writing data in a data log into an application queue when disconnection with the master node is detected, wherein the data log records data to be subjected to node consistency judgment, and the application queue records data meeting a node consistency judgment condition; matching the access request in the waiting queue with the data in the application queue, and keeping the connection with the first client corresponding to the first access request which is successfully matched, wherein the waiting queue records the access request to be responded; responding to the first access request, and returning a response result to the first client;

the client is configured to receive the response result.

According to a fourth aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions, which when executed by the processor implement the steps of the above-described access request response method.

According to a fifth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described access request response method.

According to a sixth aspect of embodiments herein, there is provided a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of the above-described access request response method.

One embodiment of the present specification implements that when detecting disconnection with a master node, a slave node of a distributed system stops writing data in a data log into an application queue, where the data log records data to be subjected to node consistency judgment, and the application queue records data meeting a node consistency judgment condition; matching the access request in a waiting queue with the data in the application queue, and keeping the connection with a first client corresponding to a first access request which is successfully matched, wherein the waiting queue records the access request to be responded; and responding to the first access request, and returning a response result to the first client.

Therefore, when the master node is disconnected, the data in the data log is stopped being written into the application queue, and the problem of consistency abnormity caused by the fact that data which is uncertain whether the master node judges the consistency of the nodes is fed back to the client side or not under the condition that the master node is disconnected is solved. On the basis, normal feedback can be performed on the data written into the application queue, so that the connection between the first clients corresponding to the first access requests matched with the data in the application queue can be maintained, the first access is responded, and the return result is returned to the first clients. Therefore, consistency abnormity caused by disconnection with a main node and response abnormity to each access request can be avoided under the condition that connection of all clients is not required to be disconnected, so that distributed system pressure caused by a large number of reconnection after all clients are disconnected is greatly reduced, and the working performance of the distributed system is ensured. Therefore, the scheme can ensure that the availability of the distributed system is higher when the distributed system is disconnected with the main node.

Drawings

FIG. 1 is a schematic diagram of a service framework of a distributed system provided by one embodiment of the present description;

fig. 2 is a flowchart of an access request response method according to an embodiment of the present specification;

FIG. 3 is a schematic diagram illustrating an architecture of a distributed system in a multi-tiered asynchronous queue scenario provided by an embodiment of the present disclosure;

fig. 4 is an exemplary diagram of states of asynchronous queues after a master node and a slave node are disconnected in a multi-layer asynchronous queue scenario provided in an embodiment of the present specification;

FIG. 5 is a diagram illustrating an example of an application result of an access request response method in a multi-layer asynchronous queue scenario according to an embodiment of the present specification;

fig. 6 is a diagram illustrating an example of a processing procedure of an access request in a scenario where a role type of a slave node is a master node type in an access request response method provided in an embodiment of the present specification;

fig. 7 is a schematic structural diagram of an access request response apparatus according to an embodiment of the present specification;

fig. 8 is a schematic structural diagram of a distributed system according to an embodiment of the present specification;

fig. 9 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

First, the noun terms referred to in one or more embodiments of the present specification are explained.

And (3) Quorum: each server set of the distributed system is generally odd in number of elements in the set. The Server is a node in the distributed system, and each Server maintains a memory database of the distributed system, and persistently stored transaction logs and snapshot data. The Quorum may also be referred to as a node set of the distributed system.

Master node (Leader): the main service end elected by at least more than half of the service ends in the node set of the distributed system is responsible for proposing a transaction operation (proxy) and synchronizing data aimed by the transaction operation to the service ends in the distributed system. The transaction operation is a "write" or the like, which can change the data consistency in the distributed system, that is, an operation that can change the state of the distributed system. In a specific application, the operation may be in the form of an access request, for example, the write operation may be a write-type access request.

Slave node (Follower): the slave server side in the node set of the distributed system directly responds to non-transaction operations such as reading of the user, transmits the transaction operations such as writing of the user to the master node for processing, receives a Proposal (Proposal) from the master node, correspondingly updates the local transaction log and mirror image of the slave node, and modifies the state of the memory database.

The consistency protocol is as follows: the method is used for maintaining the consistency of the data stored by each node in the distributed system, namely, the data stored by each node in the distributed system is the same. In this specification refers to a coherence protocol for a distributed system where there is one master node.

State Machine (State Machine): the status record of the in-memory database is generally obtained by performing structured storage on log data in a memory.

Failover (failover): when an active service or application terminates unexpectedly, a redundant or standby server, system, hardware, or network is quickly enabled to take over the active service or application for work.

In a specific application, when a master node is disconnected from a slave node, for example, when the master node is down, the software version of the master node is updated, and a network between the master node and the slave node is unavailable, the master node needs to be switched, and the master node is unavailable at this time. And the master node is responsible for transactional operation, initiates system consistency judgment on data targeted by the transactional operation, and can ensure the consistency of the data in the distributed system. Wherein, the judgment of the node consistency is as follows: the master node initiates a consistency judgment, namely proposal, for the target data aimed at by the access request to each slave node according to the consistency protocol. In this proposal, the master synchronizes the target data, i.e. the resolution, to each slave node, which checks the received target data: and judging whether the target data meets a preset check condition, recording the target data meeting the preset check condition in a data log, and sending a feedback result of whether to agree to a resolution to the master node. Wherein for each resolution sent by the master node, the slave node typically agrees to the resolution and does not agree unless the slave node lags behind or is abnormal. On the basis, when the master node determines to obtain the agreement of most slave nodes on the resolution based on the received feedback result, the master node sends a submission request of the resolution to each slave node agreeing to the resolution, and the slave nodes write the target data in the data log into the application queue according to the received submission request. Thus, the application queue records data that satisfies node consistency: the recorded data is the same in each node of the distributed system. Therefore, in order to prevent consistency abnormity caused by unavailability of the master node, the connection between each slave node and the client in the distributed system is disconnected, so that the client reconnection is triggered. Therefore, with hundreds of thousands of connections, the server is susceptible to avalanche access pressure induced collapse: the use cannot be recovered. To this end, a layer of Proxy (Proxy) may be added between the client and the distributed system for forwarding client requests. Therefore, the agent can stop interaction with the client when the main node is switched, all the clients do not need to be disconnected, and connection reconnection caused by main node switching can be reduced. However, the problem that nodes of a distributed system accessed by agents are inconsistent before and after the master node is switched due to failover easily occurs in this way, and the consistency of the order of the clients cannot be guaranteed in a scenario where the clients have order consistency requirements on access requests.

In addition, the disconnection easily causes the problem that the client who robs the distributed lock loses the lock. Wherein, the distributed lock is: a distributed coordination technique for scheduling a plurality of processes in order to prevent interference between the plurality of processes in a distributed system. The core idea of the distributed lock is as follows: in a distributed system, a method can only be executed by one thread of one machine at a time. Thus, the client that robs the distributed lock is the one machine or one thread.

As shown in fig. 1, fig. 1 is a schematic diagram illustrating a service framework of a distributed system provided by an embodiment of the present specification. The method specifically comprises the following steps: what guarantees the consistency of distributed data in the distributed system is a set of servers called Quorum, and the elements in the set are nodes in the distributed system, namely servers. When the distributed system is started, according to the current record information of each server, the distributed system selects one server as a master node, and the rest servers as slave nodes. In a specific application, more than half of the slave nodes participate in the selection of the master node. Each node includes a master node that can receive multiple access requests sent by a user through a client. In one case, the same client may require that there is order consistency among multiple access requests of the client, that is, the client sends the access requests to the nodes of the distributed system in order, and the nodes of the distributed system feed back response results of the access requests to the client in the order.

In view of the above situation, in the present specification, an access request responding method is provided, and the present specification relates to an access request responding apparatus, a distributed system, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

Referring to fig. 2, fig. 2 is a flowchart illustrating an access request response method according to an embodiment of the present specification, applied to a slave node of a distributed system, and specifically including the following steps:

s202, when the disconnection with the main node is detected, the data in the data log is stopped being written into the application queue, wherein the data log records the data to be subjected to the node consistency judgment, and the application queue records the data meeting the node consistency judgment condition.

In a specific application, the master node is disconnected from the slave nodes, for example, the master node is down, the software version of the master node is updated, and the network between the master node and the slave nodes is unavailable. For example, detecting disconnection from the master node may specifically include: and the slave node sends a heartbeat detection packet to the master node, and receives feedback of the master node when the heartbeat detection packet is not in a preset time length or does not reach a sending frequency threshold value. Or, for example, detecting disconnection from the master node may specifically include: and the slave node receives the notification that the master node updates the software version. Any way of detecting disconnection from the master node can be used in this specification, and this embodiment is not limited thereto. The node consistency judgment condition comprises the following steps: the data stored in each node in the distributed system is the same. Therefore, the node consistency determination condition may be specifically a determination condition that satisfies consistency and is specified by a consistency protocol.

And in the process of judging the consistency of the nodes, the master node initiates the consistency judgment of the target data aimed at by the access request to each slave node according to the consistency protocol, namely proposing. In this proposal, the master synchronizes the target data to each slave node, and each slave node verifies the received target data: judging whether the target data meet preset verification conditions or not, recording the target data in a data log, and feeding back a verification result to the main node; and the master node obtains a judgment result of whether the target data meets the node consistency or not based on the resolution result fed back by each slave node, sends the judgment result to each slave node, and the slave node performs processing of writing or not writing the target data in the data log into the application queue according to the received judgment result. Therefore, the data log records data to be subjected to node consistency judgment. In this way, the node consistency determination is completed by determining whether the data targeted by the access request meets the consistency determination condition. Accordingly, the application queue records data meeting the node consistency judgment condition, that is, the application queue records data meeting the node consistency.

Illustratively, under the condition that the master node and the slave node are normally connected, in order to persistently store data meeting the node consistency judgment condition, the slave node calls a consistency protocol module to complete the node consistency judgment, and obtains that the data recorded in the data log meets the node consistency condition, so that the consistency protocol module pushes the data targeted by the access request to the application queue. And under the condition that the master node is disconnected from the slave nodes, the master node cannot feed back information to the slave nodes, and the slave nodes cannot obtain the node consistency judgment result of the data recorded in the data logs. Then, whether the data recorded in the data log completes the above-mentioned node consistency determination is in a Fuzzy (Fuzzy) state for the slave node: may or may not be complete or may be partially complete. Therefore, when the data is disconnected, the data in the data log is stopped being written into the application queue, that is, the consistency protocol module stops pushing the data for which the access request is directed to the application queue, so that the problem that the consistency is abnormal because the data determined based on the consistency of the incomplete nodes is responded to the access request corresponding to the data and the response result is fed back to the client can be prevented. In addition, in consideration of the fuzzy state, before the distributed system elects a new master node, the data recorded in the data log can be regarded as the data to be subjected to the node consistency judgment.

S204, matching the access request in the waiting queue with the data in the application queue, and keeping the connection with the first client corresponding to the first access request successfully matched, wherein the waiting queue records the access request to be responded.

In a particular application, the wait queue records access requests to be responded to. Therefore, the access requests in the waiting queue are matched with the data in the application queue, and the data targeted by the first access request which is successfully matched is the data which is judged by the node consistency and meets the judgment condition of the node consistency. Therefore, the first access request can be responded, and the connection of the first client corresponding to the successfully matched first access request can be kept without disconnection under the condition of disconnection with the main node. And for the connection of the second client corresponding to the second access request which is not successfully matched with the application queue, the processing mode can be disconnection or maintenance. In one case, after the connection with the second client is disconnected, the client reconnection may be triggered, so as to resend the second access request, and the slave node may respond to the second access request when a new master node is elected. At this time, the disconnection with the second client can avoid the problem that the user can not respond in time to send a new access request to the second client because of waiting for election of a new host node, thereby reducing the negative influence of host node disconnection on user experience.

In an optional implementation manner, after the matching of the access request in the wait queue and the data in the application queue is performed, the access request responding method provided in the embodiment of the present specification may further include the following steps: and maintaining the connection of the second client corresponding to the second access request which is not matched successfully. In the present embodiment, since the writing of the data in the data log into the application queue is already stopped by the above step S202, the data in the application queue are all the data meeting the consistency condition. Therefore, when the connection with the second client is maintained, the second access request which is not successfully matched is not responded, and the consistency abnormity is not possible to occur. Therefore, the embodiment can ensure that all the clients do not need to be disconnected under the condition that the main node is disconnected, and the availability of the distributed system is higher. On this basis, under the condition that a new master node is elected, the slave node can respond to the second access request again, and therefore the node consistency judgment can be performed on the data aimed by the second access request again. In one case, different types of access requests have different consistency requirements, and therefore, the step S204 may specifically include identifying the type of the access request, and then performing different responses to the different types of access requests. Accordingly, different processing may be performed for connections between clients corresponding to different types of access requests. Alternative embodiments are described in detail below to facilitate understanding and reasonable layout.

S206, responding to the first access request, and returning a response result to the first client.

In a specific application, responding to the first access request means obtaining a response result of the first access request based on data successfully matched with the first access request in the application queue. For example, if the first access request is an operation of reading data, the data successfully matched may be used as a response result; if the first access request is an operation of writing data, it is reasonable to use information such as that data successfully matched has been written as a response result. The specific response mode may be adapted to the application scenario, which is not limited in this embodiment. In addition, in a scenario where there is a requirement for consistency in the order of access requests at the client, in response to the requirement for meeting the requirement, for convenience of understanding and reasonable layout, this is specifically described in the form of an optional embodiment. In addition, after a new master node is elected, the role type of the slave node may have different response modes to the access request received by the slave node, and for convenience of understanding and reasonable layout, this is specifically described in the form of an alternative embodiment in the following.

In an embodiment of the present specification, when a slave node of a distributed system is disconnected from a master node, data in a data log is stopped being written into an application queue, so as to avoid a problem of consistency abnormality caused by feeding back data that is uncertain whether a master node performs a node consistency judgment to a client when the master node is disconnected. On the basis, normal feedback can be performed on the data written into the application queue, so that the connection between the first clients corresponding to the first access requests matched with the data in the application queue can be maintained, the first access is responded, and the return result is returned to the first clients. Therefore, consistency abnormity caused by disconnection with a main node and response abnormity to each access request can be avoided under the condition that connection of all clients is not required to be disconnected, so that distributed system pressure caused by a large number of reconnection after all clients are disconnected is greatly reduced, and the working performance of the distributed system is ensured. Therefore, the scheme can ensure that the availability of the distributed system is higher when the distributed system is disconnected with the main node.

In an optional implementation manner, the matching of the access request in the wait queue and the data in the application queue may specifically include the following steps:

identifying the request type of the access request in the waiting queue, and matching the access request with the data in the application queue, wherein the request type of the access request is a write type;

accordingly, the responding to the first access request may specifically include the following steps:

and obtaining a response result of the access request of the write type based on the data successfully matched with the access request of the write type.

Wherein the request type of the access request is used for characterizing the operation of the access request on the data. Illustratively, the request type of the access request may include: read type, write type, etc. In a specific application, the access request may include an identifier representing a request type (type), such as an identifier "read" for a read type and an identifier "write" for a write type. Thus, the slave node may identify the request type of the access request in the wait queue by reading that the access request in the wait queue contains a request type identification. Since a write-type access request, that is, a write request, etc., may change the state of data in the distributed system, the consistency requirement of the write request is a requirement for node consistency. This means that the data for which the write request is directed is recorded in the application queue if the coherency requirement is met. Therefore, the matching with the data in the application queue can be carried out on the access request with the request type being the write type. In this way, the embodiment matches the access request with the application queue, which requires node consistency, so that redundant matching of queues which do not need to be matched can be reduced, and a more efficient effect of the access request is achieved.

Moreover, obtaining a response result of the write-type access request based on the data successfully matched with the write-type access request may specifically include: generating information that the successfully matched data is written in to obtain a response result; or acquiring the storage position of the successfully matched data, generating feedback information containing the storage position, and obtaining a response result. This is reasonable, and the response can be specifically performed according to the application requirement, which is not limited in this embodiment.

In an optional implementation manner, after the data in the data log is stopped from being written into the application queue, the access request response method provided in the embodiment of the present invention may further include the following steps:

maintaining the connection with a third client corresponding to a third access request, wherein the third access request is an access request with a read type request type;

and reading data matched with the third access request in the data locally stored in the slave node, obtaining a response result of the read-type access request, and returning the response result to the third client.

In a particular application, a read-type access request, i.e., a read request, does not change the state of data in the distributed system, and therefore, there is no need for node coherency for read requests. Therefore, in this embodiment, the access request of the read type may directly read data matching the third access request from the data locally stored in the node. For example, data matching the third access request is read from the state machine, and a response result of the read-type access request is obtained. For example, the read data may be directly used as the response result, or the read data may be encapsulated and the like to obtain the response result of the read-type access request. Accordingly, at this time, the third client corresponding to the access request of the read type can be kept connected to normally feed back the response result, so that the number of clients needing to be disconnected under the condition of disconnection from the main contact is further reduced, and the effect of higher availability of the distributed system is achieved. Moreover, the third client is usually the client which robs the distributed lock, so the embodiment can also reduce the disconnection of the client which robs the distributed lock to the greatest extent, thereby avoiding the risk of losing the lock.

In an optional implementation manner, after the matching between the access request with the write type as the request type and the data in the application queue is performed, the access request response method provided in the embodiment of the present specification may further include the following steps:

if the matching fails, polling the access requests in the waiting queue from the access request of the current write type, and adding the client corresponding to each access request of the write type into a client removal set;

and disconnecting the connection between the client contained in the client removal set and the client.

In a specific application, for a write request, if corresponding data can be found in an application Queue (Apply Queue), that is, matching is successful, the data can be applied to a state machine (StateMachine), that is, the data is stored in the state machine, so that persistent storage is realized. In this way, a read request can be obtained directly from the state machine when reading the data. If the match fails, processing may be suspended for the pending queue. Moreover, the access requests in the queue are usually processed according to a first-in first-out rule, so that the failure of the first matching indicates that no subsequent write request is added to the application queue, and the matching inevitably fails. Therefore, at this time, the access requests in the waiting queue may be polled from the current write request, and the client corresponding to each access request of the write type may be added to the client removal set. Therefore, in this embodiment, from the first matching failure, the matching between the wait queue and the application queue is not performed, and the client corresponding to each access request of the write type is directly polled and removed, so that the time consumed by matching can be reduced, and the effect of higher response efficiency of the access request is achieved.

The client corresponding to each access request of the write type is the second client, that is, the second client is added to the client removal set, for example, the client sessionID of the second client is added to the client removal set (removecciientset). In this way, the present embodiment may enable disconnection of the connection with the second client.

In an optional implementation manner, any client has a requirement on consistency of the access request sequence, request identifiers of access requests of the same client are set according to the sequence, and data requested by the access requests correspond to the request identifiers; the number of the first access requests is multiple;

obtaining a first access request queue containing the first access requests based on the sequence of the request identifiers of the first access requests;

according to the first-in first-out principle, performing request identification matching on each first access request in the first access request queue and data in the application queue, and obtaining a response result queue of each first access request based on successfully matched data;

correspondingly, the returning of the response result to the first client specifically includes the following steps:

and returning the response result in the response result queue to the first client according to a first-in first-out principle.

In a specific application, the requirement that any client has consistency on the order of access requests refers to that any client sends access requests to nodes of a distributed system in sequence, and the nodes of the distributed system feed back response results of the access requests to the client according to the sequence. In this way, the request identifiers of the access requests which are set in sequence can realize the sequential feedback of the response, and the access requests form a queue in sequence for processing, so that the requirement of the client on the sequential consistency is met more reasonably.

The following description will further describe the access request response method by taking an application of the access request response method provided in this specification in a multi-layer asynchronous queue scenario as an example, with reference to fig. 3 and fig. 4. Fig. 3 is a schematic diagram illustrating an architecture of a distributed system in a multi-layer asynchronous queue scenario according to an embodiment of the present disclosure.

For example, in the distributed system shown in fig. 3, the Client: the user may send an access request through the client. Copy Replica: the master node synchronizes transaction requests, such as write requests, to the copy of the consistency protocol module Consensus for node consistency determination by proposing to send the transaction requests to the consistency protocol module Consensus. The copy is in particular a copy of the data for which the write request proposed by the primary node is directed. The consistency protocol module Consensus is used for realizing data collaboration among the nodes, that is, can be used for finishing node consistency judgment, and can be specifically realized according to consistency protocols such as Raft, Paxos and the like. The method comprises the steps of obtaining a master node selection rule, obtaining a log copy rule, and obtaining a safety rule. Paxos is a coherence protocol that addresses how individual processes in a distributed system agree on a certain value (resolution). In this specification, the above solution requires that each node is recorded in a data log, and waits for the master node to determine whether consistent data is satisfied. Log module Log: and the data used for judging the consistency of the nodes, namely the resolution, is recorded in the data log. State Machine: and the data in the data log finishes the consistency judgment of the nodes, is added to the application queue under the condition of consistency, is added to the state machine under the condition that the data in the application queue is successfully matched with the access request in the waiting queue, and can directly acquire corresponding data from the state machine for non-transaction requests such as read requests and the like. As shown in fig. 3, the above modules are applicable to nodes of distributed systems, in which proposal by the master node cannot be made by the slave node, and therefore, the slave node does not perform the steps of initiating a consensus proposal and the above Replica.

In the case that disconnection from the master node is not detected, taking the slave node receiving an access request as an example, the processing flow of the access request by the distributed system shown in fig. 3 includes the following steps:

request: the user requests access by sending it to the slave node. Put: the access requests are put at the end of a Request queue (Request queue), and the access requests in the Request queue are processed in a first-in first-out mode one by one. ③ Forward Request: if the access request processed currently is a transaction operation such as a write request, the access request is forwarded to the main node for processing, and the access request (Put) is placed at the end of a waiting Queue (Pending Queue); if the access request currently processed is a non-transaction access request such as read, the access request is directly put at the end of the waiting queue.

Fourthly, Propose: when receiving an access request forwarded from a node, a master node places the access request into a proposal queue (for simplicity, the queue is not shown in fig. 3), and verifies the access request in the proposal queue before submitting the access request, and if the verification is successful, submits data targeted by the access request to a consistency protocol module (Consensus); wherein, the verifying may include: and checking the legality of the request, such as whether the session of the client connection is expired or not, whether the access request has the right to operate the target path or not and the like. Consensus Protocol: and the consistency Protocol module judges the consistency of the nodes according to a consistency Protocol Consensus so as to obtain a resolution which can be recorded in the data log. Sixthly, appendix: when a resolution is obtained based on the data for which the access request is directed, this data can be persistently stored in the data Log of the Log module Log. And informing the main node of the event that the data is submitted so that the main node can confirm whether the data meets the node consistency judgment condition in each node.

Seventy On Apply: and when the node consistency judgment condition is met, the data in the data log is put at the end of an application Queue (Apply Queue). (viii) Match & Apply: the data in the application queue is matched with the access request in the waiting queue (Pending queue), and if the matching is successful, the data is corresponding to the access request sent by the client on the node. Thus, the successfully matched data may be applied to, i.e., stored in, the state machine, and a Response result may be returned to the client in Response to a corresponding access request based on the data. In addition, the numbers in each queue and log module in fig. 3 are the identifiers of the access requests.

For the distributed system shown in fig. 3, when the master node goes down or upgrades the service, the slave node is disconnected from the master node. In contrast, as shown in fig. 4, in a multi-layer asynchronous queue scenario provided in an embodiment of the present specification, an exemplary diagram of states of each asynchronous queue after a master node is disconnected from a slave node is shown:

when the slave node is disconnected from the master node in the distributed system shown in fig. 3, unlike the processing manner of disconnecting all the clients, in this embodiment, the connection of all the clients is not disconnected, and the threads of the asynchronous queues suspend processing the access requests in the respective queues, and the consistency protocol module (Consensus) is called to stop putting the data targeted by the access requests into the application Queue (Apply Queue), instead of emptying the data in the Request Queue and the wait Queue pending Queue. At this time, the slave node enters a Fuzzy state. Referring to fig. 4, each access request contains the following fields: clientID, sessionID, and requestID. Wherein, clientID: for each client, the server assigns a unique sessionID to the client for marking the client. request ID: each access request of the client also has a requestID which represents the access request sequence, the integer is increased from 1, and for a system which requires the client sequence consistency, the server needs to ensure that the access request response is returned to the client according to the sequence of the requestIDs. type: the types of access requests are mainly divided into two main categories, read access requests (read) and write access requests (write). The read access request is weakly consistent read, data in a corresponding node state machine is directly read without entering a consistency protocol, and the order consistency of the client side is ensured. The write access request needs to be submitted through a Leader, and is synchronized to the nodes with the number larger than the number threshold value through a consistency protocol before being applied to the state machine.

In the case of the slave node entering the Fuzzy state, as shown in fig. 4: processing the data in each asynchronous queue according to an uncertain access request removing mechanism and an access request processing maximum response principle: all access requests in the waiting Queue (Pending Queue), here the access request handling maximum response principle is introduced: according to the first-in first-out processing, for a read access request, directly reading data in a state machine, and returning a response result corresponding to the read data to a client; for write access requests, if the corresponding access request can be found in the application Queue (Apply Queue), then the state machine (statecache) is applied. If the corresponding access request can not be found in the application Queue (Apply Queue), processing according to an uncertain access request removing mechanism: waiting for queue suspension processing; and polling the access requests in the queue from the current write access request, and adding the session ids of all the write access requests into a client removal set (RemoveClientSet), as shown in fig. 4, where the element in the client set of the waiting queue without a corresponding write access request is session 3. This is an unknown state because the client removes the access request in the set, which may have been sent to the original Leader, but the Leader may not have submitted or has submitted to the consistency protocol module. If not, the wait queue may be caused to wait for the access request to correspond and be in a wait state. Thus, client connections in the remove client set (RemoveClientSet) may be closed. And finding all access requests corresponding to the session in the client removal set in the access Request Queue (Request Queue) and the waiting Queue (Pending Queue) for removal, so as to prevent repeated access requests after the removed clients are reconnected.

When the removal of the access request is completed, the waiting Queue (Pending Queue) only remains the read access request which is not removed, and the read access request can directly read the state machine data without judging the node consistency. For data in an application Queue (Apply Queue), according to a normal processing mode, all data in the application Queue find a corresponding access request in a waiting Queue (Pending Queue) according to a first-in first-out mode, and Apply the corresponding access request to a state machine (statecache). And after the access Request queue (Request queue) removes all the access requests for removing the set session, the access Request of the client is normally received and put at the tail of the queue until the consistency protocol module processes the access Request under the condition of electing a new main node. And for the client which is not removing the set, the connection is kept normal, and the access request is continuously sent.

Based on the processing of the above-mentioned embodiment of fig. 4, as shown in fig. 5, an exemplary graph of an application result of an access request response method in a multi-layer asynchronous queue scenario is provided in an embodiment of this specification: the slave node (Follower) at least keeps the client connection in a determined state by introducing a Fuzzy mode, and normally receives the access request, so that the problem that the connection of all the clients is disconnected when the slave node is disconnected with the master node is solved, and the problem that the service is unavailable due to the fact that the server is involved in processing a large number of reconnection requests is solved. As shown in fig. 5, TCP connections are maintained for clients client1, client2, and client4, so that the clients 1, client2, and client4 can normally obtain response results. Wherein, TCP is a Transmission Control Protocol, which is called Transmission Control Protocol in english, and is a connection-oriented, reliable, byte stream-based transport layer communication Protocol. Furthermore, according to the characteristics of the client of the distributed lock, when the distributed lock is not released, only a heartbeat access request (read access request) is generally sent to maintain the validity of the distributed lock. Therefore, by introducing an uncertain access request removal mechanism and an access request processing maximum response principle, disconnection of a client robbed to the distributed lock can be reduced to the maximum extent, and therefore the risk of losing the lock is avoided.

In an optional implementation manner, after detecting that the master node is disconnected from the access request, the access request response method provided in this embodiment may further include the following steps:

when detecting that a new master node is elected, identifying the role type of the slave node;

and determining a checking mode corresponding to the role type, and checking the access request in the waiting queue according to the checking mode.

In a particular application, the role types of the nodes in the distributed system may characterize the differences in the functions of the nodes. And, differences in node functionality in the distributed system now check access requests differently. Therefore, after the master node is switched, if the slave nodes have different roles, the request checking mode is different. Therefore, the problem of abnormal verification when the node verifies the access request according to the role type before the new master node is elected can be avoided, and the effect of smoother master node switching is realized.

For example, the detecting of electing a new master node may specifically include: and receiving a notice of completion of the election of the main node sent by the consistency protocol module. Identifying the role type of the slave node may specifically include: the notification of the completion of the master node election may carry a node identifier of the master node, and the slave node may identify whether the node identifier is a node identifier of the slave node itself, if so, the role type of the slave node is the master node type, and if not, the role type of the slave node is the slave node type. Moreover, determining the verification method corresponding to the role type may specifically include: and searching a corresponding relation between the pre-established role type and the checking mode to obtain the checking mode corresponding to the role type of the slave node.

In an optional implementation manner, the verifying the access request in the waiting queue according to the verification manner may specifically include the following steps:

and if the role type of the slave node is the master node type, adding the access request in the waiting queue to the queue to be checked, and checking the access request in the queue to be checked according to a first-in first-out principle.

In a particular application, the verification may include: and checking the legality of the request, such as whether the session of the client connection is expired or not, whether the access request has the right to operate the target path or not and the like. Exemplarily, with reference to the description of the embodiment of fig. 4, as shown in fig. 6, in a scenario that a role type of a slave node is a master node type in an access request response method provided by an embodiment of this specification, an exemplary processing procedure of an access request is shown in fig.: in the case that the role of the original Follower is changed into the role of a Leader, that is, a slave node- > a master node, in this case, the access requests in the access Request Queue (Request Queue) are sequentially inserted into the proposal Queue (progress Queue), and the access requests of subsequent clients, for example, the access Request new Request of client4 and the access requests forwarded by other Follower nodes, are all put into the tail of the proposal Queue according to the receiving sequence, and then corresponding data is submitted to the consistency protocol module through the Replica. And the client with the closed connection (session3) will randomly re-establish the connection with the node that can be served in the cluster. Therefore, the embodiment provides a smooth mechanism for host node switching, receives and stores client access requests through the asynchronous queue, and realizes the process of converting from the Follower to the Leader without changing the access request sequence, thereby ensuring the consistency of the access request sequence of each client.

and if the role type of the slave node is the slave node type, sending the access request in the waiting queue to a new master node so that the new master node initiates the verification of the access request in the waiting queue.

In this embodiment, the role of the slave node is unchanged, and therefore, the access request in the waiting queue is still sent to the new master node according to the mode in the normal processing flow, so that the new master node initiates the check on the access request in the waiting queue. For a specific process in which the new master node initiates the check on the access request in the waiting queue, reference may be made to the description of the embodiment in fig. 6, and the process is the same and is not described herein again. In this way, in a scenario where a master node is switched, that is, a new master node is elected, the present embodiment can ensure processing of an access request under the condition that the role of the slave node is not changed through node role identification.

Corresponding to the above method embodiment, this specification further provides an access request responding apparatus embodiment, and fig. 7 shows a schematic structural diagram of an access request responding apparatus provided in an embodiment of this specification. As shown in fig. 7, the slave node applied to the distributed system includes:

a stopping module 702, configured to stop writing data in the data log into the application queue when detecting disconnection with the master node, where the data log records data to be subjected to node consistency judgment, and the application queue records data meeting a node consistency judgment condition;

a matching module 704, configured to match the access request in the waiting queue with the data in the application queue, and maintain a connection of the first client corresponding to the first access request that is successfully matched, where the waiting queue records the access request to be responded;

a response module 706 configured to respond to the first access request and return a response result to the first client.

Optionally, the matching module 704 is further configured to:

and after the access requests in the waiting queue are matched with the data in the application queue, maintaining the connection of the second client corresponding to the second access request which is not successfully matched.

Optionally, the matching module 704 is further configured to:

accordingly, the response module 706 is further configured to:

Optionally, the response module 706 is further configured to:

Optionally, the matching module 704 is further configured to:

after matching the access request with the data in the application queue, if the matching fails, polling the access requests in the waiting queue from the access request with the current write type, and adding the client corresponding to each access request with the write type into a client removal set;

Optionally, any client has a requirement on consistency of the access request sequence, the request identifiers of the access requests of the same client are set according to the sequence, and the data requested by the access requests correspond to the request identifiers; the number of the first access requests is multiple;

accordingly, the response module 706 is further configured to:

Optionally, the apparatus further comprises: a verification module configured to:

Optionally, the verification module is further configured to:

The foregoing is an exemplary scheme of an access request responding apparatus according to the present embodiment. It should be noted that the technical solution of the access request responding apparatus belongs to the same concept as the technical solution of the access request responding method described above, and for details that are not described in detail in the technical solution of the access request responding apparatus, reference may be made to the description of the technical solution of the access request responding method described above.

Corresponding to the above method embodiment, the present specification further provides a distributed system embodiment, and fig. 8 shows a schematic structural diagram of a distributed system provided by an embodiment of the present specification. As shown in fig. 8, the distributed system 800 includes: a slave node 802, a master node 804, and a first client 806;

the slave node 802 is configured to stop writing data in the data log into the application queue when disconnection from the master node 804 is detected, wherein the data log records data to be subjected to node consistency judgment, and the application queue records data meeting a node consistency judgment condition; matching the access requests in the waiting queue with the data in the application queue, and keeping the connection with a first client corresponding to the first access request which is successfully matched, wherein the waiting queue records the access requests to be responded; responding to the first access request and returning a response result to the first client 806;

the first client 806 is configured to receive the response result.

In an embodiment of the present specification, when a slave node of a distributed system is disconnected from a master node, data in a data log is stopped being written into an application queue, so as to avoid a problem of consistency abnormality caused by feeding back data that is uncertain whether a master node performs a node consistency judgment to a client when the master node is disconnected. On the basis, normal feedback can be performed on the data written into the application queue, and therefore, the connection between the first clients corresponding to the first access requests matched with the data in the application queue can be maintained, and the first access is responded and the return result is returned to the first client. Therefore, consistency abnormity caused by disconnection with the main node and response abnormity to each access request can be avoided under the condition that all client terminals are not required to be disconnected, so that the pressure of the distributed system caused by a large number of reconnection after all client terminals are disconnected is greatly reduced, and the working performance of the distributed system is ensured. Therefore, the scheme can ensure that the availability of the distributed system is higher when the distributed system is disconnected with the main node.

Optionally, the slave node 802, is further configured to:

maintaining the connection of a third client corresponding to a third access request, wherein the third access request is an access request with a read type request type;

Optionally, the slave node 802 is further configured to:

accordingly, the slave node 802, is further configured to:

Optionally, the slave node 802 is further configured to, after detecting disconnection from the master node, identify a role type of the slave node when detecting election of a new master node;

Optionally, the slave node 802 is further configured to:

Optionally, the slave node 802, is further configured to:

The foregoing is a schematic scheme of a distributed system according to this embodiment. It should be noted that the technical solution of the distributed system and the technical solution of the access request response method described above belong to the same concept, and details that are not described in detail in the technical solution of the distributed system can be referred to the description of the technical solution of the access request response method described above.

FIG. 9 illustrates a block diagram of a computing device, according to one embodiment of the present description. Components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is coupled to the memory 910 via a bus 930, and a database 950 is used to store data.

Computing device 900 also includes access device 940, access device 940 enabling computing device 900 to communicate via one or more networks 960. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 940 may include one or more of any type of Network Interface (e.g., a Network Interface Controller (NIC)) whether wired or Wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) Wireless Interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) Interface, an ethernet Interface, a Universal Serial Bus (USB) Interface, a cellular Network Interface, a bluetooth Interface, a Near Field Communication (NFC) Interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 900, as well as other components not shown in FIG. 9, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device structure shown in FIG. 9 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.

Computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 900 may also be a mobile or stationary server.

Wherein the processor 920 is configured to execute computer-executable instructions that, when executed by the processor, implement the steps of the above-described access request response method.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the access request response method described above belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the access request response method described above.

An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which when executed by a processor, implement the steps of the above-mentioned access request response method.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above-mentioned access request response method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above-mentioned access request response method.

An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned access request response method.

The above is an illustrative scheme of a computer program of the present embodiment. It should be noted that the technical solution of the computer program belongs to the same concept as the technical solution of the above access request response method, and for details that are not described in detail in the technical solution of the computer program, reference may be made to the description of the technical solution of the above access request response method.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts, but those skilled in the art should understand that the present embodiment is not limited by the described acts, because some steps may be performed in other sequences or simultaneously according to the present embodiment. Furthermore, those skilled in the art will appreciate that the embodiments described in this specification are presently preferred and that no acts or modules are required in the implementations of the disclosure.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the teaching of the embodiments of the present disclosure. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the embodiments. The specification is limited only by the claims and their full scope and equivalents.

Claims

1. An access request response method is applied to a slave node of a distributed system and comprises the following steps:

2. The access request responding method according to claim 1, further comprising, after the matching of the access request in the wait queue and the data in the application queue:

and maintaining the connection of the second client corresponding to the second access request which is not matched successfully.

3. The access request responding method according to claim 1, wherein the matching of the access request in the waiting queue and the data in the application queue comprises:

identifying the request type of the access request in the waiting queue, and matching the access request with the data in the application queue, wherein the request type is a write type;

accordingly, said responding to said first access request comprises:

4. The access request responding method according to claim 3, further comprising, after the stopping writing of the data in the data log to the application queue:

5. The access request response method according to claim 3, further comprising, after the matching of the access request with the data in the application queue, the request type being a write type:

and disconnecting the connection between the client terminals contained in the client terminal removing set.

6. The access request response method according to any one of claims 1 to 5, wherein any client has a requirement for consistency of an order of access requests, request identifiers of access requests of the same client are set according to the order, and data requested by the access requests correspond to the request identifiers; the number of the first access requests is multiple;

accordingly, said responding to said first access request comprises:

obtaining a first access request queue containing each first access request based on the sequence of the request identifier of each first access request;

according to the first-in first-out principle, performing request identification matching on each first access request in the first access request queue and the data in the application queue, and obtaining a response result queue of each first access request based on the successfully matched data;

the returning the response result to the first client includes:

and returning the response result in the response result queue to the first client according to the first-in first-out principle.

7. The access request responding method according to any one of claims 1 to 5, further comprising, after the detecting of the disconnection from the master node:

8. The access request responding method according to claim 7, wherein the checking the access requests in the waiting queue according to the checking manner includes:

and if the role type of the slave node is the master node type, adding the access request in the waiting queue to a queue to be checked, and checking the access request in the queue to be checked according to a first-in first-out principle.

9. The access request responding method according to claim 7, wherein the checking the access requests in the waiting queue according to the checking manner includes:

and if the role type of the slave node is the slave node type, sending the access request in the waiting queue to the new master node so that the new master node initiates verification on the access request in the waiting queue.

10. An access request response device applied to a slave node of a distributed system, comprising:

11. A distributed system, comprising: the system comprises a slave node, a master node and a first client;

the first client configured to receive the response result.

12. A computing device, comprising:

a memory and a processor;

the memory is for storing computer-executable instructions and the processor is for executing the computer-executable instructions, which when executed by the processor implement the steps of the access request response method of any one of claims 1 to 9.

13. A computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the steps of the access request response method of any one of claims 1 to 9.