CN115756855A

CN115756855A - Cluster switching method, device, terminal and storage medium

Info

Publication number: CN115756855A
Application number: CN202211477039.3A
Authority: CN
Inventors: 卢杰; 胡晓花
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2022-11-23
Filing date: 2022-11-23
Publication date: 2023-03-07

Abstract

The embodiment of the invention discloses a cluster switching method, a cluster switching device, a terminal and a storage medium. The method applies a production server, wherein the production client is currently connected to a cluster before switching and corresponds to a target theme in the cluster after switching, and the method comprises the following steps: responding to a cluster switching event, acquiring a node set of a switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node; receiving metadata returned by a metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node positioned under a target topic in a node set; and determining a current service node from the theme nodes based on the metadata, and connecting the current service node to the current service node so as to switch from the cluster before switching to the cluster after switching. The technical scheme of the embodiment of the invention can realize the automatic cluster switching process of the production client under the condition of not restarting the client.

Description

Cluster switching method, device, terminal and storage medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a cluster switching method, a cluster switching device, a terminal and a storage medium.

Background

In the message queue, ensuring high availability of Kafka clusters is a core goal of research and development teams. The highly available solutions often involve multiple considerations, of which disaster recovery solutions are in existence.

At present, a main cluster and a standby cluster are usually adopted, and when the main cluster is abnormal, a user restarts each client connected to the main cluster so as to enable the clients to be switched and connected to the standby cluster.

In the process of implementing the present invention, the inventor finds that the following technical problems exist in the prior art: the user is required to restart the client in the cluster switching process, but the client is easy to have abnormity in the restarting process, and the time and the labor are consumed, which is not friendly to the user.

Disclosure of Invention

The embodiment of the invention provides a cluster switching method, a cluster switching device, a terminal and a storage medium, which are used for realizing the automatic cluster switching process of a client under the condition of not restarting the client.

According to an aspect of the present invention, a cluster switching method is provided, which is applied to a production client, where the production client is currently connected to a cluster before switching and corresponds to a target topic in a cluster after switching, and the method may include:

responding to a cluster switching event, acquiring a node set of a switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node;

receiving metadata returned by a metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node positioned under a target topic in a node set;

and determining a current service node from the theme nodes based on the metadata, and connecting the current service node to the current service node so as to switch from the cluster before switching to the cluster after switching.

According to another aspect of the present invention, there is provided a cluster switching method applied to a consuming client, where the consuming client is currently connected to a cluster before switching and corresponds to a target topic in the cluster after switching, the method may include:

According to another aspect of the present invention, there is provided a cluster switching apparatus configured at a production client currently connected to a cluster before switching and corresponding to a target topic under the cluster after switching, the apparatus may include:

the metadata request first sending module is used for responding to a cluster switching event, acquiring a node set of a switched cluster, acquiring a metadata request node from the node set and sending a metadata request to the metadata request node;

the first metadata receiving module is used for receiving metadata returned by a metadata request node aiming at a metadata request, wherein the metadata is used for representing information of each topic node positioned under a target topic in a node set;

and the cluster first switching module is used for determining a current service node from the theme nodes based on the metadata and connecting the current service node to the cluster after switching from the cluster before switching to the cluster after switching.

According to another aspect of the present invention, there is provided a cluster switching apparatus configured at a consuming client currently connected to a cluster before switching and corresponding to a target topic under the cluster after switching, the apparatus may include:

the metadata request second sending module is used for responding to a cluster switching event, acquiring a node set of a switched cluster, acquiring a metadata request node from the node set and sending the metadata request to the metadata request node;

the metadata second receiving module is used for receiving metadata returned by the metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node positioned under the target topic in the node set;

and the cluster second switching module is used for determining a current service node from the theme nodes based on the metadata and connecting the current service node to the cluster after switching from the cluster before switching to the cluster after switching.

According to another aspect of the present invention, there is provided a terminal, which may include:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform a cluster switching method as provided by any of the embodiments of the invention when executed.

According to another aspect of the present invention, there is provided a computer-readable storage medium having stored thereon computer instructions for causing a processor to execute a cluster switching method provided by any of the embodiments of the present invention.

According to the technical scheme of the embodiment of the invention, aiming at the production client currently connected to the cluster before switching, the production client responds to the cluster switching event to obtain the node set of the switched cluster, then obtains the metadata request node from the node set and sends the metadata request to the metadata request node; receiving metadata returned by a metadata request node aiming at the metadata request; because the metadata can be used to represent information of each topic node in the node set under the target topic, where the target topic is a topic corresponding to the production client in the post-switching cluster, a current service node can be determined from each topic node based on the metadata, and the current service node is connected to the current service node, so that the production client is switched from the pre-switching cluster to the post-switching cluster. According to the technical scheme, the metadata used for representing the information of the topic nodes related to the production client side under the switched cluster is requested, and then the production client side is automatically switched and connected to the switched cluster from the cluster before switching on the basis of the metadata, so that the effect of automatic cluster switching of the production client side is achieved under the condition that the production client side does not need to be restarted, namely the production client side is continuously operated.

It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a cluster switching method according to an embodiment of the present invention;

fig. 2 is a flowchart of another cluster switching method provided in an embodiment of the present invention;

fig. 3a is a flowchart of an example of cluster switching applied to a production client in a cluster switching method according to an embodiment of the present invention;

fig. 3b is a flowchart of an example of metadata update applied to a production client in a cluster switching method according to an embodiment of the present invention;

fig. 4 is a flowchart of another cluster switching method provided in accordance with an embodiment of the present invention;

fig. 5 is a flowchart of another cluster switching method provided in accordance with an embodiment of the present invention;

fig. 6a is a flowchart of an example of cluster switching applied to a consuming client in a cluster switching method according to an embodiment of the present invention;

fig. 6b is a flowchart of an example of metadata update applied to a consuming client in a cluster switching method according to an embodiment of the present invention;

fig. 7 is a block diagram of a cluster switching device according to an embodiment of the present invention;

fig. 8 is a block diagram of another cluster switching device provided in an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a production terminal or a consumption terminal implementing the cluster switching method according to the embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein. The cases of "target", "original", etc. are similar and will not be described in detail herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a flowchart of a cluster switching method provided in an embodiment of the present invention. The method is applied to a production client which is currently connected to a cluster before switching and corresponds to a target theme in the cluster after switching, and the target theme can be understood as a theme corresponding to the production client in each theme (topic) in the cluster after switching. In combination with an application scenario that may be related to the embodiment of the present invention, both the cluster before the handover and the cluster after the handover may be a message queue cluster, such as a Kafka cluster or a pulser cluster, and may particularly be a Kafka cluster, which is not specifically limited herein.

The embodiment is applicable to the situation that the production client connected to the cluster before switching is switched to the cluster after switching, and is particularly applicable to the situation that the cluster switching process of the production client is completed under the situation that the production client is continuously operated. The method may be performed by the cluster switching device provided in the embodiment of the present invention, and the device may be implemented by software and/or hardware, and the device may be integrated on a production terminal.

Referring to fig. 1, the method of the embodiment of the present invention specifically includes the following steps:

s110, responding to a cluster switching event, acquiring a node set of the switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node.

The cluster switching event may be an event used for indicating that the production client is switched and connected to the switched cluster, and in response to the cluster switching event, a node set of the switched cluster is obtained, where the node set may include some or all nodes in the switched cluster, and the node may be understood as a server, specifically, a message queue server. In practical applications, optionally, for each node, the node may be represented based on the node identifier of the node, in other words, the node identifiers of the nodes may be stored in the node set. Further, a metadata request node for requesting metadata is acquired from the node set and connected to the metadata request node, so that a metadata request (i.e., a request for requesting metadata) is sent to the connected metadata request node to request the corresponding metadata.

S120, receiving metadata returned by the metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node under the target topic in the node set.

After receiving the metadata request sent by the production client, the metadata request node may determine metadata matching the metadata request, and then return the metadata to the production client, so that the production client receives the metadata. It should be noted that the metadata herein may be used to represent relevant information of each subject node in the node set, which is located under the target subject, such as node identifiers, node positions, node functions, and cluster identifiers of the subject nodes, which are not specifically limited herein.

On this basis, optionally, assuming that a topic corresponding to a target topic in a cluster before switching is referred to as an original topic, since a production client is currently connected to the cluster before switching, which means that metadata for representing relevant information of each topic node located under the original topic is cached in the production client, after receiving the metadata corresponding to the target topic, the cached metadata may be updated based on the received metadata, so that when the cached metadata is applied, an operation object is a cluster after switching and is not a cluster before switching any more; or, the received metadata may also be cached in the production client, at this time, at least two pieces of metadata are cached in the production client, and then when the metadata is applied, the metadata with the latest caching time may be applied to ensure that the operation object is the switched cluster; and the like, are not specifically limited herein.

S130, determining a current service node from each topic node based on the metadata, and connecting to the current service node so as to switch from the cluster before switching to the cluster after switching.

As described above, since the received metadata may be used to represent information of each topic node located under the target topic in the node set, the topic nodes may be obtained from the node set based on the metadata, and further, a current service node may be determined from the topic nodes based on the metadata, where the current service node may be understood as a topic node that currently provides a service (such as a production service or a consumption service) to the outside, and the number of the current service nodes may be one or more, which is not specifically limited herein.

In connection with an application scenario that may be involved in an embodiment of the present invention, for example, the post-switching cluster may be provided with at least one Partition (Partition) under the target topic, and for each Partition, at least one copy (Replication) may be provided in the Partition, and the copies are distributed on different nodes of the post-switching cluster. In general, only one of the copies provides a service to the outside, and then a subject node where the copy providing the service to the outside is located is a current service node (which may also be referred to as a leader node). All of the above information can be represented by the received metadata.

Because the current service node is a theme node for providing services to the outside, the production client can write the self-produced message into the target theme after being connected to the current service node, thereby ensuring the normal operation of the production client. Accordingly, the production client can disconnect the production client from the cluster before switching and then connect to the current service node. Because the current service node is a node which allows the production client to write the message currently on the switched cluster, the production client connected to the current service node is naturally and successfully switched and connected to the switched cluster. In practical applications, optionally, the step of disconnecting the production client from the cluster before switching may be performed at any time before the production client is connected to the current service node, and is not specifically limited herein.

It should be noted that, taking a certain production client C as an example, for the solution stated in the background art that a user needs to complete cluster switching by restarting the production client C, restarting the production client C means to stop operating the production client C first and then start operating the production client C. Since the production client C in the shutdown state is not connected to any cluster, this means that the production client C is not connected to any cluster at the initial time of starting to run the production client C, and then connected to the cluster after switching in such a condition that the change process of the connection state of the production client C in the running state is: not connected to any cluster-to the cluster after the switch. In contrast, according to the solution set forth in the embodiment of the present invention, since the production client C does not need to be restarted, which means that the production client C is always in the running state, the connection state of the production client C in the running state is: connected to the cluster before switching-not connected to any cluster-connected to the cluster after switching. Therefore, the two schemes are obviously different in the changing process of the connection state.

According to the technical scheme of the embodiment of the invention, aiming at the production client currently connected to the cluster before switching, the production client responds to the cluster switching event to obtain the node set of the switched cluster, then obtains the metadata request node from the node set and sends the metadata request to the metadata request node; receiving metadata returned by a metadata request node aiming at the metadata request; because the metadata can be used to represent information of each topic node in the node set under the target topic, where the target topic is a topic corresponding to the production client in the post-switching cluster, a current service node can be determined from each topic node based on the metadata, and the current service node is connected to the current service node, so that the production client is switched from the pre-switching cluster to the post-switching cluster. According to the technical scheme, the metadata used for representing the information of the subject nodes related to the production client in the switched cluster is requested, and then the production client is automatically switched and connected to the switched cluster from the cluster before switching on the basis of the metadata, so that the effect of automatic cluster switching of the production client is achieved under the condition that the production client does not need to be restarted, namely the production client is continuously operated.

An optional technical solution, in the cluster switching method, may further include: obtaining cached metadata, wherein the cached metadata is associated with an original theme corresponding to the target theme in the cluster before switching; updating the cached metadata based on the received metadata; determining a current service node from the subject nodes based on the metadata, including: a current service node is determined from the respective subject nodes based on the cached metadata. The original topic can be understood as a topic with the same function as the topic of the target topic in the cluster before switching, that is, when the production client is connected with the cluster before switching, the message produced by the production client is written into the original topic. On this basis, since the production client can write the self-produced message into the original theme, this means that the metadata associated with the original theme can be cached in the production client. To this end, in order to connect the production client to the switched cluster, upon receiving metadata related to the switched cluster, the cached metadata may be updated based on the received metadata. In this way, when the cached metadata is subsequently applied, the operation object is the switched cluster, and various requests are sent to the nodes under the switched cluster, so that the effective switching of the cluster of the production client is ensured.

Another optional technical solution is that a production client corresponds to a production thread, and the cluster switching method may further include: suspending the production thread; acquiring a cluster identifier before switching of a cluster before switching; under the condition of successfully receiving the metadata, determining a post-switching cluster identifier of the post-switching cluster based on the metadata, and comparing the post-switching cluster identifier with a pre-switching cluster identifier; and under the condition that the cluster identifier after switching is determined to be inconsistent with the cluster identifier before switching according to the obtained comparison result, awakening the suspended production thread so as to send a production request to the connected current service node through the awakened production thread. Wherein, a production thread may be understood as a thread that produces messages for a production client and writes the produced messages to a cluster. Because the cluster needs to be switched for the production client at this time, the production thread can be suspended, and the situation that the production client writes the production message into the cluster before switching is avoided. Under the condition that the cluster identifier after switching is inconsistent with the cluster identifier before switching, the received metadata is the metadata of the cluster after switching, and the metadata can enable the production client to be switched to the cluster after switching, so that the suspended production thread can be awakened, a production request can be sent to the connected current service node through the awakened production thread, and the effect of writing the produced message into the cluster after switching is achieved.

Another optional technical solution is that, acquiring a metadata request node from a node set includes: determining, for each connectable node in the set of nodes, a number of connections of connected clients that have been connected on the connectable node; the connectable node with the minimum connection number in the node set is used as a metadata request node; sending the metadata request to the metadata request node may include: connecting to the metadata request node, and sending the metadata request to the connected metadata request node. In this case, a connectable node may be understood as a node in the node set that allows a production client to connect, and in practical applications, these connectable nodes may all be used as metadata request nodes. However, it is considered that there are many production clients and consumption clients to be connected to the post-switching cluster, and a metadata request node needs to be allocated to each production client and each consumption client, respectively. Therefore, in order to reduce the request pressure on the metadata request node, the number of connections of connected clients that have been connected to the connectable node, which may be production clients or consumption clients, may be determined for each connectable node, which is not specifically limited herein. Further, the connectable node with the smallest number of connections may be used as the metadata request node, and then the production client currently being processed may be connected to the metadata request node. In this way, the generation client can successfully request the metadata, and the situation that the metadata request node crashes due to the need of processing excessive metadata requests can be avoided.

Fig. 2 is a flowchart of another cluster switching method provided in the embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, responding to the cluster switching event may include: responding to a cluster switching event under the condition that the cluster switching event from the configuration center is detected, wherein the cluster switching event is triggered when the configuration in the configuration center is changed; obtaining the node set of the cluster after switching may include: and acquiring a switched cluster identifier of the currently configured switched cluster from the configuration center, and acquiring a node set of the switched cluster based on the switched cluster identifier. The explanations of the same or corresponding terms as those in the above embodiments are omitted.

Referring to fig. 2, the method of the present embodiment may specifically include the following steps:

s210, under the condition that a cluster switching event from the configuration center is detected, responding to the cluster switching event, wherein the cluster switching event is triggered when the configuration in the configuration center is changed.

The configuration center may be understood as a preset platform for performing cluster configuration, and in particular, when the configuration in the configuration center is changed, the production client in the running state may dynamically sense the change process. Specifically, when the configuration in the configuration center is changed, this indicates that the production client needs to perform cluster switching, and therefore, when the configuration center detects that the configuration of the production client is changed, a cluster switching event may be triggered or generated. On the basis, the configuration center can actively send the cluster switching event to the production client so that the production client responds to the received cluster switching event to execute corresponding cluster switching operation; or, the production client may also acquire the cluster switching event from the configuration center when dynamically perceiving the cluster switching event generated by the configuration center, and respond to the acquired cluster switching event to execute the corresponding cluster switching operation; etc., and are not specifically limited herein.

S220, acquiring a switched cluster identifier of the switched cluster configured currently from the configuration center, and acquiring a node set of the switched cluster based on the switched cluster identifier.

Since the configuration in the configuration center has been changed, the information about the switched cluster is currently configured in the configuration center, and then the switched cluster identifier of the switched cluster can be obtained from the configuration center, and the switched cluster identifier can be represented by an address, a cluster name, or a domain name of the switched cluster, which is not specifically limited herein. Further, the node set is obtained according to the post-switching cluster identifier, for example, a storage address of the node set is obtained by parsing the post-switching cluster identifier, and the node set is obtained from the storage address, and so on.

And S230, acquiring the metadata request node from the node set, and sending the metadata request to the metadata request node.

S240, receiving metadata returned by the metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node positioned under the target topic in the node set.

And S250, determining a current service node from each topic node based on the metadata, and connecting to the current service node so as to switch from the cluster before switching to the cluster after switching.

According to the technical scheme of the embodiment of the invention, when the production client detects the cluster switching event from the configuration center, which indicates that the configuration in the configuration center has been changed, the switched cluster identifier of the switched cluster currently configured can be acquired from the configuration center by responding to the cluster switching event, and then the node set is acquired based on the switched cluster identifier, so that the corresponding cluster switching operation is executed based on the node set. According to the technical scheme, the production client in the running state can dynamically sense the cluster switching event and further trigger the corresponding cluster switching operation, so that the timeliness of cluster switching is guaranteed.

An optional technical solution, before the cluster handover event is detected from the configuration center, the cluster handover method may further include: and in response to the cluster exception event, changing the cluster identifier before switching of the cluster currently configured in the configuration center into the cluster identifier after switching. In practical application, optionally, the configuration in the configuration center may be actively changed by a user, that is, the user actively switches the cluster; the configuration in the configuration center may also be changed automatically by the production client, that is, the cluster is switched automatically when the production client senses that the cluster is abnormal before switching. The arrangement of two active and passive cluster switching strategies effectively guarantees the availability of the whole production and consumption process. For the latter, specifically, the cluster abnormal event may be an event detected by the production client and used for indicating that the cluster before switching currently connected to is abnormal, and the production client changes the cluster identifier before switching currently configured in the configuration center to the cluster identifier after switching in response to the cluster abnormal event, so as to trigger the cluster switching event to allow the production client to execute the corresponding cluster switching operation. In practical application, optionally, the production client may continuously try to connect the cluster before switching before changing the configuration in the configuration center, and after the preset connection time or the preset connection times, the cluster before switching is still unavailable, and it may be determined that the cluster before switching is really abnormal, and at this time, the configuration in the configuration center may be changed again.

In order to better understand the specific execution steps of the production client in the cluster switching process as a whole, the following describes an exemplary embodiment of the present invention with reference to specific examples. Illustratively, referring to the cluster switching example shown in fig. 3a and the metadata updating example shown in fig. 3b, the two examples will be explained in combination. In particular, the method comprises the following steps of,

and acquiring a current cluster identifier of the current configuration from the configuration center, and judging whether the current cluster identifier is consistent with the current connection identifier of the current connection cluster currently connected with the production client. If not, it indicates that the cluster is switched, the current cluster identifier in the configuration center is the post-switching cluster identifier of the post-switching cluster, and the current connection identifier is the pre-switching cluster identifier of the pre-switching cluster. And under the condition that the cluster switching event is detected, responding to the cluster switching event, and starting to execute metadata updating operation, so that the metadata which is cached in the production client and is related to the cluster before switching is updated into the metadata which is related to the cluster after switching. In particular, the method comprises the following steps of,

first, connection cluster information of a current connection cluster is obtained, where the connection cluster information may include at least a node list a and a current connection Identity (Identity, id), etc. And then, analyzing the configured current cluster identifier, acquiring a node list B of the switched cluster based on the analysis result, and updating the cached node list A based on the node list B, so that all subsequent requests can be performed by selecting nodes from the node list B, thereby ensuring that the subsequent metadata requests are initiated to the switched cluster. Then, a metadata request is sent to update the cached metadata based on the requested metadata, at which point the production thread for writing the message may be suspended waiting for the metadata update to complete. In particular, the method comprises the following steps of,

in the metadata updating process, the production client disconnects the current connection cluster, selects a node with the least connection quantity from the node list B as a metadata request node, establishes connection with the metadata request node, and requests metadata C from the metadata request node. After requesting metadata C, the representation cluster identifier represented by metadata C is parsed and compared with the representation cluster identifier represented by cached metadata D. If the two are not consistent, the metadata D requested by the request is the metadata of the switched cluster, the metadata C is updated based on the metadata D, and the cached metadata D can be connected to the switched cluster based on the metadata D until the metadata is updated, and at the moment, the production thread can be awakened, so that a message is sent to the switched cluster; if the two are consistent, which indicates that the metadata update is not complete, then the attempt to request and update the metadata may continue until the configured metadata update time is exceeded, throwing a metadata update timeout exception.

Under the condition that the cluster is not switched or the metadata update is completed, the cached connection cluster information can be updated based on the connection cluster information of the current connection cluster, so that the cached connection cluster information is ensured to be associated with the current connection cluster.

Fig. 4 is a flowchart of another cluster switching method provided in the embodiment of the present invention. The method is applied to a consuming client which is currently connected to a cluster before switching and corresponds to a target topic in the cluster after switching, and the target topic can be understood as a topic corresponding to the consuming client in each topic (topic) in the cluster after switching. In combination with an application scenario that may be related to the embodiment of the present invention, both the cluster before the handover and the cluster after the handover may be a message queue cluster, for example, a Kafka cluster or a pulser cluster, and especially may be a Kafka cluster, which is not specifically limited herein.

The embodiment is applicable to the situation that the consumption client connected to the cluster before switching is switched to be connected to the switched cluster, and is particularly applicable to the situation that the cluster switching process of the consumption client is completed under the situation that the consumption client is continuously operated. The method may be performed by a cluster switching apparatus provided in the embodiments of the present invention, and the apparatus may be implemented by software and/or hardware, and the apparatus may be integrated on a consumer terminal.

Referring to fig. 4, the method of the embodiment of the present invention specifically includes the following steps:

s310, responding to a cluster switching event, acquiring a node set of the switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node.

S320, receiving metadata returned by the metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node positioned under the target topic in the node set.

S330, determining a current service node from each topic node based on the metadata, and connecting to the current service node so as to switch from the cluster before switching to the cluster after switching.

It should be noted that the specific implementation process of the above steps is substantially the same as the specific implementation process of S110-S130, and the difference is mainly that S110-S130 are steps executed by a production client, and S310-S330 are steps executed by a consumption client, where the production client is used to produce a message to a current service node after being connected to the switched cluster, and the consumption client is used to consume a message to the current service node after being connected to the switched cluster. Therefore, the specific implementation process of the above steps is not described herein again.

According to the technical scheme of the embodiment of the invention, aiming at the consumption client side which is currently connected to the cluster before switching, the consumption client side responds to the cluster switching event to obtain the node set of the cluster after switching, then obtains the metadata request node from the node set and sends the metadata request to the metadata request node; receiving metadata returned by a metadata request node aiming at the metadata request; because the metadata can be used to represent information of each topic node in the node set under the target topic, where the target topic is a topic corresponding to the consuming client in the switched cluster, a current service node can be determined from each topic node based on the metadata, and the current service node is connected to the current service node, so that the consuming client is switched from the cluster before switching to the switched cluster. According to the technical scheme, the metadata used for representing the information of the topic nodes related to the consumption client in the switched cluster is requested, and then the consumption client is automatically switched and connected to the switched cluster from the cluster before switching based on the metadata, so that the effect of automatic cluster switching of the consumption client is achieved without restarting the consumption client, namely under the condition that the service is not perceived.

An optional technical solution is that obtaining a node set of a cluster after switching may include: aiming at an original theme corresponding to a target theme in a cluster before switching, acquiring an original writing site and an original consuming site in the original theme; and under the condition that all messages under the original topic are determined to be consumed according to the original writing sites and the original consumption sites, acquiring the node set of the cluster after switching.

The original writing position point can represent the position point of the latest written message under the original subject; the original consumption site may represent a site where a newly consumed message under the original subject is located, i.e., a message after the original consumption site and before the original consumption site has been consumed completely, and a message after the original consumption site has not been consumed. The backlog of messages for the cluster before the switch (i.e., whether there are messages that are not consumed) can be determined from the original write site and the original consumption site. Illustratively, if the original write location minus the original consumption location is 0, this indicates that there is no backlog message in the cluster before the switch, otherwise, it indicates that there is a backlog message. In order to avoid the situation that the consumption of the message written into the cluster before the switching is failed due to the cluster switching, the corresponding cluster switching operation can be executed when no backlog message exists. Therefore, when all messages under the original subject are determined to be consumed according to the original writing sites and the original consumption sites, the node set of the switched cluster can be acquired to execute corresponding cluster switching operation, and therefore the integrity of the consumed messages is guaranteed.

The technical scheme is suitable for the situation that the cluster can normally consume the messages before switching, and the message backlog situation does not need to be considered for the application scene of cluster switching due to the fact that the cluster fails before switching.

In another optional technical solution, for a consumption group to which a consumption client belongs, the cluster switching method may further include: acquiring a coordination node searching node from the node set, and sending a coordination node searching request to the coordination node searching node; receiving a coordination node returned by a coordination node searching node aiming at a coordination node searching request, so as to obtain a partition under a target theme distributed for a consumption client from the coordination node, wherein the coordination node belongs to a node set and corresponds to a consumption group; determining a current service node from the respective subject nodes based on the metadata may include: a current service node belonging to the partition is determined from the respective subject nodes based on the metadata.

In combination with an application scenario that may be involved in an embodiment of the present invention, at least one partition may be set under a target topic, and a consuming client may be configured to consume a message stored in a certain partition of the at least one partition. A consumption group may be formed based on at least one consumption client, where the consumption group may be used to completely consume all messages under the target topic, and a Coordinator (Coordinator) node corresponding to the consumption group may be used to manage relevant matters of the consumption group, for example, after cluster switching, a rebalancing partition is reallocated for each consumption client in the consumption group to perform consumption.

On this basis, for the consumption group to which the consumption client currently being processed belongs, the coordination node search node may be a node in the node set, which is used to search for a coordination node corresponding to the consumption group, and an acquisition process of the coordination node search node may be the same as or different from an acquisition process of the metadata request node, which is not specifically limited herein. And sending the coordination node searching request to the coordination node searching node, and then receiving a coordination node which belongs to the node set and is returned by the coordination node searching node aiming at the coordination node searching request, so that the partition under the target theme distributed for the consumption client side is obtained from the coordination node. In this way, since the metadata is recorded with the relevant information of the current service node in each partition under the target topic, after the partition corresponding to the consuming client is determined, the current service node capable of providing service for the consuming client can be obtained from each topic node based on the metadata, thereby ensuring the accuracy of determining the current service node.

On the basis of any of the above technical solutions, optionally, the consuming client corresponds to a consuming thread, and the cluster switching method may further include: suspending the consuming thread; acquiring a cluster identifier before switching of a cluster before switching; under the condition of successfully receiving the metadata, determining a post-switching cluster identifier of the post-switching cluster based on the metadata, and comparing the post-switching cluster identifier with a pre-switching cluster identifier; and under the condition that the cluster identifier after switching is determined to be inconsistent with the cluster identifier before switching according to the obtained comparison result, awakening the suspended consuming thread so as to send a consuming request to the connected current service node through the awakened consuming thread.

Optionally, in response to the cluster switching event, the method may include: responding to a cluster switching event under the condition that the cluster switching event from the configuration center is detected, wherein the cluster switching event is triggered when the configuration in the configuration center is changed; obtaining the node set of the cluster after switching may include: and acquiring a switched cluster identifier of the switched cluster configured currently from the configuration center, and acquiring a node set of the switched cluster based on the switched cluster identifier.

On this basis, optionally, before the case of detecting the cluster switching event from the configuration center, the cluster switching method may further include: and changing the cluster identifier before switching of the cluster currently configured in the configuration center into the cluster identifier after switching in response to the cluster abnormal event.

Optionally, obtaining the metadata request node from the node set may include: determining, for each connectable node in the set of nodes, a number of connections of connected clients that have been connected on the connectable node; the connectable node with the minimum connection number in the node set is used as a metadata request node; sending the metadata request to the metadata request node may include: connecting to the metadata request node, and sending the metadata request to the connected metadata request node.

It should be noted that the specific implementation processes of the several technical solutions executed by the consuming client are substantially the same as the specific implementation processes of the similar solutions executed by the producing client, and are not described herein again.

Fig. 5 is a flowchart of another cluster switching method provided in the embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the consuming client corresponds to a consuming thread, and the cluster switching method may further include: suspending the consuming thread and recording a suspension time point when the consuming thread is suspended; taking a history writing site of the target theme on a history time point before the suspension time point as an initial consumption site under the target theme; and waking up the suspended consumption thread to send a consumption request for consuming the message from the initial consumption site to the current service node connected through the woken consumption thread. The same or corresponding terms as those in the above embodiments are not explained in detail herein.

Referring to fig. 5, the method of this embodiment may specifically include the following steps:

s410, responding to a cluster switching event, acquiring a node set of the switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node.

S420, suspending the consumption thread corresponding to the consumption client, and recording a suspension time point when the consumption thread is suspended.

Wherein, a consuming thread can be understood as a thread that a consuming client consumes a message on a current service node. Because the cluster needs to be switched for the consuming client at this time, the consuming thread can be suspended, and the situation that the consuming client continues to consume the message on the cluster before switching is avoided. On the basis, a suspension time point when the consumption thread is suspended can be recorded so as to determine the initial consumption site of the consumption client on the cluster after switching based on the suspension time point.

And S430, writing the history of the target theme at the history time point before the suspension time point into a site as an initial consumption site under the target theme.

In order to ensure the integrity of the consumption message, that is, to ensure that each message produced is successfully consumed, and to avoid the situation that some messages produced are not consumed due to cluster switching, the history write site of the target topic at the history time point before the suspension time point can be used as the initial consumption site of the consumption client under the target topic. In this way, compared to the consuming client, even if the producing client switches to the cluster after switching to write the message to the cluster after switching, the consuming client can successfully consume the message.

S440, receiving metadata returned by the metadata request node aiming at the metadata request, wherein the metadata is used for representing information of each topic node under the target topic in the node set.

S450, determining a current service node from each topic node based on the metadata, and connecting the current service node to the current service node so as to switch from the cluster before switching to the cluster after switching.

And S460, waking up the suspended consumption thread to send a consumption request for consuming the message from the initial consumption site to the connected current service node through the awakened consumption thread.

Wherein, the suspended consuming thread is awakened, so that a consuming request for consuming the message from the initial consuming site can be sent to the connected current service node through the awakened consuming thread, thereby ensuring the integrity of the consuming message.

According to the technical scheme of the embodiment of the invention, a consumption thread is suspended, a suspension time point when the consumption thread is suspended is recorded, and then a history write site of a target topic on a history time point before the suspension time point is used as an initial consumption site of a consumption client under the target topic; therefore, after the consumption thread is awakened, the message can be consumed from the initial consumption site through the consumption thread, and the integrity of the consumption message is ensured.

In order to better understand the specific implementation steps of the consuming client in the cluster switching process as a whole, the following describes an exemplary embodiment with reference to specific examples. Illustratively, referring to the cluster switching example shown in fig. 6a and the metadata updating example shown in fig. 6b, the following will explain the two examples in combination. In particular, the method comprises the following steps of,

1) Monitoring message backlog of cluster before switching

The consuming client continuously pulls the message from the current connection cluster, namely continuously sends a message pulling request to the current connection cluster. The current cluster identifier of the current configuration is obtained from the configuration center, and whether the cluster is switched is determined based on the current cluster identifier, and the specific determination process is similar to the above example and is not described herein again. Under the condition that the cluster switching event is detected, in response to the cluster switching event, in order to ensure that the consumption message is not lost, the message backlog condition of the cluster before switching can be calculated firstly. In particular, the method comprises the following steps of,

firstly, aiming at a partition corresponding to a consumption client under an original theme of a cluster before switching, obtaining an original writing site of the partition, then obtaining an original consumption site submitted by the consumption client through a coordination node, if the original writing site minus the original consumption site is not 0 (namely backlog information exists), temporarily not executing metadata updating operation, and starting updating metadata when the difference value of the original writing site minus the original consumption site is 0.

2) Preparing to update metadata

Before the metadata update is performed, a cluster switch time point may be recorded, which may be used to reset the starting consumption site of the consuming client after the metadata update. In practical applications, the cluster switching time point may be a suspension time point of the consuming thread. In addition, before requesting to update the metadata, the consuming client may request to leave the consuming group by sending a leave group request to the coordinating node, thereby suspending the current consuming thread, and after waiting for the metadata update, reallocating the partitions by the coordinating node for each consuming client within the consuming group. Further, the cached node list is updated, and this updating step is similar to the updating step in the above example and is not described again here.

3) Updating metadata

First, it is determined whether there is currently a pending, i.e., processing, consumption request for requesting that messages be consumed on the cluster before the handover (i.e., the currently connected cluster). If yes, the metadata updating operation is directly quitted, and the consumption requests are processed and then the metadata updating operation is executed. In the case of no pending consumption request, the updating of the coordinating node is performed: and the consumption client selects the node with the least connection quantity from the updated node list as a coordination node searching node, initiates a coordination node searching request to the coordination node searching node, and tries to acquire a coordination node corresponding to the switched cluster. If the acquisition is successful, continuing to perform the next metadata updating operation; if the acquisition time of the configured coordination node is exceeded and the acquisition is not successful, the exception is thrown out, and the metadata updating operation is quitted. After the coordination node is updated, an attempt may be made to initiate a metadata request, so as to update the cached metadata based on the requested metadata, and this update process is similar to the update process in the foregoing example and is not described herein again.

4) Resetting the initial consumption site and waking up the consumption thread

After the metadata updating is completed, the coordination node redistributes partitions for all consumption clients in the consumption group to consume; the consumption client can use the recorded historical write-in site corresponding to the historical time point 5 minutes before the cluster switching time point as an initial consumption site (namely an initial pull site), so that the completeness of the pull message of the consumption client is ensured. After the reset completes the initial pull site, the consuming thread may be awakened to begin consuming messages from the initial pull site.

In order to verify the effectiveness of the technical scheme, an application scene of simultaneously switching the production client and the consumption client is tested on the online test cluster, and the test proves that the production client can be automatically, rapidly and stably switched to the switched cluster without restarting; and the consumption client is switched to the cluster after switching after processing the message backlog under the cluster before switching, and the consumption client can restart to pull the message on the cluster after switching for consumption before resetting the initial consumption site to 5 minutes.

Fig. 7 is a block diagram of a cluster switching device provided in an embodiment of the present invention, where the cluster switching device is configured to execute a cluster switching method provided in any of the above embodiments. The device and the cluster switching method in the embodiments belong to the same inventive concept, and details that are not described in detail in the embodiment of the cluster switching device may refer to the embodiment of the cluster switching method. Referring to fig. 7, the apparatus is configured at a production client, where the production client is currently connected to a cluster before switching and corresponds to a target topic in a cluster after switching, and the apparatus may specifically include: a metadata request first sending module 510, a metadata first receiving module 520, and a cluster first switching module 530. Wherein the content of the first and second substances,

a metadata request first sending module 510, configured to, in response to a cluster switching event, obtain a node set of a switched cluster, obtain a metadata request node from the node set, and send a metadata request to the metadata request node;

a metadata first receiving module 520, configured to receive metadata returned by a metadata request node for a metadata request, where the metadata is used to represent information of each topic node in a node set that is located under a target topic;

and the cluster first switching module 530 is configured to determine a current service node from the topic nodes based on the metadata, and connect to the current service node to switch from the cluster before switching to the cluster after switching.

Optionally, the cluster switching apparatus may further include:

the metadata acquisition module is used for acquiring cached metadata, wherein the cached metadata is associated with an original theme corresponding to the target theme in the cluster before switching;

the metadata updating module is used for updating the cached metadata based on the received metadata;

the cluster first switching module 530 may include:

and the current service node first determining unit is used for determining the current service node from all the subject nodes based on the cached metadata.

Optionally, the production client corresponds to a production thread, and the cluster switching apparatus may further include:

the production thread suspension module is used for suspending the production thread;

a cluster identifier before switching acquisition module, configured to acquire a cluster identifier before switching of a cluster before switching;

the cluster identifier comparison module is used for determining a switched cluster identifier of the switched cluster based on the metadata and comparing the switched cluster identifier with a cluster identifier before switching under the condition of successfully receiving the metadata;

and the production thread awakening module is used for awakening the suspended production thread under the condition that the cluster identifier after switching is determined to be inconsistent with the cluster identifier before switching according to the obtained comparison result so as to send a production request to the connected current service node through the awakened production thread.

Optionally, the metadata requesting the first sending module 510 may include:

the cluster switching event response unit is used for responding to a cluster switching event under the condition that the cluster switching event from the configuration center is detected, wherein the cluster switching event is triggered when the configuration in the configuration center is changed;

and the node set first acquisition unit is used for acquiring the switched cluster identifier of the switched cluster configured currently from the configuration center and acquiring the node set of the switched cluster based on the switched cluster identifier.

Optionally, on the basis of the above apparatus, the apparatus may further include:

and the cluster identifier switching module is used for responding to a cluster abnormal event before the cluster switching event from the configuration center is detected, and changing the cluster identifier before switching of the cluster which is currently configured in the configuration center into the cluster identifier after switching.

Optionally, the metadata requesting first sending module 510 may include:

a connection number determination unit configured to determine, for each connectable node in the node set, a connection number of connected clients that have been connected to the connectable node;

a metadata request node obtaining unit configured to use a connectable node with the smallest connection number in the node set as a metadata request node;

and the metadata request first sending unit is connected to the metadata request node and used for sending the metadata request to the connected metadata request node.

The cluster switching device provided by the embodiment of the invention is used for responding to a cluster switching event by a production client currently connected to a cluster before switching through a metadata request first sending module, acquiring a node set of the switched cluster, then acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node; receiving metadata returned by a metadata request node for a metadata request through a metadata first receiving module; since the metadata can be used to represent information of each topic node in the node set under a target topic, where the target topic is a topic corresponding to the production client in the post-switching cluster, a current service node is determined from each topic node based on the metadata through the first cluster switching module, and is connected to the current service node, so that the production client is switched from the pre-switching cluster to the post-switching cluster. According to the device, the metadata used for representing the information of the subject nodes related to the production client in the switched cluster is requested, and the production client is automatically switched and connected to the switched cluster from the cluster before switching on the basis of the metadata, so that the automatic cluster switching effect of the production client is achieved under the condition that the production client does not need to be restarted, namely the production client is continuously operated.

The cluster switching device provided by the embodiment of the invention can execute the cluster switching method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

It should be noted that, in the embodiment of the cluster switching device, each included unit and module are only divided according to functional logic, but are not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific names of the functional units are only for the convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.

Fig. 8 is a block diagram of another cluster switching device provided in an embodiment of the present invention, where the device is configured to execute the cluster switching method provided in any of the above embodiments. The device and the cluster switching method in the foregoing embodiments belong to the same inventive concept, and details that are not described in detail in the embodiments of the cluster switching device may refer to the embodiments of the cluster switching method. Referring to fig. 8, the apparatus may be configured to a consuming client, where the consuming client is currently connected to a cluster before switching and corresponds to a target topic in the cluster after switching, and the apparatus may specifically include: a metadata request second sending module 610, a metadata second receiving module 620, and a cluster second switching module 630. Wherein the content of the first and second substances,

a metadata request second sending module 610, configured to, in response to the cluster switching event, obtain a node set of the switched cluster, obtain a metadata request node from the node set, and send the metadata request to the metadata request node;

a metadata second receiving module 620, configured to receive metadata returned by a metadata request node for the metadata request, where the metadata is used to represent information of each topic node in the node set that is located under the target topic;

and the cluster second switching module 630 is configured to determine a current service node from the topic nodes based on the metadata, and connect to the current service node to switch from the cluster before switching to the cluster after switching.

Optionally, the consuming client corresponds to a consuming thread, and the cluster switching method may further include:

the suspension time point recording module is used for suspending the consumption thread and recording the suspension time point when the consumption thread is suspended;

an initial consumption site obtaining module, configured to write a history of the target topic at a history time point before the suspension time point into a site, as an initial consumption site under the target topic;

and the consumption thread awakening module is used for awakening the suspended consumption thread so as to send a consumption request for consuming the message from the initial consumption site to the connected current service node through the awakened consumption thread.

Optionally, the metadata request second sending module 610 may include:

the original consumption site acquisition unit is used for acquiring an original writing site and an original consumption site under an original theme aiming at the original theme corresponding to the target theme under the cluster before switching;

and the node set second acquisition unit is used for acquiring the node set of the switched cluster under the condition that all messages under the original theme are determined to be consumed according to the original writing site and the original consumption site.

Optionally, for the consumption group to which the consumption client belongs, the cluster switching apparatus may further include:

the coordination node searching request sending module is used for acquiring a coordination node searching node from the node set and sending a coordination node searching request to the coordination node searching node;

the coordination node receiving module is used for receiving a coordination node returned by a coordination node searching node aiming at a coordination node searching request so as to acquire a partition under a target theme distributed for a consumption client from the coordination node, wherein the coordination node belongs to a node set and corresponds to a consumption group;

the cluster second switching module 630 may include:

and the current service node second determining unit is used for determining the current service node belonging to the partition from the subject nodes based on the metadata.

In the cluster switching device provided in the embodiment of the present invention, for a consuming client currently connected to a cluster before switching, the consuming client responds to a cluster switching event through a metadata request second sending module, acquires a node set of a switched cluster, acquires a metadata request node from the node set, and sends a metadata request to the metadata request node; receiving metadata returned by the metadata request node for the metadata request through a metadata second receiving module; since the metadata may be used to represent information of each topic node in the node set under a target topic, where the target topic is a topic corresponding to the consuming client in the post-switching cluster, a current service node is determined from each topic node based on the metadata through the cluster second switching module, and is connected to the current service node, so that the consuming client is switched from the pre-switching cluster to the post-switching cluster. According to the device, the metadata used for representing the information of the topic nodes related to the consumption client in the switched cluster is requested, and the consumption client is automatically switched and connected to the switched cluster from the cluster before switching on the basis of the metadata, so that the effect of automatic cluster switching of the consumption client is achieved under the condition that the consumption client does not need to be restarted, namely the consumption client is continuously operated.

FIG. 9 illustrates a schematic diagram of a production terminal 10 or a consumer terminal 10 (hereinafter collectively referred to as terminal 10) that may be used to implement embodiments of the present invention. Terminals are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Terminals may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 9, the terminal 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the terminal 10 can also be stored. The processor 11, the ROM12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

A number of components in the terminal 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the terminal 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as the cluster switching method.

In some embodiments, the cluster switching method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed on the terminal 10 via the ROM12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cluster switching method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the cluster switching method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a terminal having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the terminal. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cluster switching method is applied to a production client, wherein the production client is currently connected to a cluster before switching and corresponds to a target theme in the cluster after switching, and the method comprises the following steps:

responding to a cluster switching event, acquiring a node set of the switched cluster, acquiring a metadata request node from the node set, and sending a metadata request to the metadata request node;

receiving metadata returned by the metadata request node for the metadata request, wherein the metadata is used for representing information of each topic node in the node set under the target topic;

and determining a current service node from the topic nodes based on the metadata, and connecting the current service node to the current service node so as to switch from the cluster before switching to the cluster after switching.

2. The method of claim 1, further comprising:

obtaining cached metadata, wherein the cached metadata is associated with an original topic corresponding to the target topic in the cluster before switching;

updating the cached metadata based on the received metadata;

the determining a current service node from the respective subject nodes based on the metadata includes:

determining a current service node from the respective subject nodes based on the cached metadata.

3. The method of claim 1, wherein the production client corresponds to a production thread, the method further comprising:

suspending the production thread;

acquiring a cluster identifier before switching of the cluster before switching;

under the condition that the metadata is successfully received, determining a post-switching cluster identifier of the post-switching cluster based on the metadata, and comparing the post-switching cluster identifier with the pre-switching cluster identifier;

and awakening the suspended production thread under the condition that the cluster identifier after switching is determined to be inconsistent with the cluster identifier before switching according to the obtained comparison result, so as to send a production request to the connected current service node through the awakened production thread.

4. The method of claim 1, wherein the responding to the cluster switch event comprises:

responding to a cluster switching event in the case of detecting the cluster switching event from a configuration center, wherein the cluster switching event is triggered when the configuration in the configuration center is changed;

the obtaining the node set of the switched cluster includes:

and acquiring a switched cluster identifier of the switched cluster which is currently configured from the configuration center, and acquiring a node set of the switched cluster based on the switched cluster identifier.

5. The method of claim 4, further comprising, prior to the detecting the cluster handover event from the configuration center:

changing a pre-switch cluster identity of the currently configured pre-switch cluster within the configuration center to the post-switch cluster identity in response to a cluster exception event.

6. The method of claim 1, wherein the obtaining metadata from the set of nodes requests a node, comprising:

determining, for each connectable node in the set of nodes, a number of connections of connected clients that have been connected on the connectable node;

taking the connectable node with the minimum connection number in the node set as a metadata request node;

the sending the metadata request to the metadata request node includes:

connecting to the metadata request node, and sending a metadata request to the metadata request node connected thereto.

7. A cluster switching method is applied to a consuming client, wherein the consuming client is currently connected to a cluster before switching and corresponds to a target theme in the cluster after switching, and the method comprises the following steps:

8. The method of claim 7, wherein the consuming client corresponds to a consuming thread, the method further comprising:

suspending the consuming thread and recording a suspension time point when suspending the consuming thread;

taking a historical writing site of the target theme on a historical time point before the suspension time point as an initial consumption site under the target theme;

waking up the suspended consumption thread to send a consumption request for consuming a message from the initial consumption site to the current service node connected through the awakened consumption thread.

9. The method of claim 7, wherein the obtaining the node set of the switched cluster comprises:

aiming at an original theme corresponding to the target theme in the cluster before switching, acquiring an original writing site and an original consuming site in the original theme;

and under the condition that all messages under the original topic are determined to be consumed according to the original writing sites and the original consumption sites, acquiring the node set of the switched cluster.

10. The method of claim 7, wherein for a consumption group to which the consumption client belongs, the method further comprises:

acquiring a coordination node searching node from the node set, and sending a coordination node searching request to the coordination node searching node;

receiving a coordination node returned by the coordination node searching node aiming at the coordination node searching request so as to acquire a partition under the target theme distributed for the consumption client from the coordination node, wherein the coordination node belongs to the node set and corresponds to the consumption group;

determining a current service node belonging to the partition from the respective subject nodes based on the metadata.

11. A cluster switching apparatus configured to a production client currently connected to a cluster before switching and corresponding to a target topic in a cluster after switching, the apparatus comprising:

a metadata request first sending module, configured to respond to a cluster switching event, obtain a node set of the switched cluster, obtain a metadata request node from the node set, and send a metadata request to the metadata request node;

a metadata first receiving module, configured to receive metadata returned by the metadata request node for the metadata request, where the metadata is used to represent information of each topic node in the node set that is located under the target topic;

and the cluster first switching module is used for determining a current service node from the theme nodes based on the metadata and connecting to the current service node so as to switch from the cluster before switching to the cluster after switching.

12. A cluster switching apparatus configured to a consuming client currently connected to a cluster before switching and corresponding to a target topic under the cluster after switching, the apparatus comprising:

a metadata request second sending module, configured to, in response to a cluster switching event, obtain a node set of the switched cluster, obtain a metadata request node from the node set, and send a metadata request to the metadata request node;

a second metadata receiving module, configured to receive metadata returned by the metadata request node for the metadata request, where the metadata is used to represent information of each topic node in the node set that is located under the target topic;

13. A terminal, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the cluster switching method of any one of claims 1-6 or the cluster switching method of any one of claims 7-10.

14. A computer-readable storage medium storing computer instructions for causing a processor to perform the cluster switching method of any one of claims 1-10 when executed.