CN117118982A

CN117118982A - Message transmission method, device, medium and equipment based on cloud primary multi-cluster

Info

Publication number: CN117118982A
Application number: CN202311077202.1A
Authority: CN
Inventors: 陈健斌
Original assignee: Hangzhou Bodun Xiyan Technology Co ltd
Current assignee: Hangzhou Bodun Xiyan Technology Co ltd
Priority date: 2023-08-24
Filing date: 2023-08-24
Publication date: 2023-11-24

Abstract

The application provides a message transmission method based on cloud native multi-cluster, a message transmission device based on cloud native multi-cluster, a computer readable storage medium and electronic equipment, and relates to the technical field of cloud native.

Description

Message transmission method, device, medium and equipment based on cloud primary multi-cluster

Technical Field

The application relates to the technical field of cloud primordia, in particular to a message transmission method based on cloud primordia multi-cluster, a message transmission device based on cloud primordia multi-cluster, a computer readable storage medium and electronic equipment.

Background

The cloud protogenesis can be understood as a behavior mode and a design concept, and the behavior or mode capable of improving the utilization rate and the application delivery efficiency of resources on the cloud can be called as the cloud protogenesis. Cloud proto is supported by a range of technologies, such as containers, service grids, micro services, immutable infrastructure, declarative APIs.

In the messaging process, some network architectures are typically implemented using cloud-native technology, which may be used to implement cloud-native based messaging. Generally, such network architectures require the provision of multiple server clusters, which are done in order to achieve message disaster recovery. When one server is down, traffic is sent to other servers in the plurality of server clusters so that the other servers can continuously deliver messages.

In this case, although the number of server clusters is large, so that the disaster recovery purpose can be achieved, the message types supported by each server cluster in the plurality of server clusters are the same, which may cause limitation in application range of the existing message passing scheme.

It should be noted that the information disclosed in the above background section is only for enhancing the understanding of the background of the application and thus may include information that does not form a related art that is already known to those of ordinary skill in the art.

Disclosure of Invention

The application aims to provide a message transmission method based on cloud native multi-cluster, a message transmission device based on cloud native multi-cluster, a computer readable storage medium and electronic equipment, wherein a target server cluster which can support a message type can be selected from a plurality of server clusters based on the message type corresponding to the target message, the target message is processed through target message middleware of the target server cluster, higher compatibility is provided for clusters configured in different message middleware, and the diversified server clusters can further improve the applicability of the diversified target message, so that the application range of a message transmission scheme is further improved.

Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.

According to an aspect of the present application, there is provided a message transmission method based on cloud native multi-cluster, the method comprising:

receiving a target message sent by a first client, and determining a message type corresponding to the target message;

determining a target server cluster from at least two server clusters based on the message type; wherein at least two server clusters exist in the at least two server clusters corresponding to different message middleware;

the target message is sent to the target server cluster so that the target server cluster runs a corresponding target message middleware to deliver the target message.

In one embodiment of the present application, the method further comprises:

creating at least one message topic in response to a user operation on a control panel with integrated creation capability, and storing metadata corresponding to the at least one message topic in a memory;

creating at least two server clusters;

the metadata is used for representing server clusters respectively corresponding to at least one message theme, and the at least one message theme comprises a target message theme corresponding to the target message.

In one embodiment of the present application, determining a target server cluster from at least two server clusters based on a message type includes:

reading target metadata corresponding to a target message theme from a memory;

determining an intermediate server cluster corresponding to the target metadata from at least two server clusters;

a target server cluster is determined from among the intermediary server clusters based on the message type.

In one embodiment of the present application, determining a target server cluster from among the intermediary server clusters based on the message type includes:

determining a target message middleware supporting the message type from message middleware corresponding to each intermediate server cluster;

and determining a reference server cluster corresponding to the target message middleware from the intermediate server clusters as a target server cluster.

In a specific embodiment of the present application, if the number of reference server clusters is greater than 1, determining, from among the intermediate server clusters, a reference server cluster corresponding to the target message middleware as the target server cluster includes:

determining each reference server cluster corresponding to the target message middleware from each intermediate server cluster;

Determining the flow load states corresponding to the reference server clusters respectively;

a target server cluster is determined from the reference server clusters based on the traffic load status.

In one embodiment of the present application, the method further comprises:

performing capability expansion on a specific server cluster in response to capability expansion operation on the specific server cluster in at least two server clusters;

the number of message types supported by the specific server cluster after the capability expansion is larger than the number of message types supported by the specific server cluster before the capability expansion.

In one embodiment of the present application, sending a target message to a target server cluster includes:

determining a cluster state corresponding to a target server cluster;

and if the cluster state characterization cluster is available, sending the target message to the target server cluster.

In one embodiment of the present application, the method further comprises:

if the cluster state characterization cluster is not available, the target message is sent to a standby cluster of the at least two server clusters.

In one embodiment of the present application, the method further comprises:

when the availability of the cluster state representation cluster is detected again, the received new message is sent to the target server cluster; wherein the new message corresponds to the same message type as the target message.

In one embodiment of the present application, the method further comprises:

and when the number of the unavailable servers in the target server cluster is detected to be larger than the preset number, judging that the cluster state representation cluster is unavailable.

In one embodiment of the present application, the method further comprises:

receiving a message pulling request for pulling a message set; wherein the message set includes a target message and other messages;

responding to the message pulling request, pulling a target message from the target message middleware, and pulling other messages from the message middleware of other server clusters to obtain a message set;

the set of messages is sent to the second client.

In one embodiment of the present application, sending a target message to a target server cluster, so that the target server cluster runs a corresponding target message middleware to deliver the target message, includes:

if the message type is the delay message type, sending the target message to the target server cluster, so that the target message middleware operated by the target server cluster transmits the target message after a preset time length;

and if the message type is the real-time message type, sending the target message to the target server cluster, so that the target message middleware operated by the target server cluster transmits the target message in real time.

In one embodiment of the present application, the method further comprises:

acquiring operation parameters corresponding to at least two server clusters respectively; wherein the operating parameters include at least one of: delay information, error reporting information, and synchronization efficiency information;

and adjusting the flow load weights corresponding to at least two server clusters respectively based on the operation parameters to obtain new flow load weights corresponding to each server cluster.

and determining a target server cluster from at least two server clusters based on the new traffic load weight and the message type corresponding to each server cluster.

According to an aspect of the present application, there is provided a message transmission apparatus based on cloud native multi-cluster, the apparatus comprising:

the message type determining unit is used for receiving the target message sent by the first client and determining the message type corresponding to the target message;

the cluster selection unit is used for determining a target server cluster from at least two server clusters based on the message type; wherein at least two server clusters exist in the at least two server clusters corresponding to different message middleware;

And the message sending unit is used for sending the target message to the target server cluster so that the target server cluster runs corresponding target message middleware to transfer the target message.

In one embodiment of the present application, the method further comprises:

a user operation response unit for creating at least one message topic in response to a user operation for a control panel having a comprehensive creation capability, and storing metadata corresponding to the at least one message topic in a memory; creating at least two server clusters;

In a specific embodiment of the present application, the cluster selection unit determines a target server cluster from at least two server clusters based on the message type, including:

reading target metadata corresponding to a target message theme from a memory;

In one embodiment of the present application, the cluster selection unit determines a target server cluster from among the intermediate server clusters based on the message type, including:

In a specific embodiment of the present application, if the number of reference server clusters is greater than 1, from among the intermediate server clusters, the cluster selection unit determines, as the target server cluster, the reference server cluster corresponding to the target message middleware, including:

In one embodiment of the present application, the method further comprises:

a capability expansion unit for performing capability expansion on a specific server cluster in response to a capability expansion operation for the specific server cluster in the at least two server clusters;

In one embodiment of the present application, a message sending unit sends a target message to a target server cluster, including:

determining a cluster state corresponding to a target server cluster;

In one embodiment of the present application, wherein:

and the message sending unit is also used for sending the target message to the standby cluster in the at least two server clusters if the cluster state representation cluster is not available.

In one embodiment of the present application, wherein:

the message sending unit is further used for sending the received new message to the target server cluster when the availability of the cluster state representation cluster is detected again; wherein the new message corresponds to the same message type as the target message.

In one embodiment of the present application, the method further comprises:

and the state judging unit is used for judging that the cluster state representation cluster is not available when the number of the unavailable servers in the target server cluster is detected to be larger than the preset number.

In one embodiment of the present application, the method further comprises:

a request receiving unit for receiving a message pulling request for pulling the message collection; wherein the message set includes a target message and other messages;

the request response unit is used for responding to the message pulling request, pulling the target message from the target message middleware, and pulling other messages from the message middleware of other server clusters to obtain a message set; the set of messages is sent to the second client.

In one embodiment of the present application, the message sending unit sends the target message to the target server cluster, so that the target server cluster runs corresponding target message middleware to transfer the target message, including:

In one embodiment of the present application, the method further comprises:

The operation parameter acquisition unit is used for acquiring operation parameters corresponding to at least two server clusters respectively; wherein the operating parameters include at least one of: delay information, error reporting information, and synchronization efficiency information;

and the traffic load weight updating unit is used for adjusting the traffic load weights corresponding to at least two server clusters respectively based on the operation parameters to obtain new traffic load weights corresponding to the server clusters.

According to an aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described above.

According to an aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of any of the above.

According to an aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of any of the above via execution of executable instructions.

Exemplary embodiments of the present application may have some or all of the following advantages:

in the message transmission method based on the cloud native multi-cluster provided by the example embodiment of the application, the target server cluster capable of supporting the message type can be selected from a plurality of server clusters based on the message type corresponding to the target message, the target message is processed through the target message middleware of the target server cluster, higher compatibility is provided for the clusters configured in different message middleware, and the diversified server clusters can further improve the applicability to diversified target messages, so that the application range of a message transmission scheme is further improved.

In addition, because the target server cluster capable of supporting the message type can be selected from the plurality of server clusters based on the message type corresponding to the target message, a specific message middleware is not required to be known for the first client, the corresponding target server cluster can be determined to process the target message no matter any message type, and the message middleware shielding for the client is realized.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.

FIG. 1 schematically illustrates a flow diagram of a method of cloud native multi-cluster based message transmission according to one embodiment of the application;

FIG. 2 schematically illustrates a proxy agent deployment scenario diagram, according to one embodiment of the present application;

FIG. 3 schematically illustrates a schematic diagram of a side car agent deployment scenario in accordance with one embodiment of the present application;

FIG. 4 schematically illustrates an architecture diagram for creating a message topic and server cluster in accordance with one embodiment of the application;

FIG. 5 schematically illustrates a primary and backup cluster schematic according to one embodiment of the application;

FIG. 6 schematically illustrates a flow switching scheme according to one embodiment of the application;

FIG. 7 schematically illustrates a schematic diagram of traffic balancing loads for clusters according to one embodiment of the application;

FIG. 8 schematically illustrates a flow load weight adjusted schematic diagram in accordance with one embodiment of the application;

FIG. 9 schematically illustrates a flow chart of a method of cloud native multi-cluster based message transmission according to another embodiment of the application;

fig. 10 schematically illustrates a structural diagram of a message transmission apparatus based on a cloud native multi-cluster according to an embodiment of the present application;

fig. 11 schematically shows a schematic of a computer system suitable for use in implementing an embodiment of the application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known aspects have not been shown or described in detail to avoid obscuring aspects of the application.

Referring to fig. 1, fig. 1 schematically illustrates a flowchart of a message transmission method based on cloud native multicluster according to one embodiment of the application. As shown in fig. 1, the method comprises the steps of: step S110 to step S130. The steps S110 to S130 may be implemented by a proxy server.

Step S110: and receiving the target message sent by the first client, and determining the message type corresponding to the target message.

Step S120: determining a target server cluster from at least two server clusters based on the message type; wherein at least two server clusters among the at least two server clusters correspond to different message middleware.

Step S130: the target message is sent to the target server cluster so that the target server cluster runs a corresponding target message middleware to deliver the target message.

By implementing the method shown in fig. 1, a target server cluster capable of supporting the message type can be selected from a plurality of server clusters based on the message type corresponding to the target message, and the target message is processed through the target message middleware of the target server cluster, so that higher compatibility is provided for clusters configured in different message middleware, and the diversified server clusters can further improve the applicability to diversified target messages, thereby improving the application range of a message transmission scheme. In addition, because the target server cluster capable of supporting the message type can be selected from the plurality of server clusters based on the message type corresponding to the target message, a specific message middleware is not required to be known for the first client, the corresponding target server cluster can be determined to process the target message no matter any message type, and the message middleware shielding for the client is realized.

Next, the above steps of the present exemplary embodiment will be described in more detail.

In order to implement the technical solution of the present application, one or more technologies of supporting cloud native may be utilized in actual requirements, such as containers, service grids, micro services, immutable infrastructures, declarative APIs, and the embodiments of the present application are not limited.

The container, which may enable application services to be separated from the underlying architecture, enables full portability (i.e., the ability to run applications on any operating system or environment), and when an application is made up of many separate components, can be assigned one container for each component.

Service Mesh (Service Mesh), an infrastructure layer for handling inter-Service communications, has complex Service topologies for cloud-native applications, in which the Service Mesh is responsible for achieving reliable delivery of requests, and in practice is typically implemented as a set of lightweight network agents deployed with applications but transparent to the applications. The Service Mesh refers to a form that is presented by applying a specific technology (e.g., a sidecar) instead of a specific technology.

Further, dividing the functionality of an application into separate processes may be considered a sidecar mode. The sidecar design mode allows many functions to be added to an application without requiring the configuration and code of additional third party components. In a software architecture, a sidecar application is connected to and extends or enhances functionality for a parent application. The application sidecar can reduce the complexity of the micro-service code by abstracting the common infrastructure associated with the functionality to a different layer, and can eliminate the writing of third party component configuration files and code to reduce the code repetition in the micro-service architecture, and can reduce the coupling of the application code and the underlying platform. The micro service (may also be referred to as a micro service architecture) is a software architecture mode, which is used to construct an application into a series of autonomous services that are divided into modules according to the service domain. In a micro-service architecture, each service is self-contained and implements a single business function. Briefly, it will be understood that the system is divided into a plurality of subsystems by service, each of which is complete and independently operable, and communication between the subsystems may be accomplished via HTTP protocol, or via message queues (e.g., roocketmq, kafaka, etc.), or in other forms.

An immutable infrastructure is a self-contained, self-describing object that can be completely migrated in different environments.

Declarative design (Declaritive) refers to the operations required to declare a desired state and reach that state based on submitting a custom API object.

As an alternative embodiment, before step S110, the following steps may be further included:

step S010: at least one message topic is created in response to a user operation directed to a control panel with integrated creation capabilities and metadata corresponding to the at least one message topic is stored in memory.

Step S020: at least two server clusters are created.

The steps S010-S020 are implemented, a control panel with strong comprehensiveness can be provided for the user, so that the user can finish the creation of the message theme and the server cluster in one step, the creation transaction is not required to be finished through various proxy server interfaces, and the creation efficiency of the server cluster and the message theme can be improved.

In particular, the present application provides a user with a control panel with comprehensive creation capabilities that can be used not only to create message topics (topics) but also to create server clusters (clusters). The control panel may be implemented as an interactive user interface.

The message theme (topic) is a logical concept, and may be further subdivided into a plurality of partitions, where a Partition belongs to only a single message theme, and generally, the partitions will also be referred to as a theme Partition (topic-Partition), where the messages contained in different partitions under the same message theme are different, and the Partition may be regarded as an appendable log (log) file at the storage level, and when the message is appended to the Partition log or file, a specific offset (offset) is allocated to each Partition, where the offset is a unique identifier of the message in the Partition. The message topics (topic) of the logic level can be physically distributed in different server clusters, that is, the messages of the same message topic (topic) can be stored in different server clusters, so that the reliability and throughput of message delivery can be improved.

Wherein a server cluster (cluster) refers to a server cluster composed of a plurality of servers (also called nodes) for achieving high availability and load balancing. Different server clusters (clusters) may implement different message middleware (a messaging system for handling messaging and event publication subscriptions in a distributed system, as may also be understood), such as activemq, rocketmq, rabbitmq, kafka. The activemq is an open-source message middleware, realizes Java Message Service (JMS) specification, supports various message modes including queues, topics and publishing/subscribing, and is suitable for scenes requiring complex message routing and processing. The locktmq is used as a distributed message middleware, can realize high-throughput and low-delay message transmission, supports the characteristics of multi-tenant, multi-data center, disaster recovery backup and the like, and is suitable for large-scale real-time message processing. The rubbitmq is a message middleware using Advanced Message Queuing Protocol (AMQP), has flexible message routing and queue management functions, and is suitable for application scenarios of asynchronous communication and task queues. kafka is a distributed streaming platform for building real-time data streaming applications and event-driven architecture, supporting high throughput, persistent storage and streaming data processing, and is suitable for the fields of logging, monitoring, data analysis, etc. The messages in kafka are categorized in units of message topics (topic), the producer (producer) is responsible for sending the messages to a specific topic, and the consumer (consumer) is responsible for subscribing to the topic and pulling the corresponding messages; wherein each message sent to the kafka server cluster needs to specify a topic. kafka guarantees the ordering of messages within a partition by means of offset, which does not span the partition, i.e., kafka guarantees that the partitions are ordered rather than the subject is ordered.

In the message middleware, a server instance (brooker) is included that carries message queues and handles message passing. In a server cluster, multiple brookers may work together to achieve high availability, load balancing, and distribution of messages together. Each broker is responsible for managing a number of partitions or queues and handling messaging operations associated with those partitions or queues.

Typically, one server node may run multiple hooks to provide better load balancing and high availability. Each server node in the server cluster does not necessarily correspond to the same topic, and one server node may carry different partitions of multiple topics to achieve load balancing and message distribution. Thus, the topic messages stored by each server node may be different, depending on the partition and the policy of the message distribution. If the same topic is corresponding between different server clusters, the topic messages stored therein are not necessarily identical. Different clusters of servers of message middleware may operate under different environments, configurations, and partition distribution policies, resulting in potentially different storage and distribution of messages. Although they may all process the same type of message, the storage status and partition data for each server cluster may be different.

It should be noted that multiple servers (also referred to as multiple nodes) in a server cluster (cluster) may collectively carry one or more topics to implement distributed messaging.

In addition, when a user operation for a control panel having integrated creation capability is detected, at least one message topic, at least two server clusters, and metadata respectively corresponding to the at least one message topic may be created in response to the user operation.

The metadata may be used to record clusters to which at least one message topic corresponds, and it may be understood that the metadata specifies in which cluster(s) the message(s) of the corresponding message topic should be stored. For example, after the user creates server cluster a, server cluster B, server cluster C, message topic T at the control panel, the metadata may specify that the target message corresponding to message topic T is stored into server cluster a, server cluster B.

And, the control panel may be adapted for any agent deployment, such as, for example, a sidecar agent deployment, a proxy agent deployment, etc. The sidecar and proxy may be implemented as proxy servers for providing functions such as inter-service communication, monitoring, and security, and may be understood as modules/components abstracted from the first client architecture, so as to track and maintain the entire first client. The first client here is understood to mean an application with an interactive interface which is provided for downloading to the user side.

In particular, proxy is a proxy service for receiving a target message for a first client and proxy the target message into a server cluster. proxy can realize functions such as information filtering, load balancing, buffering and the like, and the proxy realizes centralized service for serving a plurality of first clients. The sidecar is a special proxy, and is used as a single container, and runs in parallel with an application container in a computing unit (Kubernetes pod) of a container arrangement platform, and corresponds to an auxiliary application; among other things, the container orchestration platform (Kubernetes) can automate the deployment, management, and expansion of containerized applications, and the computing unit (Pod) is the smallest deployable computing unit that can be created and managed in Kubernetes, and that contains one or more containers that share storage, networking, and declarations to run those containers. The sidecar is responsible for handling all traffic inside the first client and provides some of the functions required by the service grid, such as traffic routing, high availability, etc.

The distinction between a sidecar and a proxy is that sidecar serves only a single first client, whereas proxy can serve multiple first clients. That is, the sidecar is in a one-to-one relationship with the first client, and the proxy is in a one-to-many relationship with the first client.

Either the side agent deployment or the proxy agent deployment follows a producer-consumer (consumer) model, in the present application, the producer is referred to by the first client and the consumer is referred to by the second client. In the practical application process, the client can be used as a producer (producer) or a consumer (consumer), when the client needs to send a message, the client is the producer (producer), and when the client needs to pull/consume the message, the client is the consumer (consumer).

For proxy agent deployment, please refer to fig. 2, fig. 2 schematically illustrates a proxy agent deployment diagram according to an embodiment of the present application. As shown in fig. 2, after creating at least one message topic (topic) and at least two server clusters (e.g., kafka-cluster240, locketmq-cluster 250), proxy 230 may receive a target message sent (send) by producer210 and determine a target server cluster from the at least two server clusters based on a message type corresponding to the target message; wherein, at least one message topic (topic) includes a target message topic corresponding to the target message, and the target server cluster may be kafka-cluster240 or rock tmq-cluster250. In turn, proxy 230 may send the target message to a target server cluster that is configured to run corresponding target message middleware (e.g., kafka or locktmq, etc.) and pass the target message into the consumer 220 via proxy 230 as the consumer 220 pulls (pull) the target message.

It should be noted that the number of kafka-cluster240 may be one or more (for example, kafka-cluster1, kafka-cluster2, … …, kafka-cluster N, N is a positive integer), and similarly, the number of locketmq-cluster 250 may be one or more (for example, locketmq-cluster 1, locketmq-cluster 2, … …, locketmq-cluster N, N is a positive integer), which is not limited by the embodiments of the present application. The multiple kafka-cluster240 and multiple locktmq-cluster 250 shown in fig. 2 are only exemplary, and in practical application, the server clusters created by the user may not be limited to the kafka-cluster240 and the locktmq-cluster 250, but may be other server clusters, and the number of server clusters corresponding to the same message middleware may be one or more.

Referring to fig. 3 for a side car agent deployment, fig. 3 schematically illustrates a side car agent deployment according to one embodiment of the application. As shown in fig. 3, after creating at least one message topic (topic) and at least two server clusters (e.g., kafka-cluster330, locketmq-cluster 340), the side 322 in server 2 320 may receive a target message sent (send) by the producer 321 in server 2 320 and determine a target server cluster from the at least two server clusters according to a message type corresponding to the target message; wherein, at least one message topic (topic) includes a target message topic corresponding to the target message, and the target server cluster may be kafka-cluster330 or rock tmq-cluster340. In turn, the side 322 may send the target message to a target server cluster that is configured to run the corresponding target message middleware (e.g., kafka or dockmq, etc.) and pass the target message into the consumer 311 via the side 312 in the server 1 310 as the consumer 311 in the server 1 pulls (pull) the target message.

It should be noted that the number of kafka-cluster330 may be one or more (for example, kafka-cluster1, kafka-cluster2, … …, kafka-cluster N, N is a positive integer), and similarly, the number of the docket-cluster 340 may be one or more (for example, docket-cluster 1, docket-cluster 2, … …, docket-cluster N, N is a positive integer), which is not limited by the embodiment of the present application. The multiple kafka-cluster330 and the multiple locktmq-cluster 340 shown in fig. 3 are only exemplary, and in practical application, the server clusters created by the user may not be limited to the kafka-cluster330 and the locktmq-cluster 340, but may be other server clusters, and the number of server clusters corresponding to the same message middleware may be one or more.

In addition, steps S010 to S020 can be implemented by the architecture shown in fig. 4, fig. 4 schematically showing an architecture diagram for creating a message topic and a server cluster according to one embodiment of the present application. As shown in fig. 4. The user may trigger the control panel 410 having integrated creation capability, and further, in response to a user operation, at least one message topic may be created and metadata corresponding to the at least one message topic may be stored in the distributed memory 420; wherein distributed memory 420 may comprise a plurality of distributed nodes to collectively store metadata. Further, a server cluster kafka-cluster 430 and a server cluster locketmq-cluster 440 may also be created. The server clusters kafka-cluster 430 and server cluster locktq-cluster 440 are only shown as examples, and the types and the numbers of the created server clusters can be personalized in the actual application process.

In step S110, a target message sent by the first client is received, and a message type corresponding to the target message is determined.

Specifically, the target message may include any message content, and for a proxy server (such as proxy server or side proxy server), it is generally only necessary to read a message type (such as delay message type or real-time message type) corresponding to the target message after receiving the target message, without identifying specific message content; the message type can be recorded in the target message or can be identified by analyzing the structure of the target message by the proxy server.

The first client refers to a producer (producer) for sending out a message, and is not specific to a certain client. The first client may send the target message to the proxy server such that the proxy server sends the message to the server cluster based on a weighted random, weighted polling, etc. policy for message processing by message middleware in the server cluster. The message processing manner of the message middleware can include storage, calculation, classification, merging, subscription and the like, and the embodiment of the application is not limited.

Wherein different message middleware may support the same message type or different message types, and embodiments of the present application are not limited. For example, message middleware a supports message types T1 and T3, message middleware B supports message types T2 and T3, and message middleware C supports message type T5.

In step S120, determining a target server cluster from the at least two server clusters based on the message type; wherein at least two server clusters among the at least two server clusters correspond to different message middleware.

In particular, in the process of creating multiple server clusters to implement message delivery, the related art generally creates multiple server clusters corresponding to the same message middleware, so that message disaster tolerance is facilitated. However, this arrangement may result in limited types of messages that can be processed, i.e. the present application may provide at least two server clusters corresponding to different message middleware, and based on the difference in message types supported between message middleware, the differential server clusters may extend the types of messages that can be processed.

The message types are supported by the target message middleware corresponding to the target server cluster. In at least two server clusters, the number of server clusters supporting the message type may be any number, and when there are a plurality of server clusters supporting the message type, a server cluster may be selected as a target server cluster.

Furthermore, it should be noted that at least two server clusters created by the server cluster description may be divided into at least two categories, where the division is based on message middleware. For example, a total of 10 server clusters are created, and depending on the message middleware, the 10 server clusters may be divided into 3 categories, i.e. there are 3 server clusters. Wherein the first server cluster comprises 2 server clusters each corresponding to message middleware kafka, the second server cluster comprises 5 server clusters each corresponding to message middleware locktmq, and the third server cluster comprises 3 server clusters each corresponding to message middleware activemq.

As an alternative embodiment, determining the target server cluster from the at least two server clusters based on the message type includes: reading target metadata corresponding to a target message theme from a memory; determining an intermediate server cluster corresponding to the target metadata from at least two server clusters; a target server cluster is determined from among the intermediary server clusters based on the message type. Therefore, a part of server clusters can be selected as the intermediate server clusters based on the target metadata, then the target server clusters are selected from the intermediate server clusters based on the message types, the matching degree of the selected target server clusters and the target message is higher, accurate target message distribution can be realized, the target message is prevented from being distributed to unmatched server clusters, and further the situation that the server clusters do not support the target message is avoided.

In particular, the target metadata is used to specifically specify a storage location of the target message topic, which may be represented as a server cluster identification (e.g., address), a server identification (e.g., address), a specific storage area, and so forth. Determining an intermediate server cluster based on the storage position specified by the target metadata, wherein the number of the intermediate server clusters can be one or more, and if the number of the intermediate server clusters is 1, directly determining the intermediate server cluster as the target server cluster; if the number of intermediary server clusters is greater than 1, a target server cluster is determined from among the intermediary server clusters based on the message type.

As an alternative embodiment, determining the target server cluster from the intermediate server clusters based on the message type includes: determining a target message middleware supporting the message type from message middleware corresponding to each intermediate server cluster; and determining a reference server cluster corresponding to the target message middleware from the intermediate server clusters as a target server cluster. Therefore, the target message middleware can be selected based on the message types supported by the message middleware, and the reference server cluster corresponding to the target message middleware is determined to be the target server cluster, so that the target server cluster can process the target message, and the situation that the server cluster does not support the target message is avoided.

Specifically, when the number of the intermediate server clusters is greater than 1, the message types that can be supported may be determined based on the message middleware operated by each intermediate server cluster, and further, the reference server cluster is selected from each intermediate server cluster to support the message types.

As an optional embodiment, if the number of reference server clusters is greater than 1, determining, from each intermediate server cluster, a reference server cluster corresponding to the target message middleware as the target server cluster includes: determining each reference server cluster corresponding to the target message middleware from each intermediate server cluster; determining the flow load states corresponding to the reference server clusters respectively; a target server cluster is determined from the reference server clusters based on the traffic load status. Therefore, under the condition that a plurality of reference server clusters exist, one reference server cluster is selected as a target server cluster according to the traffic load state, so that the message transmission efficiency is improved.

Specifically, the traffic load state can be represented by any form of numerical value, character and the like, and the traffic load state can be used for representing the traffic load condition of the corresponding reference server cluster, wherein the higher the traffic load is, the greater the traffic pressure of the corresponding reference server cluster is; the lower the traffic load, the less traffic pressure that represents the corresponding reference server cluster. Based on this, determining a target server cluster from among the reference server clusters based on the traffic load status, comprising: and determining an optimal traffic load state based on the value of the traffic load state, wherein the value of the optimal traffic load state is the smallest, and further determining the reference server cluster of the optimal traffic load state as a target server cluster.

In step S130, the target message is sent to the target server cluster, so that the target server cluster runs a corresponding target message middleware to deliver the target message.

Specifically, after sending the target message to the target server cluster, the target message middleware running in the target server cluster may store the target message in a queue, and send the target message to the proxy server when the message sending opportunity arrives, so as to trigger the proxy server to forward the target message to the second client.

The second client refers to a consumer (consumer) for pulling the message, and is not specific to a certain client. The second client may pull the target message from the target server cluster through the proxy server.

As an alternative embodiment, further comprising:

step S141: receiving a message pulling request for pulling a message set; wherein the message set includes a target message and other messages;

step S142: responding to the message pulling request, pulling a target message from the target message middleware, and pulling other messages from the message middleware of other server clusters to obtain a message set; the set of messages is sent to the second client.

By implementing steps S141-S142, message passing based on diversified message middleware can be realized, and message middleware shielding to the second client can be realized based on the function of proxy forwarding, so that message passing without perception can be realized. And the message merging and pulling function based on diversified message middleware which is not laid out in the related technology is realized, and the application range of the message transmission scheme is expanded.

Specifically, the message pull request may request to pull a message set including the target message and other messages, or may request to pull only the target message. If the message pull request only requests pulling the target message, then the target message may be pulled from the target message middleware and sent to the second client in response to the message pull request. If the message pulling request is used for pulling the target message and other messages, the target message and other messages can be pulled from each server cluster; wherein, other messages may be stored in different server clusters or may be stored in the same server cluster.

As an alternative embodiment, sending the target message to the target server cluster, so that the target server cluster runs a corresponding target message middleware to deliver the target message, includes: if the message type is the delay message type, sending the target message to the target server cluster, so that the target message middleware operated by the target server cluster transmits the target message after a preset time length; and if the message type is the real-time message type, sending the target message to the target server cluster, so that the target message middleware operated by the target server cluster transmits the target message in real time. Thus, different message passing flows can be realized according to different message types, namely, a message passing scheme capable of supporting multiple message types is realized.

Specifically, the proxy server may determine to which message type the target message belongs after receiving the message, and the message type may include, but is not limited to: the delay message type and the real-time message type are not limited by the embodiment of the application. When the message type is a delayed message type, the target message middleware may be triggered to deliver the target message to the proxy server after a preset duration (e.g., 10 minutes), and then forwarded by the proxy server to the second client. When the message type is the real-time message type, the target message middleware can be triggered to respond to the pulling request at any time, and the target message is transmitted to the proxy server and then is forwarded to the second client side by the proxy server.

As an alternative embodiment, further comprising: performing capability expansion on a specific server cluster in response to capability expansion operation on the specific server cluster in at least two server clusters; the number of message types supported by the specific server cluster after the capability expansion is larger than the number of message types supported by the specific server cluster before the capability expansion. This provides a capability extension function to extend the capabilities of a particular server cluster of the at least two server clusters so that it can support more diverse message types.

In particular, in at least two server clusters, since message middleware corresponding to different server clusters may be different, message types supported by different message middleware may also be different. Based on this, a capability extension function for extending message types may also be provided, based on which a developer may extend the message types that a particular server cluster is capable of supporting, in that the message middleware capability of the particular server cluster is modified so that it may support more diverse message types; the specific server cluster refers to a server cluster which is selected by a developer and needs to be subjected to capacity expansion, and the developer can select any one of the server clusters as the specific server cluster.

As an alternative embodiment, sending the target message to the target server cluster includes: determining a cluster state corresponding to a target server cluster; and if the cluster state characterization cluster is available, sending the target message to the target server cluster. Therefore, the target message can be sent to the target server cluster again under the condition that the cluster is available, and the target message can be timely processed.

Specifically, the cluster state represents whether the target server cluster is available, the cluster state can be represented by any form of numerical value, text, character and the like, and when the number of unavailable servers in the target server cluster is detected to be smaller than or equal to the preset number, the cluster state represents that the cluster is available.

As an alternative embodiment, further comprising: if the cluster state characterization cluster is not available, the target message is sent to a standby cluster of the at least two server clusters. Therefore, when the cluster is unavailable, the target message is sent to the standby cluster, so that smooth migration of the message middleware and no perceived flow switching of the client are realized, and the situation that the target message cannot be processed in time is avoided.

Specifically, at least two server clusters may include one or more standby clusters for the target server cluster, and when the cluster state representation cluster is unavailable, the target message may be specifically sent to any standby cluster, and the message middleware corresponding to the standby cluster may be the same as or different from the message middleware of the target server cluster.

As an alternative embodiment, further comprising: when the availability of the cluster state representation cluster is detected again, the received new message is sent to the target server cluster; wherein the new message corresponds to the same message type as the target message. Therefore, when the target server cluster is recovered to be available, the target server cluster is reused, multiplexing of the target server cluster is realized, and the cluster utilization rate is improved.

Specifically, when the cluster state characterization cluster is not available, the target message is temporarily sent to a standby cluster in at least two server clusters, and then when the cluster state characterization cluster of the target server cluster is detected to be available again (i.e. the servers in the target server cluster are recovered, and the number of unavailable servers is less than or equal to the preset number), the received new message can be continuously sent to the target server cluster, and the embodiment realizes the flow switching of the client-side non-sense; the new message may be from the first client or from another client, which is not limited in the embodiment of the present application.

As an alternative embodiment, further comprising: and when the number of the unavailable servers in the target server cluster is detected to be larger than the preset number, judging that the cluster state representation cluster is unavailable. Therefore, when the number of the unavailable servers is larger than the preset number, the cluster is timely judged to be unavailable, and the problems of message transmission interruption, untimely message transmission and the like caused by continuously sending the target message to the target server cluster are avoided.

Specifically, the number of unavailable servers in the target server cluster is greater than the preset number (e.g., 100), which indicates that most servers in the target server cluster have been down, and the servers are determined to be unavailable, so that timely traffic switching is facilitated, and the message transfer efficiency is ensured not to be reduced. Alternatively, it may be determined that the cluster state token cluster is unavailable when it is detected that the ratio of the number of unavailable servers in the target server cluster to the total number of servers in the target server cluster is greater than a preset ratio (e.g., 0.1%). Additionally, optionally, when a condition that the target server cluster has a message sending failure/a message consuming failure is detected, it may also be determined that the cluster state characterization cluster is not available.

Referring to fig. 5, fig. 5 schematically illustrates a primary and backup cluster schematic according to an embodiment of the application. As shown in fig. 5, an architecture implementing an embodiment of the present application may include: producer510, consumer520, proxy530, distributed storage 540, kafka-cluster550, locketmq-cluster 560. Among them, for example, kafka-cluster550 may be determined as a primary cluster and locketmq-cluster 560 may be determined as a backup cluster.

Specifically, proxy530 may receive the target message sent (send) by producer510, determine metadata from distributed storage 540 according to the message topic (topic) corresponding to the target message, further determine a target server cluster from at least two server clusters according to the message type and metadata corresponding to the target message, and determine the master cluster kafka-cluster550 as the target server cluster when kafka-cluster550 and docktq-cluster 560 simultaneously satisfy the condition as the target server cluster. Further, proxy530 may send the target message to kafka-cluster550, which kafka-cluster550 is configured to run the corresponding target message middleware and pass the target message into consumer520 via proxy530 when consumer520 pulls (pull) the target message.

It should be noted that fig. 5 is only one implementation manner of the embodiment of the present application, and in the actual application process, the primary cluster and the standby cluster may not be specified, that is, each server cluster is in the same position.

Referring to fig. 6 on the basis of fig. 5, fig. 6 schematically illustrates a flow switching scheme according to an embodiment of the present application. As shown in fig. 6, an architecture implementing an embodiment of the present application may include: producer610, consumer620, proxy630, distributed storage 640, kafka-cluster650, locketmq-cluster 660.

Specifically, proxy630 may receive the target message sent (send) by producer610, determine metadata from distributed storage 640 according to the message topic (topic) corresponding to the target message, and further determine a target server cluster from at least two server clusters according to the message type and metadata corresponding to the target message, and determine the master cluster kafka-cluster650 as the target server cluster when kafka-cluster650 and docktq-cluster 660 simultaneously satisfy the condition as the target server cluster and master cluster kafka-cluster650 is available.

However, when it is detected that the number of unavailable servers in the kafka-cluster650 is greater than the preset number/the ratio of the number of unavailable servers to the total number of servers of the kafka-cluster650 is greater than the preset ratio/there is a case where the message transmission fails or the message consumption fails, it is determined that the cluster state of the kafka-cluster650 characterizes the cluster as unavailable and the target message is transmitted to the backup cluster locktmq-cluster 660. In turn, proxy630 may send the target message to a locketmq-cluster 660, which locketmq-cluster 660 is configured to run the corresponding target message middleware and pass the target message to consumer620 via proxy630 as consumer620 pulls (pull) the target message. This ensures that the targeted message is delivered in a timely manner.

As an alternative embodiment, further comprising:

step S151: acquiring operation parameters corresponding to at least two server clusters respectively; wherein the operating parameters include at least one of: delay information (request-latency-avg), error information, synchronization efficiency information (byte inserse).

Step S152: and adjusting the flow load weights corresponding to at least two server clusters respectively based on the operation parameters to obtain new flow load weights corresponding to each server cluster.

By implementing the steps S151 to S152, the operation parameters can be automatically obtained, the health condition of each server cluster can be monitored in time, and the traffic load weight can be adjusted in time based on the operation parameters, so as to ensure the normal message transmission efficiency.

Specifically, obtaining operation parameters (metrics) corresponding to at least two server clusters respectively includes: and acquiring operation parameters corresponding to at least two server clusters respectively according to unit time (e.g. 3 s), and timely monitoring the health condition of each server cluster. Where metrics refers to various metrics and data collected during system, application or service operation, may be used to monitor performance, health status and behavior. The request-latency-avg is an average delay time index of the request and is used to measure the speed of the system processing the request. The byte per se represents the number of bytes received per second and is typically used to scale the rate of data transmission in terms of network or storage.

Furthermore, adjusting the traffic load weights corresponding to at least two server clusters based on the operation parameters to obtain new traffic load weights corresponding to each server cluster, including: determining a delay value limited by delay information and a synchronization efficiency value limited by synchronization efficiency information for each server cluster; calculating a first ratio of a delay value to preset delay data, a synchronization efficiency value and a second ratio of preset synchronization efficiency data, and determining error reporting frequency in error reporting information; and adjusting the traffic load weights corresponding to the server clusters respectively according to at least one of the first ratio, the second ratio and the error reporting frequency to obtain new traffic load weights corresponding to the server clusters. The purpose of the adjustment is to make the traffic load weight distributed by the server cluster with high delay value, low synchronization efficiency value or high error reporting frequency lower so as to ensure the traffic processing efficiency. Where traffic refers to a continuously received message, including a target message.

Further, according to at least one of the first ratio, the second ratio and the error reporting frequency, adjusting the traffic load weights corresponding to each server cluster respectively to obtain new traffic load weights corresponding to each server cluster, including: and merging the first ratio, the second ratio and the error reporting frequency to obtain comprehensive scores, and adjusting the flow load weights corresponding to the server clusters respectively based on the comprehensive scores to obtain new flow load weights corresponding to the server clusters.

As an alternative embodiment, determining the target server cluster from the at least two server clusters based on the message type includes: and determining a target server cluster from at least two server clusters based on the new traffic load weight and the message type corresponding to each server cluster. Therefore, a target server cluster with higher adaptation degree for the target message can be selected according to the adjusted new traffic load weight and the message type, so that the target message can be efficiently transferred.

Specifically, based on the new traffic load weight corresponding to each server cluster, a server cluster to be selected meeting the preset traffic load weight (e.g., 70%) is determined from at least two server clusters, and further, based on the message type, a target server cluster is determined from the server clusters to be selected.

Referring to fig. 7, fig. 7 schematically illustrates a flow balancing load for clusters according to one embodiment of the application. As shown in fig. 7, an architecture implementing an embodiment of the present application may include: producer710, consumer720, proxy730, distributed storage 740, kafka-cluster750, and locketmq-cluster 760. Among other things, load balancing may be provided between kafka-cluster750, locketmq-cluster 760 so that kafka-cluster750, locketmq-cluster 760 may evenly carry traffic.

Specifically, the proxy730 may receive the target message sent by the producer710 (send), determine metadata from the distributed storage 740 according to a message topic (topic) corresponding to the target message, further determine a target server cluster from at least two server clusters according to a message type and metadata corresponding to the target message, and when the kafka-cluster750 and the dockmq-cluster 760 simultaneously meet the condition as the target server cluster, select any server cluster from the target server clusters (e.g., kafka-cluster 750). Further, proxy730 can send the target message to kafka-cluster750, which kafka-cluster750 is used to run the corresponding target message middleware and pass the target message into consumer720 via proxy730 when consumer720 pulls (pull) the target message.

Referring to fig. 8, fig. 8 schematically illustrates a flow load weight adjusted schematic diagram according to one embodiment of the application. As shown in fig. 8, an architecture implementing an embodiment of the present application may include: producer810, consumer820, proxy830, distributed storage 840, kafka-cluster850, and locketmq-cluster 860. In an exemplary embodiment, after the operation parameters corresponding to the at least two server clusters are obtained, the traffic load weights corresponding to the kafka-cluster850 and the locktq-cluster 860 may be adjusted based on the operation parameters, where the adjusted kafka-cluster850 loads 30% of the traffic, and the adjusted locktq-cluster 860 loads 70% of the traffic.

Specifically, proxy830 may receive the target message sent (send) by producer810, determine metadata from distributed storage 840 according to the message topic (topic) corresponding to the target message, and further determine the target server cluster (e.g., the 70% traffic-loaded locktmq-cluster 860) from kafka-cluster850 and locktmq-cluster 860 based on the traffic load weight according to the message type and metadata corresponding to the target message. Further, proxy830 may send the target message to a locketmq-cluster 860, where the locketmq-cluster 860 is configured to run the corresponding target message middleware and pass the target message into the consumer820 via proxy830 when the consumer820 pulls (pull) the target message.

Referring to fig. 9, fig. 9 schematically illustrates a flowchart of a message transmission method based on cloud native multicluster according to another embodiment of the application. As shown in fig. 9, the message transmission method based on the cloud native multi-cluster includes: step S900 to step S920.

Step S900: creating at least one message topic in response to a user operation for a control panel with integrated creation capability, and storing metadata corresponding to the at least one message topic in a memory, and creating at least two server clusters; the metadata are used for representing server clusters corresponding to at least one message theme respectively.

Step S902: and receiving the target message sent by the first client, determining the message type corresponding to the target message, reading target metadata corresponding to the target message theme from a memory, and further determining an intermediate server cluster corresponding to the target metadata from at least two server clusters.

Step S904: and determining a target message middleware supporting the message type from the message middleware corresponding to each intermediate server cluster, and determining a reference server cluster corresponding to the target message middleware from each intermediate server cluster as the target server cluster.

Step S906: determining a cluster state corresponding to a target server cluster; and when the number of the unavailable servers in the target server cluster is detected to be larger than the preset number, judging that the cluster state representation cluster is unavailable. If the cluster state representation cluster is available, then step S908 is performed; if the cluster state representation cluster is not available, step S910 is performed.

Step S908: the target message is sent to the target server cluster so that the target server cluster runs a corresponding target message middleware to deliver the target message. Further, step S912 is performed.

Step S910: the target message is sent to a standby cluster of the at least two server clusters. Further, when the availability of the cluster state characterization cluster is detected again, the received new message is sent to the target server cluster; wherein the new message corresponds to the same message type as the target message. Further, a request to pull the target message may be received and in response to the request, the target message may be pulled from the standby cluster and sent to the second client.

Step S912: receiving a message pulling request for pulling a message set; wherein the message set includes the target message and other messages.

Step S914: and responding to the message pulling request, pulling the target message from the target message middleware, pulling other messages from the message middleware of other server clusters to obtain a message set, and sending the message set to the second client.

Step S916: acquiring operation parameters corresponding to at least two server clusters respectively; wherein the operating parameters include at least one of: delay information, error reporting information, synchronization efficiency information.

Step S918: and adjusting the flow load weights corresponding to at least two server clusters respectively based on the operation parameters to obtain new flow load weights corresponding to each server cluster, so that when other messages are received, the server clusters for processing other messages are determined from the at least two server clusters based on the new flow load weights corresponding to each server cluster and the message types.

Step S920: performing capability expansion on a specific server cluster in response to capability expansion operation on the specific server cluster in at least two server clusters; the number of message types supported by the specific server cluster after the capability expansion is larger than the number of message types supported by the specific server cluster before the capability expansion.

The execution sequence of steps S900 to S920 defined in fig. 9 is merely illustrative, and the execution sequence of steps S916 to S918 and S920 is not limited in the present application. Step S916 to step S918 may be before or after any of step S900 to step S914, and similarly, step S920 may be before or after any of step S900 to step S914, which is not limited by the embodiment of the present application.

It should be noted that, the steps S900 to S920 correspond to the steps and the embodiments thereof shown in fig. 1, and for the specific implementation of the steps S900 to S920, please refer to the steps and the embodiments thereof shown in fig. 1, and the description thereof is omitted herein.

Therefore, by implementing the method shown in fig. 9, a target server cluster capable of supporting the message type can be selected from a plurality of server clusters based on the message type corresponding to the target message, and the target message is processed through the target message middleware of the target server cluster, so that higher compatibility is provided for the clusters configured in different message middleware, and the diversified server clusters can further improve the applicability to diversified target messages, thereby further improving the application range of the message transmission scheme. In addition, because the target server cluster capable of supporting the message type can be selected from the plurality of server clusters based on the message type corresponding to the target message, a specific message middleware is not required to be known for the first client, the corresponding target server cluster can be determined to process the target message no matter any message type, and the message middleware shielding for the client is realized.

Referring to fig. 10, fig. 10 schematically illustrates a block diagram of a message transmission apparatus based on a cloud native multi-cluster in an embodiment of the present application. As shown in fig. 10, the cloud native multi-cluster based message transmission apparatus 1000 may include the following units.

A message type determining unit 1001, configured to receive a target message sent by a first client, and determine a message type corresponding to the target message;

a cluster selection unit 1002, configured to determine a target server cluster from at least two server clusters based on the message type; wherein at least two server clusters exist in the at least two server clusters corresponding to different message middleware;

the message sending unit 1003 is configured to send the target message to the target server cluster, so that the target server cluster runs a corresponding target message middleware to deliver the target message.

Therefore, by implementing the device shown in fig. 10, a target server cluster capable of supporting the message type can be selected from multiple server clusters based on the message type corresponding to the target message, and the target message is processed through the target message middleware of the target server cluster, so that higher compatibility is provided for the clusters configured in different message middleware, and the diversified server clusters can further improve the applicability to diversified target messages, thereby further improving the application range of the message transmission scheme. In addition, because the target server cluster capable of supporting the message type can be selected from the plurality of server clusters based on the message type corresponding to the target message, a specific message middleware is not required to be known for the first client, the corresponding target server cluster can be determined to process the target message no matter any message type, and the message middleware shielding for the client is realized.

In one embodiment of the present application, the method further comprises:

It can be seen that by implementing the alternative embodiment, a control panel with strong comprehensiveness can be provided for the user, so that the user can complete the creation of the message theme and the server cluster in one step, and the creation transaction does not need to be completed through various proxy server interfaces, so that the creation efficiency of the server cluster and the message theme can be improved.

In one embodiment of the present application, the cluster selection unit 1002 determines a target server cluster from at least two server clusters based on a message type, including:

reading target metadata corresponding to a target message theme from a memory;

Therefore, according to the implementation of the alternative embodiment, a part of server clusters can be selected as the intermediate server clusters based on the target metadata, then the target server clusters are selected from the intermediate server clusters based on the message types, the matching degree of the selected target server clusters and the target message is higher, the accurate target message distribution can be realized, the target message is prevented from being distributed to the unmatched server clusters, and the situation that the server clusters do not support the target message is avoided.

In one embodiment of the present application, the cluster selection unit 1002 determines a target server cluster from among the intermediate server clusters based on the message type, including:

It can be seen that, by implementing the alternative embodiment, the target message middleware can be selected based on the message type supported by the message middleware, and the reference server cluster corresponding to the target message middleware is determined as the target server cluster, so that the target server cluster can process the target message, and the situation that the server cluster does not support the target message is avoided.

In a specific embodiment of the present application, if the number of reference server clusters is greater than 1, from among the intermediate server clusters, the cluster selection unit 1002 determines, as the target server cluster, the reference server cluster corresponding to the target message middleware, including:

It can be seen that, by implementing this alternative embodiment, in the case that there are multiple reference server clusters, a reference server cluster may be selected as a target server cluster according to the traffic load status, so as to improve the message transfer efficiency.

In one embodiment of the present application, the method further comprises:

It will be seen that implementing this alternative embodiment may provide a capability extension function to extend the capabilities of a particular server cluster of the at least two server clusters so that it may support more diverse message types.

In one embodiment of the present application, the message sending unit 1003 sends the target message to the target server cluster, including:

determining a cluster state corresponding to a target server cluster;

It can be seen that by implementing this alternative embodiment, the target message may be sent to the target server cluster again in the case that the cluster is available, so as to ensure that the target message may be processed in time.

In one embodiment of the present application, wherein:

the message sending unit 1003 is further configured to send the target message to a standby cluster of the at least two server clusters if the cluster state representation cluster is not available.

Therefore, by implementing the alternative embodiment, the target message can be sent to the standby cluster when the cluster is unavailable, so that smooth migration of the message middleware and no perceived flow switching of the client side are realized, and the situation that the target message cannot be processed in time is avoided.

In one embodiment of the present application, wherein:

the message sending unit 1003 is further configured to send the received new message to the target server cluster when it is detected that the cluster state representation cluster is available again; wherein the new message corresponds to the same message type as the target message.

It can be seen that by implementing the alternative embodiment, when the target server cluster is recovered to be available, the target server cluster is reused, so that multiplexing of the target server cluster is realized, and the cluster utilization rate is improved.

In one embodiment of the present application, the method further comprises:

Therefore, by implementing the alternative embodiment, when the number of the unavailable servers is greater than the preset number, the cluster can be timely judged to be unavailable, so that the problems of message transmission interruption, untimely message transmission and the like caused by continuously sending the target message to the target server cluster are avoided.

In one embodiment of the present application, the method further comprises:

It can be seen that implementing this alternative embodiment, message passing based on diversified message middleware can be implemented, and the proxy forwarding based function implements message middleware shielding for the second client, and thus, non-aware message passing can be implemented. And the message merging and pulling function based on diversified message middleware which is not laid out in the related technology is realized, and the application range of the message transmission scheme is expanded.

In a specific embodiment of the present application, the message sending unit 1003 sends the target message to the target server cluster, so that the target server cluster runs a corresponding target message middleware to transfer the target message, including:

It will be seen that implementing this alternative embodiment, different messaging flows may be implemented according to different message types, i.e. a messaging scheme is implemented that can support multiple message types.

In one embodiment of the present application, the method further comprises:

Therefore, by implementing the alternative embodiment, the operation parameters can be automatically acquired, the health condition of each server cluster can be timely monitored, and the traffic load weight can be timely adjusted based on the operation parameters so as to ensure the normal message transmission efficiency.

It can be seen that, by implementing this alternative embodiment, a target server cluster with a higher adaptation degree for the target message may be selected according to the adjusted new traffic load weight and the message type, so as to ensure that the target message may be efficiently delivered.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

Since each functional module of the message transmission device based on the cloud native multi-cluster according to the exemplary embodiment of the present application corresponds to the steps of the foregoing exemplary embodiment of the message transmission device based on the cloud native multi-cluster, for details not disclosed in the embodiment of the device according to the present application, please refer to the foregoing embodiment of the message transmission device based on the cloud native multi-cluster according to the present application.

Referring to fig. 11, fig. 11 is a schematic diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.

It should be noted that, the computer system 1100 of the electronic device shown in fig. 11 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

As shown in fig. 11, the computer system 1100 includes a Central Processing Unit (CPU) 1101 that can execute various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data required for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

The following components are connected to the I/O interface 1105: an input section 1106 including a keyboard, a mouse, and the like; an output portion 1107 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 1108 including a hard disk or the like; and a communication section 1109 including a network interface card such as a LAN card, a modem, and the like. The communication section 1109 performs communication processing via a network such as the internet. The drive 1110 is also connected to the I/O interface 1105 as needed. Removable media 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed on drive 1110, so that a computer program read therefrom is installed as needed into storage section 1108.

In particular, according to embodiments of the present application, the processes described below with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1109, and/or installed from the removable media 1111. The computer program, when executed by a Central Processing Unit (CPU) 1101, performs the various functions defined in the method and apparatus of the present application.

As another aspect, the present application also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by one of the electronic devices, cause the electronic device to implement the methods of the above-described embodiments.

The computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

Claims

1. The message transmission method based on the cloud native multi-cluster is characterized by comprising the following steps of:

and sending the target message to the target server cluster, so that the target server cluster runs corresponding target message middleware to transfer the target message.

2. The method as recited in claim 1, further comprising:

creating the at least two server clusters;

the metadata is used for representing server clusters respectively corresponding to the at least one message theme, and the at least one message theme comprises a target message theme corresponding to the target message.

3. The method of claim 2, wherein determining a target server cluster from at least two server clusters based on the message type comprises:

reading target metadata corresponding to the target message theme from the memory;

a target server cluster is determined from the intermediate server clusters based on the message type.

4. A method according to claim 3, wherein determining a target server cluster from among the intermediate server clusters based on the message type comprises:

5. The method of claim 4, wherein if the number of reference server clusters is greater than 1, determining, from the intermediate server clusters, the reference server cluster corresponding to the target message middleware as the target server cluster comprises:

determining the traffic load states corresponding to the reference server clusters respectively;

and determining a target server cluster from the reference server clusters based on the traffic load state.

6. The method as recited in claim 1, further comprising:

performing capability expansion on a specific server cluster in the at least two server clusters in response to the capability expansion operation on the specific server cluster;

the number of message types supported by the specific server cluster after capability expansion is larger than the number of message types supported by the specific server cluster before capability expansion.

7. The method of claim 1, wherein sending the target message to the target server cluster comprises:

determining a cluster state corresponding to the target server cluster;

8. The method as recited in claim 7, further comprising:

and if the cluster state characterization cluster is not available, sending the target message to a standby cluster in the at least two server clusters.

9. The method as recited in claim 8, further comprising:

10. The method as recited in claim 8, further comprising:

and when the number of the unavailable servers in the target server cluster is detected to be larger than a preset number, judging that the cluster state representation cluster is unavailable.

11. The method as recited in claim 1, further comprising:

Receiving a message pulling request for pulling a message set; wherein the message set includes the target message and other messages;

responding to the message pulling request, pulling the target message from the target message middleware, and pulling the other messages from the message middleware of other server clusters to obtain the message set;

and sending the message set to a second client.

12. The method of claim 1, wherein sending the target message to the target server cluster to cause the target server cluster to run a corresponding target message middleware to deliver the target message comprises:

if the message type is a delay message type, the target message is sent to the target server cluster, so that a target message middleware operated by the target server cluster transmits the target message after a preset duration;

and if the message type is a real-time message type, sending the target message to the target server cluster, so that a target message middleware operated by the target server cluster transmits the target message in real time.

13. The method as recited in claim 1, further comprising:

Acquiring operation parameters corresponding to the at least two server clusters respectively; wherein the operating parameters include at least one of: delay information, error reporting information, and synchronization efficiency information;

and adjusting the flow load weights corresponding to the at least two server clusters respectively based on the operation parameters to obtain new flow load weights corresponding to the server clusters.

14. The method of claim 13, wherein determining a target server cluster from at least two server clusters based on the message type comprises:

and determining a target server cluster from at least two server clusters based on the new traffic load weight corresponding to each server cluster and the message type.

15. A message transmission device based on cloud native multi-cluster, comprising:

the message type determining unit is used for receiving a target message sent by the first client and determining a message type corresponding to the target message;

a cluster selection unit, configured to determine a target server cluster from at least two server clusters based on the message type; wherein at least two server clusters exist in the at least two server clusters corresponding to different message middleware;

16. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1-14.

17. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any one of claims 1-14 via execution of the executable instructions.