CN114726863A

CN114726863A - Method, device, system and storage medium for load balancing

Info

Publication number: CN114726863A
Application number: CN202210448147.1A
Authority: CN
Inventors: 王夕宁
Original assignee: Alibaba Cloud Computing Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2022-04-27
Filing date: 2022-04-27
Publication date: 2022-07-08
Anticipated expiration: 2042-04-27
Also published as: CN114726863B

Abstract

The application provides a method, equipment, a system and a storage medium for load balancing, wherein the method is applied to a first application service in a cluster, and the first application service comprises the following steps: a first application service instance and an xDS proxy, the first application service instance comprising: a first workload and xDS client; the method comprises the steps that first load balancing strategy information sent by a xDS server is received through an xDS proxy, the first load balancing strategy information is forwarded to a xDS client side, and when a first work load needs to call a second application service in a cluster, a call request is sent to one or more target second application service instances in the second application service through the xDS client side according to the first load balancing strategy information, so that load balancing of the second application service instances in the second application service is achieved. Through the process, load balancing can be realized in the first application service instance, and the call request does not need to be forwarded by the grid agent, so that the service response time is reduced.

Description

Method, device, system and storage medium for load balancing

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a system, and a storage medium for load balancing.

Background

In a microservice architecture, an application may include multiple application services, each of which is only concerned with completing one task, and thus, an application may be viewed as a collection of multiple application services. The respective application services for an application may be deployed independently, and each application service may be deployed on one or more servers.

Since different application services may be deployed on different servers, calls between different application services involve communication between different servers. In order to facilitate secure and reliable communication between different application services, a micro-service administration framework-service mesh (service mesh) is proposed. A services grid is a configurable infrastructure layer that serves service applications, aiming at handling a large amount of network-based inter-process communication between different application services. Specifically, in the service grid framework, a grid agent (also referred to as a sidecar) is deployed for each application service, and all traffic entering and exiting the application service passes through its corresponding grid agent first. Thus, calls between application services are translated into calls between grid proxies.

In a load balancing scenario, when a first application service calls a second application service, the first application service sends a call request to its corresponding grid proxy, and the grid proxy determines, based on a load balancing policy, to which workload (or referred to as a service instance) of the second application service the call request is forwarded, so as to ensure load balancing of workloads in the second application service. However, since all the invocation requests need to be forwarded through the mesh proxy, the service response time length is increased.

Disclosure of Invention

Embodiments of the present application provide a method, an apparatus, a system, and a storage medium for load balancing, so as to reduce a service response duration of service invocation between different application services.

In a first aspect, an embodiment of the present application provides a method for load balancing, which is applied to a first application service in a cluster, where the first application service includes: a first application service instance and service discovery xDS proxy, the first application service instance comprising: a first workload and xDS client; the method comprises the following steps:

receiving xDS first load balancing strategy information sent by a server through the xDS proxy, and forwarding the first load balancing strategy information to the xDS client;

when the first workload needs to invoke the second application service in the cluster, sending, by the xDS client, an invocation request to one or more target second application service instances in the second application service according to the first load balancing policy information, so as to implement load balancing of each second application service instance in the second application service.

In a possible implementation manner, before the xDS proxy receives xDS first load balancing policy information sent by the server, the method further includes:

when a starting instruction corresponding to the first application service instance is received, starting the xDS proxy;

obtaining, by the xDS agent, first launch configuration information from the xDS server, the first launch configuration information being configuration information required to launch the first application service instance;

and starting the first application service instance according to the first starting configuration information.

In one possible implementation, the obtaining, by the xDS proxy, the first launch configuration information from the xDS server includes:

establishing a connection with the xDS server through the xDS proxy;

receiving, by the xDS proxy, second load balancing policy information from the xDS server;

generating, by the xDS agent, the first startup configuration information according to the second load balancing policy information, the first startup configuration information including the second load balancing policy information.

In one possible implementation, establishing a connection with the xDS server through the xDS proxy includes:

acquiring certificate information through the xDS proxy, and sending a connection establishment request to the xDS server according to the certificate information, wherein the connection establishment request comprises the certificate information; wherein the certificate information is configured by an agent manager for the xDS agent; the agent manager is used for managing xDS agents in each application service in the cluster;

receiving, by the xDS proxy, a connection establishment response from the xDS server, the connection establishment response sent by the xDS server upon verification of the certificate information.

In one possible implementation, initiating the xDS proxy includes:

obtaining second launch configuration information required to launch the xDS agent, the second launch configuration information configured by the agent manager for the xDS agent;

and starting the xDS proxy according to the second starting configuration information.

In one possible implementation, when the first workload needs to invoke a second application service in the cluster, sending, by the xDS client, an invocation request to one or more target second application service instances in the second application service according to the first load balancing policy information, includes:

performing, by the xDS client, the following steps:

when the first workload is monitored to send the call request, intercepting the call request, analyzing the call request, and determining the second application service to be called;

acquiring a load balancing strategy corresponding to the second application service from the first load balancing strategy information, and determining one or more target second application service instances in a plurality of second application service instances included in the second application service according to the load balancing strategy corresponding to the second application service;

sending the invocation request to the one or more target second application service instances.

In a possible implementation manner, after analyzing the call request and determining the second application service to be called, the method further includes:

determining whether a connection channel between the first application service and the second application service exists;

and if the address information does not exist, establishing a connection channel between the first application service and the second application service according to the address information and the port information of the second application service.

In one possible implementation, the xDS agents in different application services in the cluster are implemented based on the same programming language framework.

In a second aspect, an embodiment of the present application provides a method for load balancing, which is applied to a service discovery xDS server, and the method includes:

acquiring first load balancing strategy information corresponding to a first application service instance in a cluster;

sending the first load balancing policy information to an xDS proxy corresponding to the first application service instance, so that the xDS proxy forwards the first load balancing policy information to a xDS client in the first application service instance, where the first load balancing policy information is used to implement load balancing of each second application service instance in the second application service when a first workload in the first application service instance calls the second application service in the cluster.

In a third aspect, an embodiment of the present application provides a method for load balancing, which is applied to a system for load balancing, where the system includes: a service discovery xDS server deployed at a control plane layer and a cluster deployed at a data plane layer, the cluster including a first application service and a second application service, the first application service including a first application service instance and xDS agents, the first application service instance including a first workload and xDS clients, the second application service including a plurality of second application service instances; the method comprises the following steps:

the service discovery xDS server acquires first load balancing strategy information corresponding to the first application service instance and sends the first load balancing strategy information to a xDS agent corresponding to the first application service instance;

the first application service forwarding the first load balancing policy information to the xDS client through the xDS proxy;

when the first workload needs to call the second application service in the cluster, the first application service sends a call request to one or more target second application service instances in the second application service through the xDS client according to the first load balancing policy information, so as to realize load balancing of each second application service instance in the second application service.

In a fourth aspect, an embodiment of the present application provides an apparatus for load balancing, which is applied to a first application service in a cluster, where the first application service includes: a first application service instance and a service discovery xDS proxy, the first application service instance comprising: a first workload and xDS client; the device comprises:

a policy obtaining module, configured to receive xDS first load balancing policy information sent by a server through the xDS proxy, and forward the first load balancing policy information to the xDS client;

and a policy execution module, configured to send, by the xDS client, a call request to one or more target second application service instances in the second application service according to the first load balancing policy information when the first workload needs to call the second application service in the cluster, so as to implement load balancing of each second application service instance in the second application service.

In a fifth aspect, an embodiment of the present application provides an apparatus for load balancing, which is applied to a service discovery xDS server, and the apparatus includes:

the acquisition module is used for acquiring first load balancing strategy information corresponding to a first application service instance in a cluster;

a sending module, configured to send the first load balancing policy information to an xDS proxy corresponding to the first application service instance, so that the xDS proxy forwards the first load balancing policy information to a xDS client in the first application service instance, where the first load balancing policy information is used to implement load balancing of each second application service instance in the second application service when a first workload in the first application service instance calls a second application service in the cluster.

In a sixth aspect, an embodiment of the present application provides a system for load balancing, including: a service discovery xDS server deployed at a control plane layer and a cluster deployed at a data plane layer, the cluster including a first application service and a second application service, the first application service including a first application service instance and xDS agents, the first application service instance including a first workload and xDS clients, the second application service including a plurality of second application service instances;

the xDS server is configured to send first load balancing policy information to the xDS agent;

the xDS proxy is to forward the first load balancing policy information to the xDS client;

the xDS client is configured to, when the first workload needs to invoke a second application service in the cluster, send an invocation request to one or more target second application service instances in the second application service according to the first load balancing policy information, so as to implement load balancing of each second application service instance in the second application service.

In a possible implementation, the method further includes: an agent manager deployed at the control plane layer;

the agent manager is configured to distribute the second startup configuration information to xDS agents in the application services in the cluster, and/or distribute the certificate information to xDS agents in the application services in the cluster.

In a seventh aspect, an embodiment of the present application provides an electronic device, including: a memory having stored therein a computer program configured to be executed by the at least one processor to implement the method of any one of the first aspects or the method of any one of the second aspects, and at least one processor.

In an eighth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any one of the first aspect or the second aspect.

In a ninth aspect, the present application provides a computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of the first aspect, or the method according to any one of the second aspect.

The method, the device, the system and the storage medium for load balancing provided by the embodiment of the application are applied to a first application service in a cluster, wherein the first application service comprises the following steps: a first application service instance and an xDS proxy, the first application service instance comprising: a first workload and xDS client; the method comprises the following steps: the method comprises the steps that first load balancing strategy information sent by a xDS server is received through an xDS proxy, the first load balancing strategy information is forwarded to a xDS client, and when a first work load needs to call a second application service in a cluster, a call request is sent to one or more target second application service instances in the second application service through a xDS client according to the first load balancing strategy information, so that load balancing of each second application service instance in the second application service is achieved. Through the process, load balancing can be achieved in the first application service instance, a grid proxy does not need to be deployed for the first application service instance, and when the first application service instance calls the second application service, the call request does not need to be forwarded through the grid proxy, so that service response time can be reduced.

Drawings

FIG. 1 is a schematic diagram of a services grid;

FIG. 2 is a diagram illustrating a system architecture provided in an implementation of the present application;

fig. 3 is a flowchart illustrating a method for load balancing according to an embodiment of the present application;

fig. 4 is a schematic flowchart of another method for load balancing according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of another system architecture provided by an embodiment of the present application;

fig. 6 is a schematic flowchart of an application service start management method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of another system architecture provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of an apparatus for load balancing according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of another apparatus for load balancing according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," and the like in the description and in the claims, and in the drawings, of the embodiments of the application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than described or illustrated herein.

It will be understood that the terms "comprises" and "comprising," and any variations thereof, as used herein, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the description of the embodiments of the present application, the term "correspond" may indicate that there is a direct correspondence or an indirect correspondence between the two, may also indicate that there is an association between the two, and may also indicate and is indicated, configure and is configured, and the like.

To facilitate understanding of the technical solutions of the present application, a service grid framework and related concepts and terms are first described with reference to fig. 1.

FIG. 1 is a schematic diagram of a services grid. The service grid 100 is primarily used to facilitate secure and reliable communications among multiple microservices, which refers to decomposing an application into multiple smaller services or instances and running distributed across different clusters/machines.

As shown in FIG. 1, the microservice includes an application service instance A and an application service instance B, which form a functional application layer of the services grid 100. In one embodiment, the application service instances A and B run in the form of containers/processes on a machine/workload container group (machine/pod) 108.

In one embodiment, application service instance a may be a commodity inquiry service and application service instance B may be a commodity ordering service.

As shown in FIG. 1, application service instance A and grid agent (sidecar) 103 coexist in machine/workload container set 114, and application service instance B and grid agent 105 coexist in machine/workload container 114. Mesh agents 103 and 105 form the data plane layer (data plane) of serving mesh 100. Wherein grid agent 103 is running in the form of container/process 104, grid agent 105 is running in the form of container/process 106, and both-way communication is possible between grid agent 103 and application service instance a, and both-way communication is possible between grid agent 105 and application service instance B. In addition, two-way communication may also be provided between the grid agent 103 and the grid agent 105.

In one embodiment, all traffic for application service instance A is routed through the grid proxy 103 to the appropriate destination and all network traffic for application service instance B is routed through the grid proxy 105 to the appropriate destination. It should be noted that the network traffic mentioned herein includes, but is not limited to, traffic based on the following forms: hypertext transfer protocol (HTTP), representational state transfer (REST), Remote Procedure Call (RPC), remote dictionary service (Redis), and the like.

In one embodiment, the functionality of extending the data plane layer may be implemented by writing custom filters (filters) for the grid agents 103, 105 in the service grid 100, and the service grid agent configuration may be for the service grid to properly proxy service traffic, for service interworking, and for service governance. The grid agents 103 and 105 may be configured to perform at least one of the following functions: service discovery (service discovery), health checking (health checking), routing (routing), load balancing (load balancing), authentication and authorization (authentication and authorization), and observability (observability).

As shown in fig. 1, the services grid 100 also includes a control plane layer. Where the control plane layer may be a group of services running in a dedicated namespace, these services are hosted by the hosting control plane component 101 at the machine/workload container set 102. As shown in fig. 1, the hosted control plane component 101 is in two-way communication with the mesh agent 103 and the mesh agent 105. The managed control plane component 101 is configured to perform some control management functions. For example, the hosting control plane component 101 receives data transmitted by the mesh agents 103 and 105, which may be further aggregated. The hosted control plane component 101 may also provide a user-oriented Application Programming Interface (API) to more easily manipulate network behavior, provide configuration data to the grid agents 103 and 105, and the like.

Under the micro-service architecture, the application service instance has a dynamically allocated network address, and the network address of the application service instance is dynamically changed along with automatic expansion, failure and release upgrading of the application service. Therefore, a mechanism is needed to support a service consumer to timely sense and obtain the latest address of an application service instance when the address of the application service instance provided by a service provider is changed, and the mechanism is a service discovery mechanism.

The method and the device are suitable for realizing the load balancing scene of the target application service when service calling is carried out among different application services. How to implement load balancing of target application services based on a service discovery mechanism is described below in conjunction with fig. 1.

The service discovery mechanism includes two procedures: a self-registration procedure and a service discovery procedure. In the self-registration process, the proxy grid corresponding to each application service instance is responsible for calling a registration interface of the service registry to register an instance address and calling a heartbeat interface regularly to renew the service instance registration information. For example, in fig. 1, the application service instance a registers its own address information with the hosted control plane component 101 by calling the registration interface, and the application service instance B registers its own address information with the hosted control plane component 101 by calling the registration interface. In addition, the operation and maintenance personnel may configure load balancing policy information corresponding to each application service to the managed control plane component 101.

During the service discovery process, the hosted control plane component 101 pushes the following information to the mesh proxy (e.g., mesh proxy 103, mesh proxy 105) corresponding to each application service instance: address information of each application service instance, a load balancing strategy corresponding to each application service, and the like. The push information received by the grid agent is cached locally.

When a first application service (for example, application service instance a in fig. 1) needs to invoke a second application service, the invocation request is forwarded to the grid proxy 103, and the grid proxy 103 determines an address of the second application service and a load balancing policy of the second application service by querying locally cached push information, and selects one available application service instance (for example, application service instance B in fig. 1) from a plurality of application service instances of the second application service based on the load balancing policy of the second application service. In turn, the grid proxy 103 forwards the invocation request to the grid proxy 105 corresponding to application service instance B. The grid proxy 105 sends the invocation request to application service instance B, thereby completing the invocation of application service instance a to application service instance B. Through the process, the load balance of each application service instance in the second application service is realized.

In the service discovery process, the hosted control plane component 101 and the network proxy 103 may communicate based on an (x discovery service, xDS) protocol. Wherein, the control plane component 101 is hosted as a server of xDS protocol, and the grid agent 103 is served as a client of xDS protocol. xDS includes but is not limited to the following protocols:

listener Discovery Service (LDS);

route Discovery Service (RDS);

a Scoped Route Discovery Service (SRDS);

virtual Host Discovery Service (VHDS);

cluster Discovery Service (CDS);

an Endpoint Discovery Service (EDS);

secret Discovery Service (SDS);

runtime discovery service (RTDS).

Based on the service grid shown in fig. 1, the grid agent realizes the service management function, so that each application service instance can only pay attention to the application layer function of itself, and does not need to pay attention to the service management functions such as load balancing, and the decoupling of the functional application layer and the data plane layer is realized.

In the service grid architecture shown in fig. 1, each application service instance corresponds to a grid proxy, which may be referred to as proxy (proxy) mode. When load balancing is implemented based on the proxy mode, service response time is increased because all call requests need to be forwarded by the mesh proxy.

In order to solve the foregoing technical problem, embodiments of the present application provide a method, an apparatus, a system, and a storage medium for load balancing. In the technical solution of the present application, the application service instance may adopt a proxy-free (proxy) mode, that is, a grid proxy corresponding to the application service instance is cancelled, and load balancing is implemented inside the application service instance. An application scenario related to the embodiment of the present application is described below with reference to fig. 2.

Fig. 2 is a schematic diagram of a system architecture provided in an implementation of the present application. As shown in fig. 2, the system architecture includes: xDS servers deployed at the control plane level, and a microservice architecture-based cluster deployed at the data plane level. Wherein xDS server corresponds to the hosted control plane component 101 in FIG. 1. A plurality of application services are included in the cluster. The first application service and the second application service are illustrated in fig. 2 as an example. It should be understood that the number of clusters included in the system architecture may be one or more, which is not limited in this embodiment.

With continued reference to fig. 2, the first application service includes: a first application service instance, the first application service instance comprising: a first workload and xDS clients. Wherein, the first workload is used for providing certain services, such as commodity inquiry service, commodity ordering service, etc. xDS client as the client of xDS protocol and xDS server as the server of xDS protocol, both implement the service discovery process based on xDS protocol.

The first workload may invoke other application services. Taking a scenario of calling the second application service as an example, the second application service includes a plurality of second application service instances, and fig. 2 illustrates 3 second application service instances as an example.

With continued reference to fig. 2, the first application service further includes: xDS proxy (xDS agent). In the embodiment of the application, the xDS client and the xDS server do not directly establish a connection channel, but a xDS proxy is deployed between the xDS server and the xDS client. That is, the xDS proxy acts as an intermediary between the xDS client and the xDS server, i.e., the xDS proxy establishes a connection with the xDS server and the xDS proxy establishes a connection with the xDS client. As such, the xDS proxy can receive xDS server pushed load balancing policy information and provide xDS clients with the load balancing policy information. Thus, when the first workload invokes the second application service, the xDS client may send an invocation request to one or more target second application service instances of the second application service based on the load balancing policy information to implement load balancing of the second application service.

It should be understood that multiple first application service instances, only one of which is illustrated in fig. 2, may be included in the first application service. When the first application service comprises a plurality of first application service instances, each first application service instance comprises a first workload and a corresponding xDS client. There is one xDS proxy for each first application service instance.

In the embodiment of the application, the first application service instance adopts the proxy-free mode, and when the first application service instance calls the second application service, the call request does not need to be forwarded by the grid proxy, so that the service response time can be reduced.

The technical solutions provided in the embodiments of the present application are described in detail below with specific embodiments. It should be noted that the technical solutions provided in the embodiments of the present application may include part or all of the following contents, and these specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 3 is a flowchart illustrating a method for load balancing according to an embodiment of the present application. The method of the present embodiment can be applied to the system architecture shown in fig. 2. The system architecture comprises: xDS servers deployed at the control plane layer and clusters deployed at the data plane layer. Illustratively, the cluster may be a kubernets cluster.

A plurality of application services are included in the cluster. Taking an example that the cluster includes a first application service and a second application service, the first application service includes: a first application service instance and an xDS proxy. The first application service instance includes a first workload and an xDS client. The second application service includes a plurality of second application service instances.

As shown in fig. 3, the method of this embodiment includes:

s301: xDS the server sends xDS the proxy first load balancing policy information.

In this embodiment, xDS servers are deployed at the control plane layer, and xDS servers adopt a hosting mode for the management of clusters. The hosting mode refers to that components for realizing the cluster management function run in an independent server instead of a certain cluster of a data plane layer. Optionally, the xDS server may be a component hosted in a cloud network, so that a management function of a cluster may be implemented in the cloud, and complexity and cost of operation and maintenance of a cluster user are reduced.

xDS the server and xDS proxy may communicate based on the xDS protocol. Specifically, xDS servers and xDS proxies communicate between themselves through APIs of the xDS protocol, which are essentially publish/subscribe mechanisms, each of which allows for subscription to a different type of service. xDS, the letter x indicates that it is a variable, and possible options x include, but are not limited to: LDS, RDS, SRDS, VHDS, CDS, EDS, SDS, RTDS. Illustratively, all application service instances in the cluster register themselves with xDS servers through the EDS API. When an application service instance is overloaded, it can self-unregister with the xDS server, and then when it is available again, it can register again with the xDS server.

In this embodiment of the application, taking a scenario in which the first application service invokes the second application service as an example, the first load balancing policy information sent by the xDS server to the xDS proxy at least includes a load balancing policy corresponding to the second application service. For example, the first load balancing policy information may include load balancing policies corresponding to all application services in the cluster. For another example, the first load balancing policy information may include load balancing policies corresponding to other application services that the first application service needs to invoke.

At the control plane layer, a user can configure a load balancing policy of each application service to the xDS server according to the load balancing requirement of each application service in the cluster. For example, taking the second application service as an example, assuming that the second application service includes 3 second application service instances, which are the second application service instance a, the second application service instance B, and the second application service instance C, respectively, the user may configure, to the xDS server, the traffic proportion corresponding to each second application service instance, for example, the traffic proportion of the second application service instance a is 20%, the traffic proportion of the second application service instance B is 30%, and the traffic proportion of the second traffic service instance C is 50%. In this case, the load balancing policy corresponding to the second application service is used to indicate traffic ratio information corresponding to each of the 3 second application service instances.

In this embodiment, when a load balancing policy corresponding to a certain application service is changed, for example, when traffic proportion information configured by a user for a second application service is changed, or when a certain application service instance in the second application service is logged off/re-registered, the xDS server pushes updated first load balancing policy information to the xDS proxy.

S302: xDS the proxy forwards the first load balancing policy information to xDS client.

In the embodiment of the application, the xDS proxy serves as an intermediary between the xDS server and the xDS client, and the xDS proxy forwards the first load balancing information to the xDS client after receiving the first load balancing policy information from the xDS server. xDS after receiving the first load balancing information, the client may cache the first load balancing information for use when a subsequent first application service instance needs to make a service call.

S303: when the first workload needs to invoke the second application service, sending, by the xDS client, an invocation request to one or more target second application service instances in the second application service according to the first load balancing policy information, so as to implement load balancing of each second application service instance in the second application service.

For example, suppose that the xDS client determines, according to the first load balancing policy information, that traffic ratios corresponding to 3 second application service instances in the second application service are: 20%, 30%, 50%, the xDS client can determine which second application service instance or instances to send the invocation request to based on the traffic duty. In this process, the xDS client may determine one or more target second application service instances in a plurality of second application service instances included in the second application service, and send a call request to the one or more target second application service instances, according to the call condition of the client itself to each instance of the second application service within the historical period and the first load balancing policy information, thereby implementing load balancing of each second application service instance in the second application service.

The method for load balancing provided by this embodiment is applied to a first application service in a cluster, where the first application service includes: a first application service instance and an xDS proxy, the first application service instance comprising: a first workload and xDS client, which receives xDS first load balancing strategy information sent by a server through xDS agents and forwards the first load balancing strategy information to the xDS client; when the first workload needs to invoke the second application service in the cluster, sending, by the xDS client, an invocation request to one or more target second application service instances in the second application service according to the first load balancing policy information, so as to implement load balancing of each second application service instance in the second application service. Through the process, load balancing can be achieved in the first application service instance, a grid proxy does not need to be deployed for the first application service instance, and when the first application service instance calls the second application service, the call request does not need to be forwarded through the grid proxy, so that service response time can be reduced.

In addition, in this embodiment of the application, each second application service instance in the second application service may be in a proxy mode or in a non-proxy mode, which is not limited in this embodiment of the application. When the second application service instance is in the proxy mode, the second application service also comprises a grid proxy corresponding to each second application service instance. When the second application service instance is in the proxy-free mode, the second application service instance comprises: a second workload and xDS clients.

Further, in an actual application scenario, each application service in the cluster is typically implemented based on a specific language framework, and the language framework based on which different application services are typically different. The application service instances in each application service are commonly referred to as framework or Software Development Kit (SDK) specific xDS processors. In embodiments of the present application, the first application service instance and the xDS proxy may be implemented decoupled. That is, for each application service, its corresponding xDS agent need not be based on the particular language framework implementation of that application service, but the xDS agents in different application services in the cluster may be implemented based on the same programming language framework. Wherein, the implementation based on the same programming language framework means that: the method is realized by adopting the same programming language, or based on the same programming framework, or based on the same SDK.

Therefore, the corresponding xDS agent does not need to be developed for each application service, but a unified xDS agent can be developed for all application services, and all application services in the cluster can share one code of the xDS agent, so that the development cost and the maintenance cost can be reduced.

Therefore, the embodiment of the application realizes a mechanism for uniformly managing various application services in the cluster through the xDS server. By deploying xDS agents for application service instances and implementing xDS agents in different application services based on the same programming language framework, uniform management of application services can be conveniently achieved even in a scenario where the application services in a cluster are based on different programming language frameworks.

In addition, in the embodiment of the application, the example of one part of the application service in the cluster is allowed to adopt a proxy mode, and the example of the other part of the application service adopts a non-proxy mode. That is, even in the case of a heterogeneous cluster (application services in both proxy mode and non-proxy mode), it is possible to realize xDS server-based unified management of each application service.

On the basis of the embodiment shown in fig. 3, the technical solution of the present application is described in more detail below with reference to a specific embodiment.

Fig. 4 is a flowchart illustrating another method for load balancing according to an embodiment of the present application. As shown in fig. 4, the method of the present embodiment includes:

s401: xDS the server sends xDS the proxy first load balancing policy information.

S402: xDS the proxy forwards the first load balancing policy information to xDS client.

It should be understood that the specific implementation manner of S401 and S402 is similar to that of the embodiment shown in fig. 3, and is not described herein again.

S403: xDS, when monitoring that the first workload sends a call request, the client intercepts the call request, analyzes the call request, and determines a second application service to be called.

The first workload may send a call request to the other application service when the other application service needs to be called. In this embodiment, the xDS client has a function of monitoring the first workload, and can monitor a call request sent by the first workload. After the client monitors the call request at xDS, the call request may be parsed to determine the application service to be called.

For example, assuming that the address + port of the second application service is "mytest.com: 8888", and the call request intercepted by the xDS client is "xds: mytest.com: 8888" or "xds:// mytest.com: 8888", the application service to be called is determined to be the second application service by parsing "mytest.com: 8888" in the call request.

S404: xDS the client obtains the load balancing policy corresponding to the second application service from the first load balancing policy information, and determines one or more target second application service instances in a plurality of second application service instances included in the second application service according to the load balancing policy corresponding to the second application service.

Illustratively, the first load balancing policy information includes identifiers of a plurality of application services and load balancing policies corresponding to the identifiers. xDS after determining the second application service to be invoked, the client may extract the load balancing policy corresponding to the second application service from the first load balancing policy information according to the identifier of the second application service. And further, according to a load balancing strategy corresponding to the second application service, determining one or more target second application service instances in a plurality of second application service instances included in the second application service.

For example, assume that the second application service includes: second application service instance a, second application service instance B, and second application service instance C, the xDS client may regard the second application service instance a as the target second application service instance, or the second application service instance a and the second application service instance B as the target second application service instance, based on the load balancing policy corresponding to the second application service.

xDS the client sends the invocation request to the one or more target second application service instances.

Since the one or more target second application service instances are determined based on the load balancing policy corresponding to the second application service, the xDS client sends the invocation request to the one or more target second application service instances, so that load balancing among the second application service instances in the second application service can be achieved.

In some possible implementations, after S403, the method may further include: determining whether a connection channel between the first application service and the second application service exists; and if the address information does not exist, establishing a connection channel between the first application service and the second application service according to the address information and the port information of the second application service. If so, in S405, the xDS client sends the invocation request to the one or more target second application service instances through the connection channel.

Based on the embodiment shown in fig. 4, a specific implementation of the xDS client is illustrated in the following with reference to fig. 5. Fig. 5 is a schematic diagram of another system architecture according to an embodiment of the present application. As shown in fig. 5, on the basis of the system architecture shown in fig. 2, in this embodiment, the xDS client may include: xDS interceptor, xDS parser, channel creator, listen requestor, sub-policy creator, and sender.

Wherein, the xDS interceptor is used for intercepting the call request sent by the first workload. The xDS resolver is used for resolving the intercepted call request. The channel creator is used for establishing a connection channel between the first application service and the second application service. The monitoring requester and the sub-policy creator are used for acquiring a load balancing policy corresponding to the second application service from the first load balancing policy information. The transmitter is configured to transmit a call request to one or more target second application service instances in the second application service based on the load balancing policy.

Based on the system architecture shown in fig. 5, when the first workload sends a call request (e.g., "xds: mytest. com: 8888" or "xds:///mytest. com: 8888"), the call request is intercepted by a xDS interceptor, and the intercepted call request is parsed by a xDS parser, so that the application service to be called is determined to be the second application service (i.e., the application service indicated by "mytest. com: 8888"). Then, the xDS client determines whether a connection channel between the first application service and the second application service exists, and if not, establishes the connection channel between the first application service and the second application service through the channel creator according to the address information and the port information of the second application service. The connection channel is used for the first application service to send a call request to the second application service. Further, a load balancing strategy corresponding to the second application service is obtained from the first load balancing strategy information through the monitoring requester and the sub-strategy creator. For example, referring to fig. 5, the monitoring requester acquires the CDS policy corresponding to the second application service from the first load balancing policy information, and further acquires the EDS policy corresponding to the second application service through the sub-policy creator. The EDS policy serves as a sub-policy of the CDS policy. In this way, the CDS policy and the EDS policy serve as load balancing policies corresponding to the second application service. Further, the sender determines one or more target second application service instances from a plurality of second application service instances included in the second application service based on the CDS policy and the EDS policy, and sends a call request to the one or more target second application service instances through the connection channel.

In this embodiment, the xDS proxy provides the first load balancing policy information to the xDS client, and the xDS client intercepts the call request sent by the first workload and forwards the intercepted call request to the available instance of the second application service based on the first load balancing policy, so that load balancing control is implemented in the first application service instance, and thus, a grid proxy does not need to be deployed for the first application service instance, the grid proxy does not need to forward the call request, and the service response time is reduced.

On the basis of any of the above embodiments, the xDS server can also use xDS agent to implement unified startup management of application service instances in the cluster. This is described below in conjunction with fig. 6 and 7.

Fig. 6 is a flowchart illustrating an application service start management method according to an embodiment of the present application. Fig. 7 is a schematic diagram of another system architecture provided in the embodiment of the present application. The method of the present embodiment may be performed prior to the embodiment shown in fig. 3 or fig. 4. As shown in fig. 6, the method of the present embodiment includes:

s601: and when a starting instruction corresponding to the first application service instance is received, starting the xDS proxy.

The starting instruction is used for indicating the starting of the first application service. Illustratively, when a first application service instance needs to be launched, a user or a control device inputs a launch instruction to the first application service. After the first application service receives the launch instruction corresponding to the first application service instance, the agent is launched xDS.

In one possible implementation, the agent may be launched xDS as follows: second launch configuration information required to launch xDS agents is obtained, the second launch configuration information configured by the agent manager for xDS agents. Illustratively, parameters required for the proxy boot process may be included xDS in the second boot configuration information. And in turn, activates xDS the agent according to the second activation configuration information.

This will be explained with reference to fig. 7. As shown in fig. 7, a proxy manager is also deployed at the control plane layer. The agent manager may be deployed as a component of the control plane layer. The agent manager is used for managing xDS agents in each application service in the cluster.

In this embodiment, the agent manager may implement unified management on the start process of the xDS agents in each application service. In one example, the agent manager may distribute second launch configuration information to xDS agents in each application service, the second configuration information being configuration information required by the xDS agent launch process. For example, the agent manager may write second startup configuration information required for each xDS agent startup process into a configuration file under a preset path corresponding to the xDS agent. Thus, when the xDS agent needs to be booted, the second boot configuration information can be read from the configuration file under the preset path, and the agent is booted xDS according to the second boot configuration information.

In this embodiment, the agent manager is deployed at the control plane layer and configured to configure the second startup configuration information required by the startup process for the xDS agent in each application service, so that the agent manager can implement unified management on the startup process for the xDS agent in each application service, and improve the management efficiency of the xDS agent.

S602: first launch configuration information is obtained from a xDS server through an xDS proxy, wherein the first launch configuration information is configuration information required for launching the first application service instance.

In this embodiment, after the xDS proxy is started, first startup configuration information required for starting a first application service instance (e.g., a first workload, xDS client) may be obtained through an interactive process with the xDS server. The first start-up configuration information may include parameters required during the start-up of the first application service instance.

In one possible implementation manner, S602 may be implemented as follows:

(1) xDS the proxy establishes a connection with the xDS server.

xDS the agent and xDS server establish a communication connection based on xDS protocol. Illustratively, the communication connection between the two is established as follows: xDS proxy obtains certificate information and sends connection establishment request to xDS server according to the certificate information, the connection establishment request includes the certificate information. xDS, the server verifies the certificate information and sends xDS proxy connection establishment response after verification is passed. Accordingly, the xDS proxy receives a connection setup response from the xDS server. Thus, xDS proxies connect communication connections with xDS servers. In the above process, the certificate information is configured by the agent manager for the xDS agent.

With continued reference to FIG. 7, the proxy manager may distribute the credential information to xDS proxies in the respective application services. For example, the agent manager may write the certificate information corresponding to each xDS agent to the configuration file under the preset path corresponding to the xDS agent. Thus, when the xDS agent needs to establish a connection with the xDS server, the certificate information can be read from the configuration file under the preset path and carried to the connection establishment request, so that the xDS server authenticates the xDS agent based on the certificate information.

In this embodiment, by deploying xDS agents for each application service instance and deploying an agent manager on a control plane layer, the agent manager configures certificate information for each xDS agent, so that on one hand, a uniform certificate issuance management mechanism of each xDS agent is implemented, and on the other hand, a uniform connection mechanism between each xDS agent and a xDS server is implemented. In addition, the application service instance is also enabled to support the ability to locally manage the security certificate lifecycle.

(2) xDS the proxy receives second load balancing policy information from the xDS server.

For example, after the xDS proxy establishes a connection with the xDS server, the xDS proxy may request the xDS server for the second load balancing policy information, or the xDS server may actively send the second load balancing policy information to the xDS proxy. The second load balancing policy information includes: when the xDS proxy establishes a connection with the xDS server, the load balancing policy corresponding to all application services in the cluster, or the load balancing policy corresponding to other application services that the first application service instance needs to invoke. If the load balancing policy corresponding to a certain application service (e.g., a second application service) is not configured when the proxy establishes a connection with the xDS server at xDS, the second load balancing policy may include a default load balancing policy corresponding to the application service.

(3) xDS, the agent generates the first start-up configuration information according to the second load balancing policy information, the first start-up configuration information including the second load balancing policy information.

The first start-up configuration information is configuration information required for starting the first application service instance, and the first start-up configuration information may include parameters required for a start-up process of the first application service instance. In this embodiment, the xDS agent may generate the first launch configuration information required by the first application service instance based on the second load balancing policy information. The first startup configuration information may include second load balancing policy information.

S603: and starting the first application service instance according to the first starting configuration information.

Illustratively, the first workload is launched and the xDS client is launched according to the first launch configuration information, such that the launching of the first application service instance is completed.

It should be noted that fig. 7 mainly illustrates a starting process of the first application service instance. For the second application service, if the second application service uses the agentless mode, the starting process of each instance in the second application service is similar to that of the first application service instance, and details are not repeated here.

In this embodiment, by deploying xDS a proxy for an application service instance, when the application service instance is started, first start-up configuration information may be acquired from xDS server through xDS the proxy, and the application service instance is started based on the first start-up configuration information. Therefore, a mechanism that the xDS server uniformly manages the starting process of each application service instance is realized, and the starting management efficiency of each application service instance is improved.

The method for load balancing provided by the embodiment of the present application is described above, and the apparatus for load balancing provided by the embodiment of the present application will be described below.

In the embodiment of the present application, the functional modules of the apparatus for load balancing may be divided according to the method embodiment, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a form of hardware or a form of a software functional module.

It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation. The following description will be given by taking an example in which each functional module is divided by using a corresponding function.

Fig. 8 is a schematic structural diagram of an apparatus for load balancing according to an embodiment of the present application. The apparatus provided in this embodiment may be applied to a first application service in a cluster, where the first application service includes: a first application service instance and xDS proxy, the first application service instance comprising: a first workload and xDS clients. As shown in fig. 8, the apparatus 800 for load balancing provided in this embodiment includes: a policy acquisition module 801 and a policy enforcement module 802. Wherein the content of the first and second substances,

a policy obtaining module 801, configured to receive xDS first load balancing policy information sent by a server through the xDS proxy, and forward the first load balancing policy information to the xDS client;

a policy execution module 802, configured to send, by the xDS client, a call request to one or more target second application service instances in the second application service according to the first load balancing policy information when the first workload needs to call the second application service in the cluster, so as to implement load balancing of each second application service instance in the second application service.

In a possible implementation manner, the apparatus provided in this embodiment further includes: a start module to:

In a possible implementation manner, the starting module is specifically configured to:

establishing a connection with the xDS server through the xDS proxy;

obtaining certificate information through the xDS proxy, and sending a connection establishment request to the xDS server according to the certificate information, wherein the connection establishment request comprises the certificate information; wherein the certificate information is configured by an agent manager for the xDS agent; the agent manager is used for managing xDS agents in each application service in the cluster;

In a possible implementation manner, the policy execution module 802 is specifically configured to:

performing, by the xDS client, the following steps:

acquiring a load balancing strategy corresponding to the second application service from the first load balancing strategy information, and determining the one or more target second application service instances in a plurality of second application service instances included in the second application service according to the load balancing strategy corresponding to the second application service;

In a possible implementation manner, the policy execution module 802 is further configured to:

The apparatus for load balancing provided in this embodiment may perform the method for load balancing implemented by the first application service in any of the above method embodiments, and the implementation principle and the technical effect are similar, and are not described herein again.

Fig. 9 is a schematic structural diagram of another apparatus for load balancing according to an embodiment of the present disclosure. The device provided by the embodiment can be applied to xDS servers. As shown in fig. 9, the apparatus 900 for load balancing provided in this embodiment includes: an obtaining module 901 and a sending module 902. Wherein the content of the first and second substances,

an obtaining module 901, configured to obtain first load balancing policy information corresponding to a first application service instance in a cluster;

a sending module 902, configured to send the first load balancing policy information to an xDS proxy corresponding to the first application service instance, so that the xDS proxy forwards the first load balancing policy information to a xDS client in the first application service instance, where the first load balancing policy information is used to implement load balancing of each second application service instance in the second application service when a first workload in the first application service instance invokes a second application service in the cluster.

The apparatus for load balancing provided in this embodiment may be configured to execute the method for load balancing executed by the xDS server in any of the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

An embodiment of the present application further provides a system for load balancing, where the system includes: a service discovery xDS server deployed at a control plane layer and a cluster deployed at a data plane layer, the cluster including a first application service and a second application service, the first application service including a first application service instance and xDS agents, the first application service instance including a first workload and xDS clients, the second application service including a plurality of second application service instances;

In a possible implementation manner, the agent manager is configured to distribute the second startup configuration information to xDS agents in each application service in the cluster, and/or distribute the certificate information to xDS agents in each application service in the cluster.

The system provided by this embodiment may be used to implement the method for load balancing provided by any of the above method embodiments, and the implementation principle and technical effect are similar, which are not described herein again.

Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 10, the electronic device 1000 provided in this embodiment includes: a memory 1001 and at least one processor 1002; the memory 1001 stores therein a computer program configured to be executed by the at least one processor 1002 to implement the method for load balancing implemented by the first application service in any of the above method embodiments, or the method for load balancing implemented by the xDS server, which has similar implementation principles and technical effects and will not be described herein again.

Alternatively, the memory 1001 may be separate or integrated with the processor 1002. When the memory 1001 is a separate device from the processor 1002, the electronic device 1000 further includes: a bus 1003 is used to connect the memory 1001 and the processor 1002.

The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method for load balancing implemented by the first application service in any of the foregoing method embodiments, or the method for load balancing implemented by the xDS server, where the implementation principle and the technical effect are similar, and details are not described herein.

An embodiment of the present application provides a computer program product, including a computer program, where the computer program, when executed by a processor, implements the method for load balancing implemented by the first application service in any of the foregoing method embodiments, or implements the method for load balancing implemented by the xDS server, and implementation principles and technical effects of the method are similar, which are not described herein again.

An embodiment of the present application further provides a chip, including: the load balancing method implemented by the first application service in any of the above method embodiments, or the load balancing method implemented by the xDS server, is similar in implementation principle and technical effect, and is not described herein again.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.

The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.

The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A method for load balancing, applied to a first application service in a cluster, the first application service comprising: a first application service instance and service discovery xDS proxy, the first application service instance comprising: a first workload and xDS client; the method comprises the following steps:

2. The method of claim 1, wherein before receiving xDS the first load balancing policy information sent by the server through the xDS proxy, the method further comprises:

3. The method of claim 2, wherein obtaining, by the xDS proxy, first launch configuration information from the xDS server comprises:

establishing a connection with the xDS server through the xDS proxy;

generating, by the xDS agent, the first boot configuration information according to the second load balancing policy information, the first boot configuration information including the second load balancing policy information.

4. The method of claim 3, wherein establishing a connection with the xDS server through the xDS proxy comprises:

5. The method of claim 4, wherein initiating the xDS agent comprises:

6. The method of any of claims 1 to 5, wherein sending, by the xDS client, a call request to one or more target second application service instances in the second application service according to the first load balancing policy information when the first workload requires a call to the second application service in the cluster comprises:

performing, by the xDS client, the steps of:

7. The method of claim 6, wherein after parsing the invocation request and determining the second application service to be invoked, further comprising:

8. The method of any of claims 1 to 5, wherein xDS proxies in different application services in the cluster are implemented based on the same programming language framework.

9. A method for load balancing, applied to a service discovery xDS server, the method comprising:

10. A method for load balancing, applied to a system for load balancing, the system comprising: a service discovery xDS server deployed at a control plane layer and a cluster deployed at a data plane layer, the cluster including a first application service and a second application service, the first application service including a first application service instance and xDS agents, the first application service instance including a first workload and xDS clients, the second application service including a plurality of second application service instances; the method comprises the following steps:

11. A system for load balancing, comprising: a service discovery xDS server deployed at a control plane layer and a cluster deployed at a data plane layer, the cluster including a first application service and a second application service, the first application service including a first application service instance and xDS agents, the first application service instance including a first workload and xDS clients, the second application service including a plurality of second application service instances;

12. The system of claim 11, further comprising: an agent manager deployed at the control plane layer;

13. An electronic device, comprising: a memory having stored therein a computer program configured to be executed by the at least one processor to implement the method of any one of claims 1 to 8 or the method of claim 9, or at least one processor.

14. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 8, or the method of claim 9.