CN113467892A

CN113467892A - Distributed cluster resource configuration method and corresponding device, equipment and medium

Info

Publication number: CN113467892A
Application number: CN202110794106.3A
Authority: CN
Inventors: 蔡云雷; 石志伟
Original assignee: Guangzhou Huaduo Network Technology Co Ltd
Current assignee: Guangzhou Huaduo Network Technology Co Ltd
Priority date: 2021-07-14
Filing date: 2021-07-14
Publication date: 2021-10-01

Abstract

The application relates to the technical field of e-commerce information, and discloses a distributed cluster resource configuration method and a corresponding device, equipment and medium thereof, wherein the method comprises the following steps: cloning a mirror image of a running container deployed in a distributed cluster's microservice architecture; detecting pressure measurement indexes generated by performing pressure tests on the mirror images, and triggering a container configuration event when at least one target pressure measurement index overflows a corresponding preset range; responding to the container configuration event, and calculating and determining the lowest price cost in a plurality of configuration schemes according to unit pricing of each hardware resource required by the operation of the container; and configuring a container template of the micro-service framework according to a configuration scheme with the lowest price cost so that the micro-service framework obtains each hardware resource of a corresponding quota according to a container derived from the container template. The method and the device can ensure that the utilization rate of the micro-service architecture to the cluster hardware resources is maximized, and simultaneously ensure that the cost is minimized.

Description

Distributed cluster resource configuration method and corresponding device, equipment and medium

Technical Field

The embodiment of the application relates to the technical field of e-commerce information, in particular to a distributed cluster resource configuration method and a corresponding device, equipment and medium thereof.

Background

In order to realize effective calling of server resources in a distributed cluster, a relatively efficient mode is to deploy a micro-service architecture in the distributed cluster, and containerize the server hardware resources in the cluster through the micro-service architecture, so that internet services can be operated in each corresponding container instance, and direct management of the hardware resources is not required to be considered in the operation process, so that the micro-service architecture can realize efficient calling of the hardware resources of the distributed cluster only by carrying out logic management on each container.

The internet service widely uses a micro service architecture to realize the utilization of the distributed cluster, and typical internet services are search engines, e-commerce systems, live network systems and the like, wherein the e-commerce systems have higher requirements on the resource configuration capability of the micro service architecture, because the e-commerce systems require that the online service thereof can instantaneously respond, efficiently process concurrent data, and flexibly and compatibly have rapid changes of data traffic, and the like, the e-commerce field has higher requirements on the efficiency of the micro service architecture for realizing the resource configuration.

Specifically, in the e-commerce field, search services are applied to various scenes, and relate to thousands of search clusters, and the capacity of the clusters is dynamically changed along with continuous change of services, so how to automatically evaluate the maximum capacity of each cluster, optimize the configuration of containers in a micro-service architecture, reasonably guide services to expand and contract the capacity in time, and avoid hardware resource waste is a problem to be solved urgently.

In the prior art, methods for evaluating cluster capacity make a resource scheduling strategy for pursuing high resource utilization rate, and neglect the factor of optimal cost. For example, a search engine with 3 copies and 4 fragments has 16 container instances, each container is 2 cores 32G in specification, and even if the CPU utilization rate reaches 70%, the deployment structure still has half of the waste of memory resources. Therefore, the resource configuration technology for the distributed cluster adopted by the micro service architecture in the prior art is prone to cause resource waste and result in higher overall cost, and needs to be improved.

Disclosure of Invention

The present application is directed to at least some of the disadvantages in the prior art, and provides a distributed cluster resource configuration method and a corresponding apparatus, computer device, and storage medium.

In order to solve the technical problem, the application adopts a technical scheme that:

the distributed cluster resource configuration method comprises the following steps:

cloning a mirror image of a running container deployed in a distributed cluster's microservice architecture;

detecting pressure measurement indexes generated by performing pressure tests on the mirror images, and triggering a container configuration event when at least one target pressure measurement index overflows a corresponding preset range;

in response to the container configuration event, calculating and determining the lowest price cost of a plurality of configuration schemes according to unit pricing of each hardware resource required by the operation of the container, wherein the configuration schemes comprise quotas of each hardware resource allocated for creating the container with the better target pressure measurement index;

and configuring a container template of the micro-service framework according to a configuration scheme with the lowest price cost so that the micro-service framework obtains each hardware resource of a corresponding quota according to a container derived from the container template.

In one embodiment, detecting pressure measurement indicators generated by performing a pressure test on the image, and triggering a container configuration event when at least one target pressure measurement indicator overflows a corresponding preset range includes the following steps:

calling a test interface to carry out maximum pressure test aiming at the target pressure test index on the mirror image;

acquiring all pressure measurement indexes generated after the maximum pressure test is carried out, and determining the target pressure measurement index;

and judging whether the target pressure measurement index is lower than a corresponding preset threshold value, and if so, judging that the target pressure measurement index overflows a preset range defined by the preset threshold value, thereby triggering a container configuration event.

In a preferred embodiment, the target pressure measurement indicator is any one of throughput, query rate per second, and number of concurrent users.

In one embodiment, the method for computationally determining the lowest price cost of the plurality of configuration solutions according to unit pricing of each hardware resource required for the operation of the container comprises the steps of:

acquiring quotas of hardware resources corresponding to the configuration schemes;

adapting to each configuration scheme, calculating the single cost of each hardware resource, and linearly integrating the single cost of all hardware resources required by the same configuration scheme into the price cost of the current configuration scheme;

and searching and determining the configuration scheme with the lowest price cost from all the configuration schemes.

In a preferred embodiment, each configuration scheme is adapted, a single cost of each hardware resource is calculated, and the single costs of all hardware resources required by the same configuration scheme are linearly merged into a price cost of the current configuration scheme, which includes the following steps:

and calling a preset pricing formula to calculate the price cost for each configuration scheme, wherein in the pricing formula, the unit cost of each hardware resource is determined by the product of the unit pricing of each hardware resource and the quota of each hardware resource, and the unit cost of each hardware resource is matched with different weight parameters respectively and then summed to obtain the price cost of each configuration scheme.

In a preferred embodiment, each hardware resource required for the container to run includes a CPU, a memory, and a hard disk of a computer device, and the relationship between the respective weight parameters is arranged from large to small according to the characterized weight: CPU, internal memory, hard disk.

In a further extended embodiment, before the step of obtaining the quotas of the hardware resources corresponding to the configuration schemes, the method includes the following steps:

receiving input information of quotas of hardware resources for constructing a new configuration scheme;

applying for a container for temporary testing to the micro service architecture according to the quota;

carrying out maximum pressure test on the container aiming at the target pressure measurement index to obtain a corresponding generated pressure measurement index;

and verifying whether the target pressure measurement index is superior to a target pressure measurement index obtained by performing pressure test on the mirror image in the generated pressure measurement indexes, and converting the input information into a corresponding configuration scheme when the target pressure measurement index is judged to be superior to the target pressure measurement index.

In order to solve the above technical problem, another technical solution adopted by the present application is:

the distributed cluster resource configuration device comprises a mirror image cloning module, an event triggering module, an event response module and a container configuration module, wherein the mirror image cloning module is used for cloning a mirror image of a running container deployed in a micro service architecture of a distributed cluster; the event triggering module is used for detecting pressure measurement indexes generated by performing pressure test on the mirror image and triggering a container configuration event when at least one target pressure measurement index overflows a corresponding preset range; the event response module is used for responding to the container configuration event, and calculating and determining the lowest price cost in a plurality of configuration schemes according to unit pricing of each hardware resource required by the operation of the container, wherein the configuration schemes comprise quotas of each hardware resource distributed for creating the container with the better target pressure measurement index; the container configuration module configures the container template of the micro-service framework according to a configuration scheme with the lowest price cost, so that the micro-service framework obtains each hardware resource of a corresponding quota according to a container derived from the container template.

In one embodiment, the event triggering module comprises: the pressure testing implementation sub-module is used for calling a testing interface to implement the maximum pressure test aiming at the target pressure testing index on the mirror image; the index acquisition submodule is used for acquiring all pressure measurement indexes generated after the maximum pressure test is carried out and determining the target pressure measurement index; and the overflow triggering submodule is used for judging whether the target pressure measurement index is lower than a corresponding preset threshold value, and if so, judging that the target pressure measurement index overflows a preset range defined by the preset threshold value, thereby triggering a container configuration event.

In one embodiment, the event response module comprises: the quota obtaining sub-module is used for obtaining the quotas of the hardware resources corresponding to the configuration schemes; the cost calculation submodule is used for adapting to each configuration scheme, calculating the single cost of each hardware resource, and linearly integrating the single cost of all the hardware resources required by the same configuration scheme into the price cost of the current configuration scheme; and the cost optimization submodule is used for searching and determining the configuration scheme with the lowest price cost from all the configuration schemes.

In a preferred embodiment, the cost calculation submodule is called to implement the following functions when running: and calling a preset pricing formula to calculate the price cost for each configuration scheme, wherein in the pricing formula, the unit cost of each hardware resource is determined by the product of the unit pricing of each hardware resource and the quota of each hardware resource, and the unit cost of each hardware resource is matched with different weight parameters respectively and then summed to obtain the price cost of each configuration scheme.

In a further expanded embodiment, the event response module comprises the following pre-operational sub-modules: the quota input submodule is used for receiving input information of quotas of all hardware resources for constructing a new configuration scheme; the container application submodule is used for applying a container for temporary testing to the micro service architecture according to the quota; the temporary pressure measurement submodule is used for carrying out maximum pressure test on the container aiming at the target pressure measurement index to obtain a corresponding generated pressure measurement index; and the quota verification sub-module is used for verifying whether the target pressure measurement index is superior to a target pressure measurement index obtained by performing pressure test on the mirror image in the generated pressure measurement indexes, and converting the input information into a corresponding configuration scheme when the target pressure measurement index is judged to be superior to the target pressure measurement index.

In order to solve the above technical problem, the present application further provides a computer device, which includes a memory and a processor, where the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the distributed cluster resource configuration method.

In order to solve the above technical problem, an embodiment of the present application further provides a storage medium storing computer readable instructions, which when executed by one or more processors, cause the one or more processors to execute the steps of the distributed cluster resource configuration method.

Compared with the prior art, the method has the following advantages:

the method utilizes the mirror image of the container clone operated in the distributed cluster to carry out pressure test, carries out evaluation and analysis of the configuration scheme of the container template according to the pressure test index, implementing a cost optimization principle in the evaluation and analysis process, determining the price cost according to the relationship between preset unit pricing and quota of hardware resources in the distributed cluster, selecting a configuration scheme with optimal price cost, thereby providing a new more reasonable container configuration scheme which can realize the minimization of the overall cost under the condition of ensuring the high utilization rate of hardware resources, the method can save economic cost while ensuring that partial hardware resource waste is not caused, fully ensures the scientific scheduling of the micro-service architecture to the hardware resources in the distributed cluster, and is particularly suitable for container allocation scenes required by providing commodity search service in the E-commerce field.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a schematic basic flow chart of a distributed cluster resource configuration method according to the present application;

FIG. 2 is a schematic flow chart illustrating pressure measurement performed by the distributed cluster resource allocation method of the present application;

fig. 3 is a schematic flowchart of an evaluation configuration scheme of the distributed cluster resource configuration method according to the present application;

fig. 4 is a schematic flowchart of a verification configuration scheme of the distributed cluster resource configuration method according to the present application;

fig. 5 is a schematic diagram of a basic structure of a distributed cluster resource configuration apparatus according to the present application;

fig. 6 is a block diagram of a basic structure of a computer device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

As will be appreciated by those skilled in the art, "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, which are devices having only wireless signal receivers without transmit capability, and devices that are receive and transmit hardware, which have receive and transmit hardware capable of two-way communication over a two-way communication link. Such a device may include: cellular or other communication devices such as personal computers, tablets, etc. having single or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, facsimile and/or data communication capabilities; a PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, internet/intranet access, a web browser, a notepad, a calendar and/or a GPS (Global Positioning System) receiver; a conventional laptop and/or palmtop computer or other device having and/or including a radio frequency receiver. As used herein, a "client," "terminal device" can be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or situated and/or configured to operate locally and/or in a distributed fashion at any other location(s) on earth and/or in space. The "client", "terminal Device" used herein may also be a communication terminal, a web terminal, a music/video playing terminal, such as a PDA, an MID (Mobile Internet Device) and/or a Mobile phone with music/video playing function, and may also be a smart tv, a set-top box, and the like.

The hardware referred to by the names "server", "client", "service node", etc. is essentially an electronic device with the performance of a personal computer, and is a hardware device having necessary components disclosed by the von neumann principle such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, an output device, etc., a computer program is stored in the memory, and the central processing unit calls a program stored in an external memory into the internal memory to run, executes instructions in the program, and interacts with the input and output devices, thereby completing a specific function.

It should be noted that the concept of "server" as referred to in this application can be extended to the case of a server cluster. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided, and in physical space, the servers may be independent from each other but can be called through an interface, or may be integrated into one physical computer or a set of computer clusters. Those skilled in the art will appreciate this variation and should not be so limited as to restrict the implementation of the network deployment of the present application.

According to the technical scheme, the cloud server can be deployed, data communication connection can be achieved between the cloud server and servers related to business to coordinate online services, a logically-related distributed cluster can be formed between the cloud server and other related servers, the micro-service framework is deployed by taking the distributed cluster as a unit, the online services are deployed through container examples in the micro-service framework, and specific access services are provided for related terminal devices such as smart phones, personal computers and third-party servers through the online services. The smart phone and the personal computer can both access the internet through a known network access mode, and establish a data communication link with the server of the application so as to access and use the service provided by the server.

For the server, a corresponding program interface is opened by a service engine providing an online service for remote invocation by various terminal devices, and the related technical solution applicable to be deployed in the server in the present application can be implemented in the server in this way.

The computer program, i.e., the application program, referred to in the present application, is developed in a computer program language, and is installed in a computer device, including a server, a terminal device, and the like, to implement the relevant functions defined in the present application, regardless of the development language used therein unless otherwise specified.

The person skilled in the art will know this: although the various methods of the present application are described based on the same concept so as to be common to each other, they may be independently performed unless otherwise specified. In the same way, for each embodiment disclosed in the present application, it is proposed based on the same inventive concept, and therefore, concepts of the same expression and concepts of which expressions are different but are appropriately changed only for convenience should be equally understood.

The embodiments to be disclosed herein can be flexibly constructed by cross-linking related technical features of the embodiments unless the mutual exclusion relationship between the related technical features is stated in the clear text, as long as the combination does not depart from the inventive spirit of the present application and can meet the needs of the prior art or solve the deficiencies of the prior art. Those skilled in the art will appreciate variations therefrom.

Referring to fig. 1, a basic flow diagram of a distributed cluster resource configuration method in an exemplary embodiment of the distributed cluster resource configuration method according to the present application is shown, where the distributed cluster resource configuration method is programmed as an application program, deployed in a server, and returns a corresponding execution result in response to an interface call, and includes the following steps:

step S11, cloning the mirror image of the running container deployed in the micro service architecture of the distributed cluster:

the distributed cluster is used for deploying the micro-service architecture. The micro-service architecture is used for realizing each specific online service of the Internet service and opening the management function of each online service for the deployment party through the open background management interface. The background of each specific online service of the micro service architecture, such as the e-commerce product search service of the e-commerce system, is typically deployed to run in an instance derived from a container planned for the micro service architecture. The management user can plan in advance for the container of the micro service framework, configure information such as quota and unit pricing of each hardware resource required to be occupied by the micro service framework, and form a container template. The hardware resources refer to a CPU, a memory, and a hard disk in a computer device of the server in the distributed cluster. The hardware resources distributed on different servers in the cluster are logically and uniformly distributed by the micro service architecture in a centralized scheduling manner, and then, each container is allowed to obtain the corresponding quota of each hardware resource. Therefore, for the micro service architecture, the various hardware resources invoked by the micro service architecture are irrelevant to the physical position of the micro service architecture in the distributed cluster.

When a container is put into use and is in a running state, the online service running on the container is generally not prone to be interrupted so as not to influence the normal open business service of the online user. In order to examine the operation state of one or more containers, the container cloning technology supported by a distributed cluster is required to clone and copy the one or more containers to create a mirror image of the one or more containers, so that the normal operation of the original container is not influenced by the operation on the mirror image, and the aim of related operation can be fulfilled. The technology for cloning the running container to obtain its corresponding mirror image is known to those skilled in the art, and is also one of the functions supported by the micro service architecture, so it will not be described in detail here.

In order to achieve the purpose of performing a pressure force measurement test on a container subsequently in the present application, one or more containers may be cloned as needed to obtain corresponding mirror images.

Step S12, detecting pressure measurement indicators generated by performing a pressure test on the mirror image, and triggering a container configuration event when at least one target pressure measurement indicator overflows a corresponding preset range:

firstly, starting a pressure test on one or more mirror images obtained in the previous step, so as to obtain a corresponding performance pressure test index generated by the test after the pressure test, which is called a pressure test index for short. These pressure measurement indicators generally include, but are not limited to:

throughput: the number of requests or tasks per second that the online service running in the container instance can handle;

response time: the online service running in the container instance is time consuming to process a request or a task.

Error rate: the online service running in the container instance processes the same batch of requests with a proportion of the requests having the result of errors.

QPS: query per second rate, which is a measure of how much traffic a particular online service is handling in a given time.

The number of concurrent users: refers to the number of users within a container instance that can be simultaneously carried by an online service that normally uses system functionality.

These pressure measurement indicators are concepts well known to those skilled in the art, and respectively reflect the operation performance of an online service from different sides, and for the micro service architecture of the present application, the operation performance of a container instance, in which the pressure measurement indicator with QPS as the core comprehensively reflects the maximum throughput capability of a container in the process of providing a service response request by using its limited hardware resources. Therefore, in the present application, QPS is mainly determined as a target pressure measurement indicator so as to detect the actual operating efficiency of one vessel according to QPS.

Of course, by taking the typical embodiment of using QPS as the target pressure measurement indicator as reference, those skilled in the art can also take the other one or more pressure measurement indicators such as throughput, response time, number of concurrent users, etc. as detection dimensions to make a decision on the actual operation efficiency of the corresponding container, thereby achieving flexible performance without being limited to the example of the present application without departing from the inventive spirit of the present application.

In the prior art, a plurality of pressure measurement tools exist to perform pressure measurement on a container instance to obtain various corresponding pressure measurement indexes, for example, wrk, ab, meter and other tools to obtain a QPS-related pressure measurement tool. Wherein: wrk is a modern HTTP performance testing tool that can generate significant stress even when running on a single core CPU. The biggest advantage is that the system supports multithreading, so that the capacity of a multi-core CPU is more easily exerted, and the limit capacity of the system is more easily tested; ab is the abbreviation of Apache Benchmark, which is the pressure test tool of the Apache; jmeter, known collectively as Apache Jmeter, is a Java-based stress testing tool developed by the Apache organization.

Another way is to consider the specific case of distributed cluster, and often use Docker as a container platform to perform deployment of a micro-service architecture, and Docker is equipped with a corresponding Stress testing tool Stress, through which image creation of a Docker-based container and Stress testing can be more efficiently performed, so as to obtain relevant Stress testing parameters.

And carrying out pressure test on the mirror image by using a corresponding tool, and inspecting the target pressure measurement index after the target pressure measurement index is finally obtained, wherein the inspection is mainly to inspect whether the target pressure measurement index overflows a preset range. The preset range may be a threshold, for example, a desired threshold is set for the index of the query rate per second of the container, when the query rate per second obtained by pressure measurement of one container is higher than the desired threshold, it may be determined that the corresponding container meets the requirement, and when the query rate per second obtained by pressure measurement of one container is lower than the desired threshold, it may be determined that the specific target test index of the query rate per second of the container exceeds the corresponding preset range. The term "preset range" is used herein in a broad sense to mean an abstract concept, not a mathematical interval concept, but includes a mathematical interval meaning, for example, when the query rate per second occurs between the expected threshold and infinity, the container is considered to satisfy the relevant requirement, in which case, the preset range actually implies the interval between the expected threshold and infinity. Similarly, the skilled person can flexibly set the preset range according to the mathematical and physical meanings of each specific pressure measurement index, so as to examine the target pressure measurement index, and determine whether the corresponding mirrored target pressure measurement index overflows the preset range.

When the target pressure measurement index is judged to be beyond the preset range according to the target pressure measurement index, the container corresponding to the current mirror image is regarded as not meeting related business requirements, and the allocated hardware resources are insufficient, so that a configuration scheme related to various hardware resources needs to be provided for a new container instance again, and a container configuration event is triggered, so that the container configuration event is further responded to for carrying out re-customization on the container configuration scheme.

Step S13, responding to the container configuration event, and according to the unit pricing of each hardware resource required by the operation of the container, calculating and determining the lowest price cost in a plurality of configuration schemes, where the configuration schemes include the quota of each hardware resource allocated to create the container with the better target pressure measurement indicator:

triggered by the container configuration event, an evaluation analysis of a plurality of configuration scenarios is initiated.

In order to realize the evaluation analysis of a plurality of configuration schemes, since various hardware resources in the system are determined and the cost is basically fixed, the cost of the various hardware resources can be quantified in advance, and the price of the unit cost of each hardware resource, namely the unit pricing of each hardware resource is adapted. The unit pricing can be flexibly implemented, for example, the unit pricing can be determined according to the actual cost of hardware, or according to market statistics, and the like, in summary, the unit pricing of various given hardware resources is determined firstly and then is unit, for example, the unit pricing of a CPU can be understood as a price determined by taking a core number as a pricing unit, the unit pricing of a memory can be understood as a price determined by taking a G as a unit, and similarly, the unit pricing of a hard disk can be understood as a price determined by taking a G as a unit. The dimensions occupied by the units themselves can be flexibly determined by those skilled in the art and need not be limited to the examples herein.

The unit pricing needed by each hardware resource can be preset and stored in a database so as to be called in the step, the user input can be immediately obtained in the step for determination, and the unit pricing can be obtained from a price inquiry interface provided by a third party in the step for determination, so that the method can be flexibly implemented by a person skilled in the art.

The configuration scheme at least includes data of the quota required by each hardware resource, that is, a mapping relationship between a specific hardware resource and a specific quota thereof is established, so that the corresponding quota of each specific hardware resource can be known through the configuration scheme. The configuration scheme is essentially intended to describe the size of the various specific hardware resources that a container instance can obtain, and thus, the form thereof can be roughly expressed as: the term "CPU: 2 issues, memory: 2G, and hard disk: 32G", may be simply expressed as "2, 2, 32", and so on, as long as the corresponding quotas of the hardware resources can be read out according to the explicit mapping relationship in this step.

The configuration scheme should preferably ensure that various hardware resources obtained according to their specified quotas can enable the container instance created according to the configuration scheme to obtain a better target pressure measurement indicator after being put into use, for example, when the target pressure measurement indicator is a QPS, a QPS of a new container instance derived according to the configuration scheme should be higher than a QPS of an old container instance. The superiority of the better target pressure measurement indicator referred to herein is not necessarily a new value greater than an old value, and sometimes a new value lower than an old value, depending on the specific target pressure measurement indicator content, for example, if the response time is the target pressure measurement indicator, then when the new response time is less than the old response time, in which case the new value is the better target pressure measurement indicator, and those skilled in the art can use the a priori knowledge to realize flexible understanding. To achieve this goal, one way can be determined by those skilled in the art flexibly with empirical knowledge, for example, theoretically increasing the core number of the CPU helps to increase QPS, i.e., the core number of the CPU can be increased to give a new configuration. Alternatively, this may be accomplished by performing a pre-test verification of the configuration scheme, which is described further below and is not shown here.

The configuration scheme may be correspondingly pre-stored in the database, or recommended by a suggestion mechanism or a trial computation mechanism inherent to the micro service architecture, or may be provided by a background management user through one-by-one input in this step, which can be implemented by a person skilled in the art, and how to implement the specific implementation manner of the configuration scheme does not affect the embodiment of the inventive spirit of the present application.

When there are multiple configuration schemes, each configuration scheme includes quotas corresponding to various hardware resources, and unit pricing of various hardware resources is also known. Under the condition that the cost price of each configuration scheme is known, the minimum value of the configuration schemes can be determined, and the configuration scheme with the lowest cost price is determined. Since the configuration scheme firstly ensures that the container instance can guide to obtain a better target pressure measurement index, and the cost price is the lowest, in this case, the optimal configuration scheme required for creating the container is actually determined, and the configuration scheme essentially plays a role of a container template according to which the container is created.

Step S14, configuring the container template of the micro service framework according to the configuration scheme with the lowest price cost, so that the micro service framework obtains each hardware resource of the corresponding quota according to the container derived from the container template:

in order to enable the micro service architecture to dispatch a new container instance according to the configuration scheme with the lowest cost, and even to reconfigure the existing container, the configuration scheme with the lowest price and the lowest cost needs to be stored as a container template in the micro service architecture, so as to implement the configuration of the container template of the micro service architecture.

The micro-service framework may subsequently derive a corresponding container instance from specific quotas including CPU, memory, hard disk, etc. for use in response to the container instance creation instruction according to each hardware resource specified in the new container template.

The configuration scheme with the lowest price cost is essentially the optimal configuration scheme for scheduling each hardware resource occupied by the container in the distributed cluster, not only the cost is considered to be optimal, but also the utilization rate during operation is considered to meet the requirement of related target pressure measurement indexes, so that the high-efficiency scheduling utilization of the distributed cluster by the micro-service architecture is realized, the overall utilization rate of the cluster is maximized, and the overall cost is minimized.

Referring to fig. 2, in an embodiment extracted for improving the testing efficiency, the step S12 includes the following steps:

step S121, calling a test interface to carry out maximum pressure test aiming at the target pressure test index on the mirror image:

the micro-service architecture can package a third-party pressure test tool or a business logic for implementing pressure test realized by the micro-service architecture into a test interface convenient to call, and when the pressure test needs to be started, the maximum pressure test aiming at the target pressure test index can be started by calling the test interface.

Step S122, obtaining all pressure measurement indexes generated after the maximum pressure test is carried out, and determining the target pressure measurement indexes from the pressure measurement indexes:

depending on the function packaged by the test interface, sometimes, one pressure test may generate a plurality of associated pressure measurement indicators, and as a result of the interface call, a plurality of pressure measurement indicators may be obtained.

Step S123, determining whether the target pressure measurement indicator is lower than a corresponding preset threshold, and if yes, determining that the target pressure measurement indicator overflows a preset range defined by the preset threshold, thereby triggering a container configuration event:

the preset threshold is a subjective expected threshold, which is used for participating in defining a preset range, for example, a desired threshold is set for an index of a query rate per second of a container, when the query rate per second obtained by pressure measurement of a container is higher than the desired threshold, it can be determined that the corresponding container meets requirements, and when the query rate per second is lower than the desired threshold, it can be determined that a specific target test index of the query rate per second of the container overflows its corresponding preset range. The term "preset range" is used herein in a broad sense to mean an abstract concept, not a mathematical interval concept, but includes a mathematical interval meaning, for example, when the query rate per second occurs between the expected threshold and infinity, the container is considered to satisfy the relevant requirement, in which case, the preset range actually implies the interval between the expected threshold and infinity. Similarly, the skilled person can flexibly set the preset threshold value according to the mathematical and physical meanings of each specific pressure measurement index to define different preset ranges, so as to comprehensively examine the target pressure measurement index, and determine whether the corresponding mirrored target pressure measurement index overflows the preset range.

In this embodiment, after determining that the target pressure measurement indicator is lower than the preset threshold according to the target pressure measurement indicator, it is regarded that the container corresponding to the current mirror image does not satisfy the related service requirement, and it is insufficient to previously obtain the allocated hardware resource, and therefore, it is necessary to re-provide the configuration scheme for various hardware resources for a new container instance, thereby triggering a container configuration event, so as to further perform re-customization of the container configuration scheme in response to the container configuration event.

The embodiment is mainly applicable to the case that a QPS or a similar pressure measurement indicator is used as a target pressure measurement indicator, and in this case, whether a container configuration event needs to be triggered can be efficiently determined by setting a preset threshold, so as to determine whether to restart the decision logic of the configuration scheme.

The method and the device are particularly suitable for an internet service system which has a high concurrency rate and needs to process a large number of requests, and particularly suitable for a search system in the E-commerce field.

Referring to fig. 3, in an example provided for guiding the implementation of the configuration solution obtaining process, in step S13, the step of calculating and determining the lowest price cost of the plurality of configuration solutions according to the unit pricing of each hardware resource required by the container to operate includes the following steps:

step S135, obtaining quotas of the hardware resources corresponding to the configuration schemes:

as described above, the configuration scheme may be pre-stored in the database, recommended by a suggestion mechanism or a trial computation mechanism inherent in the microservice architecture, or provided by a background management user through one-by-one input in this step, and those skilled in the art can implement the configuration scheme, and how to implement the configuration scheme does not affect the embodiment of the inventive spirit of the present application. The present embodiment recommends receiving user inputs one by one to provide each configuration scheme.

Step S136, adapting to each configuration scheme, calculating the single cost of each hardware resource, and linearly integrating the single cost of all hardware resources required by the same configuration scheme into the price cost of the current configuration scheme:

specifically, pricing is performed for each configuration scheme with respect to price costs to determine price costs corresponding to each configuration scheme.

In one embodiment, the single cost of each hardware resource may be obtained by directly multiplying the unit pricing of the hardware resource by the quota of each hardware resource in each configuration scheme, and then the single cost of all the hardware resources is summed to obtain the price cost corresponding to the configuration scheme. The method is simple and easy to use, and can improve decision-making efficiency.

In another embodiment, a preset pricing formula is called to calculate the price cost for each configuration scheme, in the pricing formula, the unit cost of each hardware resource is determined by the product of the unit pricing of each hardware resource and the quota of each hardware resource, and the unit costs of the hardware resources are respectively matched with different weight parameters and then summed to obtain the price cost of each configuration scheme. Different weight parameters are allocated to the single cost of each hardware resource, so that the linear fusion between the single cost of each hardware resource can be realized more flexibly, and the action of each hardware resource in the whole configuration scheme can be determined flexibly. Generally, each hardware resource required by the container operation includes a CPU, a memory, and a hard disk of the computer device, and the relationship between the respective weight parameters is arranged from large to small according to the characterized weight: CPU, internal memory, hard disk. The arrangement mainly considers the practical situation of resource unit cost of CPU, memory and hard disk, so that the evaluation and analysis result of the application has more practical significance.

Those skilled in the art can flexibly adjust pricing schemes related to price costs based on cost-optimization principles in accordance with the principles disclosed herein.

Step S137, finding and determining the configuration scheme with the lowest price cost from all configuration schemes:

finally, aiming at the price cost corresponding to each configuration scheme, a minimum value solving function is operated to find out the configuration scheme with the lowest price cost, namely the optimal configuration scheme required by the micro service architecture for subsequently creating the container.

The embodiment provides a plurality of effective examples of the evaluation analysis scheme of the hardware resources required by the evaluation analysis container, has different advantages, is beneficial to programming realization, and can improve the decision efficiency of a computer.

Referring to fig. 4, in a further extended embodiment for acquiring configuration schemes, before the step S135, that is, before the step of acquiring quotas of the hardware resources corresponding to the configuration schemes, the method includes the following steps:

step S131, receiving input information of quotas of each hardware resource for constructing a new configuration scheme:

in this embodiment, relevant information of each configuration scheme entered by a management user may be received in a background management page of the micro service architecture, and for each configuration scheme, an entry worker is required to input data corresponding to a specific quota of each hardware resource, thereby implementing acquisition of a quota corresponding to each hardware resource in the configuration scheme. The configuration schemes can be allowed to be input once, and related quotas can be acquired independently according to each configuration scheme.

Step S132, applying a container for temporary testing to the micro service framework according to the quota:

for a configuration scheme, since the quota related to each invoked hardware resource is already obtained, because the container creation request can be extracted from the micro service architecture, the micro service architecture is requested to derive a new container instance corresponding to the configuration scheme according to the configuration of each hardware resource provided herein, the container instance is temporary, and is used for testing whether the target pressure measurement index of the container obtained according to the configuration scheme is better than that of the existing container, and therefore, the container can be cleaned and recycled subsequently.

Step S133, performing a maximum pressure test on the container according to the target pressure measurement index, to obtain a correspondingly generated pressure measurement index:

similarly, by means of a third-party tool or a test interface packaged by the micro-service architecture, a maximum pressure test of the container obtained by the temporary application is initiated, and a generated pressure test index is obtained.

Step S134, verifying whether the target pressure measurement index is superior to a target pressure measurement index obtained by performing pressure test on the mirror image in the generated pressure measurement indexes, and if so, converting the input information into a corresponding configuration scheme:

among the generated pressure measurement indexes, only the target pressure measurement index is required to be used. The target pressure measurement index is compared with a target pressure measurement index obtained by performing pressure test on the mirror image in the application, whether the target pressure measurement index is superior to the target pressure measurement index is judged, when the target pressure measurement index is superior to the target pressure measurement index, the current situation shows that a new configuration scheme can obtain a container which can better meet the performance requirement of the online service, and information related to quota input by a user can be converted into corresponding new configuration scheme data; otherwise, the new configuration scheme has no practicability and can be rejected. It should be noted that the comparison of the relative superiority or inferiority of the two target test indicators is not simply dependent on the relative superiority or inferiority between the two data, as described above, depending on the specific content and nature of the target test indicators, as will be appreciated by those skilled in the art.

In this embodiment, the step of evaluating and analyzing the price cost is performed in advance to implement pre-verification of the configuration scheme, so that it is ensured that the container obtained according to the configuration scheme can meet the operation performance requirements related to the online service, and on this basis, the lowest cost evaluation principle is applied, and finally it is ensured that the hardware resources in the distributed cluster are used according to the principle of higher cost performance, so as to implement reasonable configuration of the container instance.

Referring to fig. 5, an embodiment of the present application further provides a distributed cluster resource configuration apparatus, which includes a mirror cloning module 11, an event triggering module 12, an event response module 13, and a container configuration module 14, where the mirror cloning module 11 is configured to clone a mirror image of a running container deployed in a micro service architecture of a distributed cluster; the event triggering module 12 is configured to detect a pressure measurement indicator generated by performing a pressure test on the mirror image, and trigger a container configuration event when at least one target pressure measurement indicator overflows a corresponding preset range; the event response module 13 is configured to respond to the container configuration event, and calculate and determine a lowest price cost in a plurality of configuration schemes according to unit pricing of each hardware resource required for the operation of the container, where the configuration schemes include quotas of each hardware resource allocated for creating a container with a better target pressure measurement indicator; the container configuration module 14 configures the container template of the micro service architecture according to the configuration scheme with the lowest price cost, so that the micro service architecture obtains each hardware resource of the corresponding quota according to the container derived from the container template.

In one embodiment, the event triggering module 12 comprises: the pressure testing implementation sub-module is used for calling a testing interface to implement the maximum pressure test aiming at the target pressure testing index on the mirror image; the index acquisition submodule is used for acquiring all pressure measurement indexes generated after the maximum pressure test is carried out and determining the target pressure measurement index; and the overflow triggering submodule is used for judging whether the target pressure measurement index is lower than a corresponding preset threshold value, and if so, judging that the target pressure measurement index overflows a preset range defined by the preset threshold value, thereby triggering a container configuration event.

In one embodiment, the event response module 13 comprises: the quota obtaining sub-module is used for obtaining the quotas of the hardware resources corresponding to the configuration schemes; the cost calculation submodule is used for adapting to each configuration scheme, calculating the single cost of each hardware resource, and linearly integrating the single cost of all the hardware resources required by the same configuration scheme into the price cost of the current configuration scheme; and the cost optimization submodule is used for searching and determining the configuration scheme with the lowest price cost from all the configuration schemes.

In a further expanded embodiment, the event response module 13 includes the following pre-operational sub-modules: the quota input submodule is used for receiving input information of quotas of all hardware resources for constructing a new configuration scheme; the container application submodule is used for applying a container for temporary testing to the micro service architecture according to the quota; the temporary pressure measurement submodule is used for carrying out maximum pressure test on the container aiming at the target pressure measurement index to obtain a corresponding generated pressure measurement index; and the quota verification sub-module is used for verifying whether the target pressure measurement index is superior to a target pressure measurement index obtained by performing pressure test on the mirror image in the generated pressure measurement indexes, and converting the input information into a corresponding configuration scheme when the target pressure measurement index is judged to be superior to the target pressure measurement index.

An embodiment of the present application further provides a computer device, as shown in fig. 6, an internal structure diagram of the computer device is provided. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The non-volatile storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store control information sequences, and the computer readable instructions, when executed by the processor, can cause the processor to implement a distributed cluster resource configuration method. The processor of the computer device is used for providing calculation and control capability and supporting the operation of the whole computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, may cause the processor to perform a distributed cluster resource configuration method. The network interface of the computer device is used for connecting and communicating with the terminal. Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In this embodiment, the processor is configured to execute specific functions of each module/sub-module in fig. 5, and the memory stores program codes and various data required for executing the modules. The network interface is used for data transmission to and from a user terminal or a server. The memory in this embodiment stores program codes and data required for executing all the sub-modules in the distributed cluster resource configuration device, and the server can call the program codes and data of the server to execute the functions of all the sub-modules.

The present application also provides a storage medium storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the steps of the distributed cluster resource configuration method of any of the above embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).

In summary, the present application can provide a configuration technology for scheduling hardware resources in a distributed cluster for a micro service architecture, so as to ensure that the utilization rate of the micro service architecture on the cluster hardware resources is maximized, and simultaneously ensure that the cost is minimized.

Those of skill in the art will appreciate that the various operations, methods, steps in the processes, acts, or solutions discussed in this application can be interchanged, modified, combined, or eliminated. Further, other steps, measures, or schemes in various operations, methods, or flows that have been discussed in this application can be alternated, altered, rearranged, broken down, combined, or deleted. Further, steps, measures, schemes in the prior art having various operations, methods, procedures disclosed in the present application may also be alternated, modified, rearranged, decomposed, combined, or deleted.

The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims

1. A distributed cluster resource configuration method is characterized by comprising the following steps:

2. The method according to claim 1, wherein detecting pressure measurement indicators generated by performing a pressure test on the mirror image, and triggering a container configuration event when at least one target pressure measurement indicator overflows a corresponding preset range comprises:

3. The method according to claim 2, wherein the target pressure measurement indicator is any one of throughput, query rate per second, and number of concurrent users.

4. The method of claim 1, wherein the step of computationally determining the lowest cost of the plurality of configuration solutions according to unit pricing of each hardware resource required for operation of the container comprises the steps of:

5. The method according to claim 4, wherein the method adapts to each configuration scheme, calculates the cost of each hardware resource, and linearly merges the cost of each hardware resource required by the same configuration scheme into the cost of the current configuration scheme, and the process is as follows:

6. The method according to claim 5, wherein each hardware resource required for the container operation includes a CPU, a memory, and a hard disk of a computer device, and the relationship between the respective weight parameters is arranged from large to small according to the characterized weight: CPU, internal memory, hard disk.

7. The method according to claim 4, wherein before the step of obtaining the quotas of the hardware resources corresponding to the configuration schemes, the method comprises the following steps:

8. A distributed cluster resource configuration apparatus, comprising:

the mirror image cloning module is used for cloning the mirror image of the running container deployed in the micro service architecture of the distributed cluster;

the event triggering module is used for detecting pressure measurement indexes generated by performing pressure test on the mirror image and triggering a container configuration event when at least one target pressure measurement index overflows a corresponding preset range;

an event response module, configured to respond to the container configuration event, and calculate and determine a lowest price cost of multiple configuration schemes according to unit pricing of each hardware resource required for operation of the container, where the configuration schemes include quotas of each hardware resource allocated for creating a container with a better target pressure measurement indicator;

and the container configuration module is used for configuring the container template of the micro service architecture according to a configuration scheme with the lowest price cost so as to enable the micro service architecture to obtain each hardware resource of the corresponding quota according to the container derived from the container template.

9. A computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions which, when executed by the processor, cause the processor to perform the steps of the distributed cluster resource configuration method of any of claims 1 to 7.

10. A storage medium having computer-readable instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of distributed cluster resource configuration of any of claims 1 to 7.