CN113568741A - Service expansion and contraction method, device, equipment and storage medium of distributed system - Google Patents

Service expansion and contraction method, device, equipment and storage medium of distributed system Download PDF

Info

Publication number
CN113568741A
CN113568741A CN202110816331.2A CN202110816331A CN113568741A CN 113568741 A CN113568741 A CN 113568741A CN 202110816331 A CN202110816331 A CN 202110816331A CN 113568741 A CN113568741 A CN 113568741A
Authority
CN
China
Prior art keywords
hour
service
micro
distributed system
year
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110816331.2A
Other languages
Chinese (zh)
Other versions
CN113568741B (en
Inventor
王晓炜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
MIGU Culture Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
MIGU Culture Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, MIGU Culture Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202110816331.2A priority Critical patent/CN113568741B/en
Priority claimed from CN202110816331.2A external-priority patent/CN113568741B/en
Publication of CN113568741A publication Critical patent/CN113568741A/en
Application granted granted Critical
Publication of CN113568741B publication Critical patent/CN113568741B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request

Abstract

The invention discloses a service expansion and contraction method, a device, equipment and a computer readable storage medium of a distributed system, wherein the method comprises the following steps: determining a contrast coefficient corresponding to the hour to be predicted based on the historical change trend and the current change trend; determining the average value of micro service volume corresponding to the hour to be predicted; inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity; determining the micro service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro service quantity average value, the hour request quantity and the started micro service quantity; and performing micro-service capacity expansion operation on the distributed system based on the number of the micro-services. According to the invention, the micro-service capacity expansion and reduction is carried out on the distributed system by combining the micro-service quantity obtained by historical data prediction, so that the automatic capacity expansion and reduction of the micro-service in the distributed system is realized, and the efficiency and the accuracy of the capacity expansion and reduction of the distributed system of the website are improved.

Description

Service expansion and contraction method, device, equipment and storage medium of distributed system
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for expanding and contracting service volumes of a distributed system.
Background
With the rapid development of internet technology, internet users have greatly increased, high requirements are provided for website performance, websites are required to support large-flow access and respond quickly, and good experience of users is improved, especially in various festivals (such as double 11), when users access and request websites with high density and high quantity, the server system of the websites should meet the requirements of non-downtime, quick response and the like.
At present, the capacity expansion and contraction mode of a distributed system of a website is to perform manual watching at various possible peak access times, monitor a server in cooperation with operation and maintenance, and perform manual capacity expansion and contraction operation when the distributed system needs to expand and contract the capacity. However, the manual watch needs to hire a large number of operation and maintenance personnel, the capacity of the system is expanded and contracted in an artificial mode of monitoring relevant data indexes and the like, the experience requirement on the operation and maintenance personnel is high when the capacity of the system is expanded and contracted, and when the operation and maintenance personnel are tired manually, the capacity expansion and contraction operation easily fails, so that the efficiency and the accuracy of the capacity expansion and contraction of the distributed system of the website are low.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a service capacity expansion and reduction method, a service capacity expansion and reduction device, service capacity expansion and reduction equipment and a computer readable storage medium for a distributed system, and aims to solve the technical problems of low efficiency and low accuracy rate of capacity expansion and reduction of the distributed system of the existing website.
In order to achieve the above object, the present invention provides a service expansion and contraction method for a distributed system, where the service expansion and contraction method for the distributed system includes the following steps:
determining a comparison coefficient corresponding to the hours to be predicted based on a historical change trend corresponding to the number of requests of each hour in each year and a current change trend corresponding to the number of requests of each hour in the current year in the distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service;
determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system;
inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted;
acquiring an hour request quantity of micro-services processing requests per hour and the current started micro-service quantity in a distributed system, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, a comparison coefficient, a micro-service quantity average value, the hour request quantity and the started micro-service quantity;
and executing micro-service capacity expansion operation on the distributed system based on the number of the micro-services corresponding to the hour to be predicted.
Further, the step of determining the contrast coefficient corresponding to the hour to be predicted based on the historical change trend corresponding to the number of requests per hour per year and the current change trend corresponding to the number of requests per hour per year in the distributed system includes:
determining a historical change trend corresponding to the request quantity of each hour every year based on the request quantity of each hour in a preset year before the current year;
determining a current change trend corresponding to the request quantity of each hour in the current year based on the request quantity of each hour in the current year before the current moment;
and determining a contrast coefficient corresponding to the hour to be predicted based on the historical change trend and the current change trend.
Further, the step of determining the historical trend of the number of requests per hour per year based on the number of requests per hour in a preset year before the current year includes:
determining a first request quantity deviation value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the request quantity of each hour in the preset year before the current year;
determining a first requested quantity mean value of each hour per year based on the first requested quantity offset value;
and determining the historical change trend corresponding to the request quantity of each hour every year based on the coordinate curve corresponding to the first request quantity average value.
Further, the step of determining a current trend of change corresponding to the number of requests for each hour in the current year based on the number of requests for each hour in the current year before the current time includes:
determining a second request quantity deviation value of each hour in the current year before the current time through a kernel function mean shift algorithm based on the request quantity of each hour in the current year before the current time;
determining a second request quantity average value of each hour in the current year based on the second request quantity deviation value;
and determining the current change trend corresponding to the request quantity of each hour in the current year based on the coordinate curve corresponding to the second request quantity mean value.
Further, the step of determining a contrast ratio corresponding to the hour to be predicted based on the historical trend and the current trend includes:
acquiring a first change trend corresponding to a target moment every day from historical change trends, wherein the target moment is the moment of the hour to be predicted on the day;
acquiring a second variation trend corresponding to each day of target time at the current variation trend;
and determining a contrast coefficient corresponding to the hour to be predicted based on the mean value of the first change trend and the mean value of the second change trend.
Further, the step of determining the average value of the micro service volumes corresponding to the hours to be predicted by using a kernel function mean shift algorithm based on the micro service volumes of the preset hours in the distributed system before the current year includes:
determining a micro-service quantity deviation value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the micro-service quantity of each hour in the preset year before the current year;
determining the average value of the micro service volume of each hour every year based on the offset value of the micro service volume of each hour in a preset year before the current year;
and determining the average value of the micro service volume corresponding to the hour to be predicted in the average value of the micro service volume of each hour every year.
Further, before the step of inputting the number of requests for each hour in the current year in the distributed system into the pre-trained request quantity prediction model for model training to obtain the predicted request quantity corresponding to the hour to be predicted, the method further includes:
acquiring first historical request data in a preset year before the current year, services with the same service scenes as the first historical request data and second historical request data in the preset year before the current year;
and performing model training on the request quantity prediction model based on the first historical request data and the second historical request data to obtain a pre-trained request quantity prediction model.
In addition, to achieve the above object, the present invention further provides a service scaling device for a distributed system, where the service scaling device for the distributed system includes:
the first determining module is used for determining a comparison coefficient corresponding to an hour to be predicted based on a historical change trend corresponding to the number of requests of each hour every year and a current change trend corresponding to the number of requests of each hour every year in a distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service;
the second determination module is used for determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system;
the training module is used for inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted;
the acquisition module is used for acquiring the hourly request quantity of the requests processed by the micro-services per hour in the distributed system and the current started micro-service quantity, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro-service quantity mean value, the hourly request quantity and the started micro-service quantity;
and the capacity expansion and reduction module is used for executing micro-service capacity expansion and reduction operation on the distributed system based on the number of the micro-services corresponding to the hour to be predicted.
In addition, to achieve the above object, the present invention further provides a service scaling device for a distributed system, where the service scaling device for the distributed system includes: the system comprises a memory, a processor and a service scaling program of the distributed system, wherein the service scaling program of the distributed system is stored on the memory and can run on the processor, and when being executed by the processor, the service scaling program of the distributed system realizes the steps of the service scaling method of the distributed system.
In addition, to achieve the above object, the present invention further provides a computer readable storage medium, where a service scaling program of a distributed system is stored, and when executed by a processor, the service scaling program of the distributed system implements the steps of the service scaling method of the distributed system.
The method comprises the steps of determining a comparison coefficient corresponding to the hour to be predicted based on a historical change trend corresponding to the number of requests of each hour in each year and a current change trend corresponding to the number of requests of each hour in the current year in a distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service; then, determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system; inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted; then acquiring an hour request quantity of micro-services processing requests per hour and the current started micro-service quantity in the distributed system, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro-service quantity average value, the hour request quantity and the started micro-service quantity; and finally, performing micro-service capacity expansion and reduction operation on the distributed system based on the micro-service quantity corresponding to the hour to be predicted, and performing micro-service capacity expansion and reduction on the distributed system by combining the micro-service quantity obtained by historical data prediction, so that the automatic capacity expansion and reduction of the micro-service in the distributed system are realized, and the efficiency and the accuracy of capacity expansion and reduction of the distributed system of the website are improved. Meanwhile, the manual participation is reduced in the micro-service scaling process, the micro-service scaling error caused by human errors can be avoided, the manual pressure can be reduced, and the cost can be saved.
Drawings
Fig. 1 is a schematic structural diagram of a service scaling device of a distributed system in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a service expansion and contraction method for a distributed system according to the present invention;
fig. 3 is a functional block diagram of an embodiment of a service scaling device of a distributed system according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a service scaling device of a distributed system in a hardware operating environment according to an embodiment of the present invention.
The service expansion and contraction device of the distributed system in the embodiment of the invention can be a PC, and can also be a mobile terminal device with a display function, such as a smart phone, a tablet computer, an electronic book reader, an MP3(Moving Picture Experts Group Audio Layer III, dynamic video Experts compression standard Audio Layer 3) player, an MP4(Moving Picture Experts Group Audio Layer IV, dynamic video Experts compression standard Audio Layer 4) player, a portable computer, and the like.
As shown in fig. 1, the service scaling device of the distributed system may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Optionally, the service scaling device of the distributed system may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors. Of course, the service expansion and contraction device of the distributed system may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and so on, which are not described herein again.
Those skilled in the art will appreciate that the terminal architecture shown in fig. 1 does not constitute a limitation of the service scale-up facilities of the distributed system, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a service scaling program of a distributed system.
In the service scaling device of the distributed system shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to invoke a service scaling program of the distributed system stored in the memory 1005.
In this embodiment, the service scaling device of the distributed system includes: the system comprises a memory 1005, a processor 1001 and a service scaling program of the distributed system, wherein the service scaling program of the distributed system is stored in the memory 1005 and can run on the processor 1001, and when the processor 1001 calls the service scaling program of the distributed system stored in the memory 1005, the steps of the service scaling method of the distributed system in each of the following embodiments are executed.
Referring to fig. 2, fig. 2 is a schematic flow diagram illustrating a first embodiment of a service expansion and reduction method for a distributed system according to the present invention.
In this embodiment, the service expansion and contraction method of the distributed system includes the following steps:
step S101, determining a comparison coefficient corresponding to an hour to be predicted based on a historical change trend corresponding to the number of requests of each hour every year and a current change trend corresponding to the number of requests of each hour every year in a distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service;
the request of this embodiment is a request for accessing a website of the distributed system, and a historical change trend corresponding to the number of requests in each hour per year may be determined according to the number of historical requests in the distributed system at the beginning of each year, where the number of historical requests may include the number of historical requests in N years before the current year, the number of historical requests is divided into the number of historical requests in each hour per year, and the historical change trend corresponding to the number of requests in each hour per year is determined according to the number of historical requests in each hour. Meanwhile, the current change trend is determined according to the number of requests of each hour in the current year.
And then, calculating a contrast coefficient corresponding to the hour to be predicted according to the historical change trend and the current change trend, specifically, acquiring the historical change trend at the same time as the hour to be predicted from the historical change trend, acquiring the current change trend at the same time as the hour to be predicted from the current change trend, and then calculating the contrast coefficient corresponding to the hour to be predicted.
The hour to be predicted may be one hour after the current time.
Step S102, determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system;
in the embodiment, the micro-service quantity of each hour in a preset year before the current year in the distributed system is obtained first, and the micro-service quantity deviation value of each hour in the preset year before the current year is determined through a kernel function mean shift algorithm; and determining the average value of the micro service volume corresponding to the hour to be predicted according to the micro service volume deviation value of each hour in a preset year before the current year.
Wherein the preset year can be 3 years, 5 years, 10 years and the like.
Step S103, inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted;
before the micro-service capacity expansion operation, model training is performed on the request quantity prediction model in advance according to first historical request data in a preset year before the current year and a service with the same service scene as the first historical request data and second historical request data in the preset year before the current year to obtain a pre-trained request quantity prediction model.
In this embodiment, the request quantity of each hour in the current year in the distributed system is obtained, and the request quantity of each hour in the current year is the request quantity of each hour in the current year before the current time in the distributed system, then the request quantity of each hour in the current year is input into the pre-trained request quantity prediction model for model training, and after the model training is completed, the output result of the pre-trained request quantity prediction model is used as the prediction request quantity corresponding to the hour to be predicted.
Step S104, acquiring an hour request quantity of micro-services per hour processing requests and the current started micro-service quantity in the distributed system, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro-service quantity mean value, the hour request quantity and the started micro-service quantity;
in this embodiment, a predicted request amount corresponding to an hour to be predicted is obtained, an hour request amount for processing a request per hour by a micro service in a distributed system and a current number of started micro services are obtained, where the hour request amount is the number of requests that can be processed per hour by each micro service, and generally, the number of requests that can be processed per hour by each micro service may be small, and at this time, an average value of the number of requests that can be processed per hour by all micro services is taken as the hour request amount.
Then, based on the predicted request amount, the comparison coefficient, the average micro-service amount, the hour request amount and the started micro-service amount, determining the micro-service amount corresponding to the hour to be predicted, specifically, the formula of the micro-service amount is as follows:
C＝Math.ceil((Pt/Q)*R+Math.floor(Math.abs((Bc/S_AVGs-1)*Bc)))；
wherein, the Math.ceil function is an upward rounding function; floor function is a rounding function; abs is an absolute value function, Pt is a predicted request amount, Q is an hour request amount, R is a contrast coefficient, Bc is an initiated microservice amount, and S _ AVGs is a microservice amount average.
And step S105, performing micro-service capacity expansion operation on the distributed system based on the number of the micro-services corresponding to the hour to be predicted.
In this embodiment, when the number of micro services is obtained, a micro service capacity expansion operation is performed on the distributed system based on the number of micro services corresponding to an hour to be predicted, specifically, a difference between the number of micro services and the number of started micro services is calculated first, if the difference is greater than 0, the micro service capacity expansion operation is performed on the distributed system according to the difference, if the difference is less than 0, the micro service capacity reduction operation is performed on the distributed system according to the difference, and since each micro service in the distributed system corresponds to one message queue partition, when the micro service capacity expansion operation is performed, the message queue is increased or decreased and the micro services are simultaneously subjected to capacity expansion.
In the service expansion and contraction method for the distributed system provided by this embodiment, the comparison coefficient corresponding to the hour to be predicted is determined based on the historical change trend corresponding to the number of requests per hour in the distributed system and the current change trend corresponding to the number of requests per hour in the current year, where the number of requests is the number of requests corresponding to the micro-service; then, determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system; inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted; then acquiring an hour request quantity of micro-services processing requests per hour and the current started micro-service quantity in the distributed system, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro-service quantity average value, the hour request quantity and the started micro-service quantity; and finally, performing micro-service capacity expansion and reduction operation on the distributed system based on the micro-service quantity corresponding to the hour to be predicted, and performing micro-service capacity expansion and reduction on the distributed system by combining the micro-service quantity obtained by historical data prediction, so that the automatic capacity expansion and reduction of the micro-service in the distributed system are realized, and the efficiency and the accuracy of capacity expansion and reduction of the distributed system of the website are improved. Meanwhile, the manual participation is reduced in the micro-service scaling process, the micro-service scaling error caused by human errors can be avoided, the manual pressure can be reduced, and the cost can be saved.
Based on the first embodiment, a second embodiment of the service expansion and contraction method of the distributed system of the present invention is provided, in this embodiment, step S101 includes:
step S201, determining a historical change trend corresponding to the request quantity of each hour every year based on the request quantity of each hour in a preset year before the current year;
step S202, determining a current change trend corresponding to the request quantity of each hour in the current year based on the request quantity of each hour in the current year before the current time;
step S203, determining a contrast coefficient corresponding to the hour to be predicted based on the historical change trend and the current change trend.
In this embodiment, the request quantity of each hour in a preset year before the current year in the distributed system is first obtained, where the request quantity may include the historical request quantity of N years before the current year, the historical request quantity is divided into the hour request quantity, and then, according to the request quantity of each hour in the current year before the current time, the current change trend corresponding to the request quantity of each hour in the current year is determined through a kernel function mean shift algorithm.
Specifically, in an embodiment, the step S201 includes:
step a, determining a first request quantity deviation value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the request quantity of each hour in the preset year before the current year;
b, determining a first request quantity mean value of each hour every year based on the first request quantity offset value;
and c, determining the historical change trend corresponding to the request quantity of each hour every year based on the coordinate curve corresponding to the first request quantity average value.
In this embodiment, the request quantity of each hour in a preset year before the current year is obtained, then a first request quantity offset value of each hour in the preset year before the current year is determined through a kernel function mean shift algorithm, for the request quantity of each hour, a center point is initially randomly determined in a space with N data points through the kernel function mean shift algorithm, vector values of all points in a circular space with a radius r are calculated, finally, a mean value of all vectors in the whole circle is obtained, an offset mean value is obtained, the center point is moved to an offset mean value position, a next offset mean value is calculated again until the offset mean value meets that a flow peak value is equivalent to a server processing capacity value in a certain period of time, and then the current offset mean value is used as the first request quantity offset value. Specifically, the formula of the first request amount offset value is as follows:
Figure BDA0003168888130000101
wherein Mr (X) is the first request offset value, n is, X is the center point of the request, X isiFor requests within the range, n is the number of points, i.e., requests, and r is the radius.
Determining a first requested quantity mean value of each hour per year based on the first requested quantity offset value; specifically, the first request quantity deviation value is split, the request quantity deviation value of the same hour in the preset year is obtained, the mean value of the request quantity deviation values of the same hour in the preset year is used as the first request quantity mean value corresponding to the hour, and then the first request quantity mean value of each hour in each year is obtained, for example, ai，jIs the first request quantity offset value of the j hour of the ith year, j is 1, 2, 3, 4 … 24, j<8760; the first request quantity average value AVGj＝(ai，j+ai+1，j+…+an，j) And n is the number of years included in the preset year.
Determining a historical change trend corresponding to each hourly request quantity every year based on a coordinate curve corresponding to the first request quantity average value, specifically, generating a coordinate curve according to each first request quantity average value and corresponding hour j, wherein the abscissa of the coordinate curve is hour j, the abscissa of the coordinate curve is first request quantity average value, and then determining a historical change trend corresponding to each hourly request quantity every year according to a point corresponding to each first request quantity average value in the coordinate curve, for example, (AVG)j，j) The historical trend of the point can be determined by the slope of the point and its neighbors, for example, in a coordinate curve (AVG)j-1，j-1)、(AVGj，j)、(AVGj+1J +1) as the historical trend of the corresponding hour. The historical change trend can be accurately determined according to the first request deviation value, and the efficiency and the accuracy of the expansion and contraction of the distributed system are further improved.
And then, acquiring the request quantity of each hour in the current year before the current time in the distributed system, and determining the current change trend corresponding to the request quantity of each hour in the current year according to the request quantity of each hour in the current year before the current time. Specifically, in another embodiment, the step S202 includes:
step d, determining a second request quantity deviation value of each hour in the current year before the current time through a kernel function mean shift algorithm based on the request quantity of each hour in the current year before the current time;
step e, determining a second request quantity mean value of each hour in the current year based on the second request quantity deviation value;
and f, determining the current change trend corresponding to the request quantity of each hour in the current year based on the coordinate curve corresponding to the second request quantity average value.
In this embodiment, the second requested variable value, the second requested variable mean value, and the current variation trend are respectively calculated in the same manner as the first requested variable value, the first requested variable mean value, and the historical variation trend, and are not described herein again. The current change trend can be accurately determined according to the second request deviation value, and the efficiency and the accuracy of the expansion and contraction capacity of the distributed system are further improved.
Finally, based on the historical trend and the current trend, a contrast ratio corresponding to the hour to be predicted is determined, specifically, in another embodiment, the step S203 includes:
step g, acquiring a first variation trend corresponding to a target moment every day from historical variation trends, wherein the target moment is the moment of the hour to be predicted on the day;
step h, acquiring a second variation trend corresponding to the target time every day on the current variation trend;
and i, determining a contrast coefficient corresponding to the hour to be predicted based on the mean value of the first change trend and the mean value of the second change trend.
In this embodiment, the target time, that is, the time corresponding to the hour to be predicted, is determined, where the target time is the time of the hour to be predicted on the same day, specifically, the time of the day of an hour after the current hour, and specifically, one of 1, 2, 3, and 4 … 24. Then, a first change trend corresponding to the target time of each day is obtained in the historical change trends, and a second change trend corresponding to the target time of each day is obtained in the current change trend.
And then calculating the mean value of the first change trend and the mean value of the second change trend, and determining a contrast coefficient corresponding to the hour to be predicted based on the mean value of the first change trend and the mean value of the second change trend, wherein the contrast coefficient is the mean value of the second change trend/the mean value of the first change trend. The historical change trend and the current change trend of the current year are determined according to the number of requests in the historical data, the number of micro services can be accurately predicted according to the change trend, and the efficiency and the accuracy of expanding and contracting the capacity of the distributed system are further improved.
In the service expansion and contraction method for the distributed system, the historical change trend corresponding to the number of requests in each hour every year is determined based on the number of requests in each hour in a preset year before the current year; then, determining a current change trend corresponding to the request quantity of each hour in the current year based on the request quantity of each hour in the current year before the current time; and then, based on the historical change trend and the current change trend, determining a contrast coefficient corresponding to the hour to be predicted, determining the historical change trend and the current change trend of the current year according to the number of requests in the historical data, accurately predicting the number of micro services according to the change trend, and further improving the efficiency and the accuracy of the expansion and contraction of the distributed system.
Based on the first embodiment, a third embodiment of the service expansion and contraction method of the distributed system of the present invention is provided, in this embodiment, step S102 includes:
step S301, determining a micro-service volume offset value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the micro-service volume of each hour in the preset year before the current year;
step S302, determining the average value of the micro service volume of each hour every year based on the micro service volume deviant of each hour in a preset year before the current year;
and step S303, determining the average value of the micro service volume corresponding to the hour to be predicted in the average value of the micro service volume of each hour every year.
In this embodiment, the micro-service quantity of each hour in the preset year before the current year is obtained, and then the micro-service quantity offset value of the micro-service quantity of each hour in the preset year before the current year is determined through a kernel function mean shift algorithm, where the micro-service quantity offset value is the same as a calculation formula of the first request quantity offset value. When the calculation formula of the first request quantity deviation value is applied to the calculation of the micro service quantity deviation value, wherein Mr (X) is the micro service quantity deviation value, n is, X is the central point of the micro service, and X is the central point of the micro serviceiFor microservices within a range, n is the number of points, i.e., the number of requests, and r is the radius.
Then, calculating the average value of the micro service volume of each hour every year based on the offset value of the micro service volume of each hour in a preset year before the current year; specifically, the micro-service volume deviation value is split, the micro-service volume deviation value of the same hour in the preset year is obtained, the mean value of the micro-service volume deviation values of the same hour in the preset year is used as the mean value of the micro-service volume corresponding to the hour, and then the mean value of the micro-service volume of each hour in each year is obtained, for example, ai，jIs the micro-service volume offset value of j hour of i year, j is 1, 2, 3, 4 … 24, j<8760; then AVG average of micro-service volumej＝(ai，j+ai+1，j+…+an，j) And n is the number of years included in the preset year.
And finally, determining the average value of the micro service volume corresponding to the hour to be predicted in the average value of the micro service volume of each hour every year. The specific time of the hour to be predicted in the current year is determined, for example, the time in each year includes 1, 2, 3, 4 … 8760, and then the average value of the micro service volume average value of each hour per year, which corresponds to the same time as the specific time, is taken as the micro service volume average value of the hour to be predicted.
In the service expansion and capacity reduction method for the distributed system, based on the number of micro services in each hour in a preset year before the current year, the offset value of the micro service volume in each hour in the preset year before the current year is determined through a kernel function mean shift algorithm; then, determining the average value of the micro service volume of each hour every year based on the offset value of the micro service volume of each hour in a preset year before the current year; and then, in the average value of the micro service volume of each hour of each year, determining the average value of the micro service volume corresponding to the hour to be predicted, and screening the average value of the micro service volume corresponding to the hour to be predicted from the average value of the micro service volume of each hour obtained by the historical quantity of the micro service volume of each hour, so that the average value of the micro service volume can be accurately obtained according to historical data, and the efficiency and the accuracy of the distributed system for carrying out capacity expansion and contraction are further improved.
Based on the foregoing embodiments, a fourth embodiment of the service scaling method for a distributed system according to the present invention is provided, in this embodiment, before step S103, the service scaling method for a distributed system further includes:
step S401, acquiring first historical request data in a preset year before the current year, services with the same service scenes as the first historical request data and second historical request data in the preset year before the current year;
step S402, model training is carried out on the request quantity prediction model based on the first historical request data and the second historical request data, so that a pre-trained request quantity prediction model is obtained.
In the embodiment, first historical request data in a preset year before the current year and services with the same service scenes as the first historical request data are acquired, second historical request data in the preset year before the current year are acquired, then a request quantity prediction model is constructed according to sample data formed by the first historical request data and the second historical request data, a network comprising L hidden layers is created according to the number of samples in the sample data, each layer comprises 2n units, only one unit is arranged on the last layer of the network, the unit is not activated and is a linear layer, and a variable trend loss function is used for compiling the current network to obtain the request quantity prediction model; wherein, L and n can be set reasonably according to the size of the sample data.
Then, model training is carried out on the request quantity prediction model based on the first historical request data and the second historical request data, so that a pre-trained request quantity prediction model is obtained. Specifically, the sample data is divided into r groups of sub-samples, the sample correction variance of the ith group is Si2, N-r is the degree of freedom, and N is the number of samples in the sample data. Then, r groups of sub samples are trained for a preset round of model training until a pre-trained request quantity prediction model is obtained, for each round of model training, r groups of sub samples are traversed sequentially, a target sub sample is input into the request quantity prediction model for model training, the target sub sample is any one group of samples in the r groups of currently traversed sub samples, the trained request quantity prediction model is obtained, then the trained request quantity prediction model is used as the request quantity prediction model, the r groups of sub samples are traversed continuously until the traversal of the r groups of sub samples is completed, the request quantity prediction model and a loss function of the current round are obtained, and if the loss function is smaller than the preset loss, the request quantity prediction model of the current round is used as the pre-trained request quantity prediction model.
In the service expansion and contraction method for the distributed system, by acquiring first historical request data in a preset year before the current year, a service with the same service scene as the first historical request data, and second historical request data in the preset year before the current year; and then model training is carried out on the request quantity prediction model based on the first historical request data and the second historical request data to obtain a pre-trained request quantity prediction model, the request quantity prediction model is trained through the request data of other services with the same service scene of the historical request data of the distributed system, so that the prediction of the pre-trained request quantity prediction model is more accurate and credible, the overall algorithm is closer to the change trend of the actual service, the prediction of future micro-service capacity is more accurate, and the efficiency and the accuracy of the capacity expansion and contraction of the distributed system are further improved.
The present invention further provides a service scaling device of a distributed system, and referring to fig. 3, the service scaling device of the distributed system includes:
the first determining module 10 is configured to determine a comparison coefficient corresponding to an hour to be predicted based on a historical change trend corresponding to the number of requests per hour per year and a current change trend corresponding to the number of requests per hour per year in a distributed system, where the number of requests is the number of requests corresponding to micro services;
the second determining module 20 is configured to determine, in the distributed system, a mean value of micro service volumes corresponding to hours to be predicted through a kernel function mean shift algorithm based on the micro service volumes of each hour in a preset year before the current year;
the training module 30 is configured to input the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training, so as to obtain a prediction request quantity corresponding to an hour to be predicted;
the acquiring module 40 is configured to acquire an hour request quantity of the micro services processing the request per hour in the distributed system and a current number of started micro services, and determine a number of micro services corresponding to an hour to be predicted based on the predicted request quantity, the comparison coefficient, the average value of the micro services, the hour request quantity and the number of started micro services;
and the capacity expansion module 50 is configured to perform a micro-service capacity expansion operation on the distributed system based on the number of micro-services corresponding to the hour to be predicted.
The method executed by each program unit can refer to each embodiment of the service expansion and contraction method of the distributed system of the present invention, and is not described herein again.
The invention also provides a computer readable storage medium.
The computer readable storage medium of the present invention stores a service scaling program of a distributed system, and the service scaling program of the distributed system, when executed by a processor, implements the steps of the service scaling method of the distributed system as described above.
The method implemented when the service scaling program of the distributed system running on the processor is executed may refer to each embodiment of the service scaling method of the distributed system of the present invention, and is not described herein again.
In addition, an embodiment of the present invention further provides a computer program product, where the computer program product includes a service scaling program of a distributed system, and when executed by a processor, the service scaling program of the distributed system implements the steps of the service scaling method of the distributed system as described above.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. A service expansion and contraction method of a distributed system is characterized by comprising the following steps:
determining a comparison coefficient corresponding to the hours to be predicted based on a historical change trend corresponding to the number of requests of each hour in each year and a current change trend corresponding to the number of requests of each hour in the current year in the distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service;
determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system;
inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted;
acquiring an hour request quantity of micro-services processing requests per hour and the current started micro-service quantity in a distributed system, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, a comparison coefficient, a micro-service quantity average value, the hour request quantity and the started micro-service quantity;
and executing micro-service capacity expansion operation on the distributed system based on the number of the micro-services corresponding to the hour to be predicted.
2. The method as claimed in claim 1, wherein the step of determining the contrast ratio corresponding to the hour to be predicted based on the historical trend of change corresponding to the number of requests per hour per year and the current trend of change corresponding to the number of requests per hour per year in the distributed system comprises:
determining a historical change trend corresponding to the request quantity of each hour every year based on the request quantity of each hour in a preset year before the current year;
determining a current change trend corresponding to the request quantity of each hour in the current year based on the request quantity of each hour in the current year before the current moment;
and determining a contrast coefficient corresponding to the hour to be predicted based on the historical change trend and the current change trend.
3. The method for service scaling of a distributed system according to claim 2, wherein the step of determining the historical trend of change corresponding to the number of requests per hour per year based on the number of requests per hour in a preset year before the current year comprises:
determining a first request quantity deviation value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the request quantity of each hour in the preset year before the current year;
determining a first requested quantity mean value of each hour per year based on the first requested quantity offset value;
and determining the historical change trend corresponding to the request quantity of each hour every year based on the coordinate curve corresponding to the first request quantity average value.
4. The method for service scaling and capacity shrinkage of a distributed system according to claim 2, wherein the step of determining the current trend of change corresponding to the number of requests of each hour of the current year based on the number of requests of each hour of the current year before the current time comprises:
determining a second request quantity deviation value of each hour in the current year before the current time through a kernel function mean shift algorithm based on the request quantity of each hour in the current year before the current time;
determining a second request quantity average value of each hour in the current year based on the second request quantity deviation value;
and determining the current change trend corresponding to the request quantity of each hour in the current year based on the coordinate curve corresponding to the second request quantity mean value.
5. The service scaling method for distributed systems according to claim 2, wherein the step of determining the contrast ratio corresponding to the hour to be predicted based on the historical trend of change and the current trend of change comprises:
acquiring a first change trend corresponding to a target moment every day from historical change trends, wherein the target moment is the moment of the hour to be predicted on the day;
acquiring a second variation trend corresponding to each day of target time at the current variation trend;
and determining a contrast coefficient corresponding to the hour to be predicted based on the mean value of the first change trend and the mean value of the second change trend.
6. The method for service scaling and capacity shrinkage of a distributed system according to claim 1, wherein the step of determining the average value of the micro services corresponding to the hour to be predicted by using a kernel function mean shift algorithm based on the number of the micro services of each hour in a preset year before the current year in the distributed system comprises:
determining a micro-service quantity deviation value of each hour in a preset year before the current year through a kernel function mean shift algorithm based on the micro-service quantity of each hour in the preset year before the current year;
determining the average value of the micro service volume of each hour every year based on the offset value of the micro service volume of each hour in a preset year before the current year;
and determining the average value of the micro service volume corresponding to the hour to be predicted in the average value of the micro service volume of each hour every year.
7. The service expansion and contraction method of the distributed system according to any one of claims 1 to 6, wherein before the step of inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain the predicted request quantity corresponding to the hour to be predicted, the method further comprises:
acquiring first historical request data in a preset year before the current year, services with the same service scenes as the first historical request data and second historical request data in the preset year before the current year;
and performing model training on the request quantity prediction model based on the first historical request data and the second historical request data to obtain a pre-trained request quantity prediction model.
8. A service scaling apparatus for a distributed system, the service scaling apparatus for the distributed system comprising:
the first determining module is used for determining a comparison coefficient corresponding to an hour to be predicted based on a historical change trend corresponding to the number of requests of each hour every year and a current change trend corresponding to the number of requests of each hour every year in a distributed system, wherein the number of the requests is the number of the requests corresponding to the micro-service;
the second determination module is used for determining the average value of the micro-service volume corresponding to the hour to be predicted through a kernel function mean shift algorithm based on the micro-service volume of each hour in a preset year before the current year in the distributed system;
the training module is used for inputting the request quantity of each hour in the current year in the distributed system into a pre-trained request quantity prediction model for model training to obtain a prediction request quantity corresponding to the hour to be predicted;
the acquisition module is used for acquiring the hourly request quantity of the requests processed by the micro-services per hour in the distributed system and the current started micro-service quantity, and determining the micro-service quantity corresponding to the hour to be predicted based on the predicted request quantity, the comparison coefficient, the micro-service quantity mean value, the hourly request quantity and the started micro-service quantity;
and the capacity expansion and reduction module is used for executing micro-service capacity expansion and reduction operation on the distributed system based on the number of the micro-services corresponding to the hour to be predicted.
9. A service scaling device of a distributed system, the service scaling device of the distributed system comprising: memory, processor and a service scaling program of a distributed system stored on the memory and executable on the processor, the service scaling program of the distributed system implementing the steps of the service scaling method of a distributed system according to any of claims 1 to 7 when executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a service scaling program of a distributed system, which when executed by a processor implements the steps of the service scaling method of the distributed system according to any one of claims 1 to 7.
CN202110816331.2A 2021-07-19 Service expansion and contraction method, device, equipment and storage medium of distributed system Active CN113568741B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110816331.2A CN113568741B (en) 2021-07-19 Service expansion and contraction method, device, equipment and storage medium of distributed system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110816331.2A CN113568741B (en) 2021-07-19 Service expansion and contraction method, device, equipment and storage medium of distributed system

Publications (2)

Publication Number Publication Date
CN113568741A true CN113568741A (en) 2021-10-29
CN113568741B CN113568741B (en) 2024-05-10

Family

ID=

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019134623A1 (en) * 2018-01-02 2019-07-11 中国移动通信有限公司研究院 Capacity expansion and reduction method, device, apparatus and computer readable storage medium
CN110457287A (en) * 2019-07-03 2019-11-15 北京百度网讯科技有限公司 The scalable content processing method and device of database, computer equipment and readable medium
CN112199150A (en) * 2020-08-13 2021-01-08 北京航空航天大学 Online application dynamic capacity expansion and contraction method based on micro-service calling dependency perception
CN112559191A (en) * 2020-12-23 2021-03-26 平安银行股份有限公司 Method and device for dynamically deploying GPU (graphics processing Unit) resources and computer equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019134623A1 (en) * 2018-01-02 2019-07-11 中国移动通信有限公司研究院 Capacity expansion and reduction method, device, apparatus and computer readable storage medium
CN110457287A (en) * 2019-07-03 2019-11-15 北京百度网讯科技有限公司 The scalable content processing method and device of database, computer equipment and readable medium
CN112199150A (en) * 2020-08-13 2021-01-08 北京航空航天大学 Online application dynamic capacity expansion and contraction method based on micro-service calling dependency perception
CN112559191A (en) * 2020-12-23 2021-03-26 平安银行股份有限公司 Method and device for dynamically deploying GPU (graphics processing Unit) resources and computer equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王木旺;: "基于集群的高性能家庭影院认证授权系统研究", 现代电影技术, no. 06, pages 6 - 11 *

Similar Documents

Publication Publication Date Title
CN109074607B (en) Learned model providing method and learned model providing device
CN110321958B (en) Training method of neural network model and video similarity determination method
WO2022022152A1 (en) Video clip positioning method and apparatus, and computer device and storage medium
CN107451832B (en) Method and device for pushing information
CN112101169B (en) Attention mechanism-based road image target detection method and related equipment
CN111144584A (en) Parameter tuning method, device and computer storage medium
US20180197302A1 (en) Method and apparatus for determining to-be-superimposed area of image, superimposing image and presenting picture
CN112037223B (en) Image defect detection method and device and electronic equipment
CN115145801B (en) A/B test flow distribution method, device, equipment and storage medium
CN111222509A (en) Target detection method and device and electronic equipment
JP2022500786A (en) Information processing methods and devices, electronic devices, storage media and computer programs
CN111309240B (en) Content display method and device and electronic equipment
CN110069997B (en) Scene classification method and device and electronic equipment
KR20210048369A (en) Content recommendation system and content recommendation method using the same
CN114880310A (en) User behavior analysis method and device, computer equipment and storage medium
CN114420135A (en) Attention mechanism-based voiceprint recognition method and device
CN110555861B (en) Optical flow calculation method and device and electronic equipment
CN110378936B (en) Optical flow calculation method and device and electronic equipment
CN113568741A (en) Service expansion and contraction method, device, equipment and storage medium of distributed system
CN113568741B (en) Service expansion and contraction method, device, equipment and storage medium of distributed system
CN116415744A (en) Power prediction method and device based on deep learning and storage medium
CN114234984B (en) Indoor positioning track smoothing method, system and equipment based on difference matrix
CN111832354A (en) Target object age identification method and device and electronic equipment
CN114020755A (en) Image map tile publishing method, image map tile generating method and device
CN113283115B (en) Image model generation method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant