CN107450968B - Load reduction method, device and equipment - Google Patents

Load reduction method, device and equipment Download PDF

Info

Publication number
CN107450968B
CN107450968B CN201610377787.2A CN201610377787A CN107450968B CN 107450968 B CN107450968 B CN 107450968B CN 201610377787 A CN201610377787 A CN 201610377787A CN 107450968 B CN107450968 B CN 107450968B
Authority
CN
China
Prior art keywords
kernel function
resource dimension
resource
load
replay
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610377787.2A
Other languages
Chinese (zh)
Other versions
CN107450968A (en
Inventor
韩锐
王振涛
袁泉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Institute of Computing Technology of CAS
Original Assignee
Huawei Technologies Co Ltd
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, Institute of Computing Technology of CAS filed Critical Huawei Technologies Co Ltd
Priority to CN201610377787.2A priority Critical patent/CN107450968B/en
Publication of CN107450968A publication Critical patent/CN107450968A/en
Application granted granted Critical
Publication of CN107450968B publication Critical patent/CN107450968B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the invention provides a load reduction method, a device and equipment. The method comprises the following steps: acquiring a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; acquiring a load replay log according to the evaluation requirement of the user and the public cloud log; acquiring a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set according to the total resource utilization rate of each playback unit in each resource dimension and the load behavior of each kernel function in the first kernel function set; and acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension. The method greatly improves the precision of load reduction.

Description

Load reduction method, device and equipment
Technical Field
The embodiment of the invention relates to computer technology, in particular to a load restoration method, a device and equipment.
Background
On a modern public cloud platform, a large number of tenants (i.e., cloud computing users) share cloud computing infrastructure resources, which include Central Processing Unit (CPU) resources, memory, and Input/Output (I/O) resources, and are used to perform various types of operations (e.g., CPU-intensive, I/O-intensive operations). Therefore, many jobs (or loads) are often executed concurrently at the same node of the data center, forming a mixed load. According to corresponding statistical data, the public cloud mixed load has two typical characteristics: the load behavior of each load in the mixed load is highly differentiated, and different loads are different in load behaviors such as submission modes, operation time, resource requirements, architecture characteristics and the like; secondly, a large amount of loads running in a short period are submitted and completed continuously, and the load behavior of the mixed load changes frequently.
Generally, when a new system is developed, the performance of the system needs to be evaluated. Therefore, when the system is evaluated, it is often necessary to restore a true mixed load according to the load behavior of the mixed load recorded in the public cloud log, and make the mixed load actually run on the system, and evaluate the performance of the system by looking at the load behavior or the running time of the actual running of the mixed load. The load reduction method in the prior art is specifically based on Facebook and Yahoo! The cloud platform log is a method for restoring Hadoop MapReduce operation widely existing in a cloud platform, and specifically comprises the following steps:
firstly, describing the load behavior of MapReduce operation, and then sampling real logs according to an evaluation time period required by a user to obtain load replay logs; and finally, generating MapReduce load input data based on the load replay log, and further generating simulated MapReduce operation based on I/O resource dimension on a Hadoop platform.
However, in the load restoration method in the prior art, only the restoration of the load is considered from the IO resource dimension, and the actual execution logic of the load is not considered, that is, the usage rates of the CPU and the memory resource and the load characteristics of the corresponding architecture are not considered, so that the accuracy of the load restoration is not high.
Disclosure of Invention
The embodiment of the invention provides a load reduction method, a device and equipment, which are used for solving the technical problems of single dimension consideration and low load reduction accuracy of load reduction in the prior art
In a first aspect, an embodiment of the present invention provides a load reduction method, including:
acquiring a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions;
acquiring a load replay log according to the evaluation requirement of the user and the public cloud log; wherein each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively;
acquiring a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set according to the total resource utilization rate of each playback unit in each resource dimension and the load behavior of each kernel function in the first kernel function set; wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension;
and acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension.
In the load restoration method provided by the first aspect, the first kernel function set is obtained from the kernel function library according to the load behavior information recorded in the public cloud log, and the load replay log is obtained according to the evaluation requirement of the user and the public cloud log, so that the load restoration device obtains the second kernel function set corresponding to each resource dimension of each replay unit from the first kernel function set according to the total resource utilization rate of each replay unit in each resource dimension and the load behavior of each kernel function in the first kernel function set, and further obtains the mixed load for evaluating the system performance according to the second kernel function set of all replay units in the load replay log in each resource dimension. The method provided by the embodiment of the invention considers the load constraint of a plurality of resource dimensions during load reduction, the accuracy of the reduced mixed load is high, and the kernel function is adopted to reduce the load in the embodiment of the invention, so that the time for system evaluation can be shortened by shortening the running time of the kernel function, and the evaluation efficiency of the system is greatly improved.
In one possible design, each replay unit further includes a target architecture feature value of all loads in the current replay unit in each resource dimension; the obtaining, according to the second kernel function sets of all the playback units in the load playback log in each resource dimension, a mixed load for evaluating system performance specifically includes:
determining a kernel function sequence of each playback unit on each resource dimension according to a second kernel function set corresponding to each playback unit on each resource dimension;
determining an optimal kernel function combination of each replay unit in each resource dimension from the kernel function sequence according to a target architecture characteristic value of each replay unit in each resource dimension; wherein the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum;
and acquiring the mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension.
In a possible design, the determining, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension according to a target architecture feature value of each playback unit in each resource dimension specifically includes:
and determining the optimal kernel function combination of each replay unit in each resource dimension from the kernel function sequence by adopting a dynamic programming method according to the target architecture characteristic value of each replay unit in each resource dimension.
According to the load restoration method provided by the possible design, the kernel function sequence of each replay unit on each resource dimension is determined according to the second kernel function set corresponding to each replay unit on each resource dimension, the optimal kernel function combination of each replay unit on each resource dimension is determined from the kernel function sequence according to the target architecture characteristic value of each replay unit on each resource dimension, and further the mixed load for evaluating the system performance is obtained according to the optimal kernel function combination of all replay units in the load replay log on each resource dimension. By fully considering the system structure characteristics of the load during load restoration, the optimal kernel function combination with the minimum error of the target system structure characteristic value recorded in the load replay log is selected, so that the error between the system structure characteristic value generated when the restored mixed load runs on the system to be evaluated and the target system structure characteristic value on the corresponding resource dimension is minimum, and the restoration accuracy of the mixed load is greatly improved.
In one possible design, after obtaining the optimal kernel function combination of each of the playback units in the spatial resource dimension, the method further includes:
acquiring load behavior information of each playback unit corresponding to the optimal kernel function combination in the space resource dimension in the CPU resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
In one possible design, the set of spatial kernels is a set of memory kernels or a set of IO kernels;
the acquiring, according to a total resource usage rate of each playback unit in each resource dimension and a load behavior of each kernel function in the first kernel function set, a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set specifically includes:
step A: determining a first space capacity of the playback unit according to the total resource utilization rate of the playback unit in space resource dimensions, and determining the maximum use number of each space kernel function under the first space capacity according to the first space capacity and the space resource utilization rate of each space kernel function in the space kernel function set;
and B: determining a second kernel function set corresponding to the space resource dimension of the playback unit according to the maximum using number of each space kernel function under the first space capacity;
and C: determining a second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource utilization rate of the replay unit in the CPU resource dimension;
step D: determining the maximum using number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU core function in the CPU core function set;
step E: determining a second kernel function set corresponding to the CPU resource dimension of the playback unit according to the maximum using number of each CPU kernel function under the second space capacity;
step F: and B, respectively executing the step A to the step E on each replay unit in the load replay log to obtain a second kernel function set corresponding to all replay units in the load replay log in a space resource dimension and a CPU resource dimension.
In a possible design, the determining, by using a dynamic programming method, an optimal kernel function combination of each playback unit in each resource dimension from the kernel function sequence according to a target architecture feature value of each playback unit in each resource dimension specifically includes:
step G: determining the optimal kernel function combination of the replay unit on the space resource dimension from the kernel function sequence of the replay unit on the space resource dimension by adopting a dynamic programming method according to the first target architecture characteristic value of the replay unit on the space resource dimension;
step H: determining a third target architecture characteristic value of the replay unit on the CPU resource dimension according to the optimal kernel function combination of the replay unit on the space resource dimension, the CPI value generated during the system operation and the second target architecture characteristic value of the replay unit on the CPU resource dimension;
step I: according to a third target architecture characteristic value of the replay unit in the CPU resource dimension, determining an optimal kernel function combination of the replay unit in the CPU resource dimension from a kernel function sequence of the replay unit in the CPU resource dimension by adopting a dynamic programming method;
step J: and G to I are respectively executed for each replay unit in the load replay log, and the optimal kernel function combination corresponding to all replay units in the load replay log in the space resource dimension and the CPU resource dimension is obtained.
According to the load restoration method provided by the possible design, different types of jobs (or loads) are abstracted by considering the information of public cloud tenants in combination with kernel functions in a function library, the distribution condition of the loads is analyzed from multiple resource dimensions, in combination with replay units in a load replay log, the optimal kernel function combination with different resource dimensions is obtained for each replay unit, load restoration is carried out based on the optimal kernel function combination, the restored mixed loads can reproduce the real scene of a data center, the precision is high, and therefore an evaluation basis with quantification and high reliability is provided for an evaluation system; meanwhile, the optimal kernel function combination of each replay unit in each resource dimension is obtained by adopting a dynamic programming method, the obtaining efficiency of the optimal kernel function combination is greatly improved, the evaluation time of the system can be shortened by adjusting the running time of the kernel function, and the evaluation efficiency of the system performance is improved to a certain extent.
In a possible design, the obtaining a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log specifically includes:
determining a load behavior range according to the load behavior information recorded in the public cloud log;
and acquiring the kernel functions meeting the load behavior range from the kernel function library according to the load behavior and the load behavior range of each kernel function in the kernel function library to form the first kernel function set.
According to the load reduction method provided by the possible design, a load behavior range is determined according to load behavior information recorded in a public cloud log, and according to the load behavior of each kernel function in a kernel function library and the load behavior range, kernel functions meeting the load behavior range are obtained from the kernel function library to form a first kernel function set, so that the kernel functions in the first kernel function set have certain typicality, the first kernel function set can provide better basis when the optimal kernel functions in each resource dimension are determined subsequently, and the effectiveness and the accuracy of load reduction are ensured.
In a possible design, the evaluation requirement of the user includes at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
In a possible design, the obtaining a load replay log according to the evaluation requirement of the user and the public cloud log specifically includes:
selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units, and each first playback unit comprises the resource utilization rate of each load in each current first playback unit in each resource dimension;
summing the resource utilization rate of each load in each first playback unit in each resource dimension to obtain the total resource utilization rate of each first playback unit in each resource dimension;
carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain a target architecture characteristic value of each first playback unit on each resource dimension;
and acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
According to the load restoration method provided by the possible design, the log segment is selected from the public cloud log according to the evaluation requirement of the user, the content of each first playback unit in the log segment is converted, the content of the playback unit in the load playback log is obtained, and the load playback log is further obtained, so that the load restoration device can restore based on the fine-grained playback unit when load restoration is performed, and the restored load is higher in accuracy.
In a second aspect, an embodiment of the present invention provides a load reduction apparatus, including:
the kernel function analysis and acquisition module is used for acquiring a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions;
the replay log obtaining module is used for obtaining a load replay log according to the evaluation requirement of the user and the public cloud log; wherein each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively;
a kernel function matching module, configured to obtain, from the first kernel function set, a second kernel function set corresponding to each playback unit in each resource dimension according to a total resource usage rate of each playback unit in each resource dimension and a load behavior of each kernel function in the first kernel function set; wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension;
and the load restoration module is used for acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension.
In one possible design, each replay unit further includes a target architecture feature value of all loads in the current replay unit in each resource dimension; the load reduction module specifically comprises:
a sequence determining unit, configured to determine, according to a second kernel function set corresponding to each of the playback units in each of the resource dimensions, a kernel function sequence of each of the playback units in each of the resource dimensions;
a combination optimization unit, configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension according to a target architecture feature value of each playback unit in each resource dimension; wherein the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum;
and the load restoration unit is used for acquiring the mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension.
In a possible design, the combination optimization unit is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
In one possible design, the apparatus further includes:
an obtaining module, configured to obtain load behavior information of the optimal kernel function combination of each playback unit in the space resource dimension, corresponding to the CPU resource dimension, after the combination optimization unit obtains the optimal kernel function combination of each playback unit in the space resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
In one possible design, the set of spatial kernels is a set of memory kernels or a set of IO kernels; the kernel function matching module specifically includes:
a determining unit, configured to determine a first space capacity of the playback unit according to a total resource usage rate of the playback unit in a space resource dimension, and determine a maximum number of used space kernel functions in the first space capacity according to the first space capacity and a space resource usage rate of each space kernel function in the space kernel function set; determining a second kernel function set corresponding to the replay unit in the space resource dimension according to the maximum use number of each space kernel function under the first space capacity, and determining the second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource use rate of the replay unit in the CPU resource dimension; determining the maximum use number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource use rate of each CPU core function in the CPU core function set, and determining a second core function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU core function under the second space capacity;
and the circulation unit is used for instructing the determining unit to acquire the second kernel function sets corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension respectively.
In a possible design, the combinatorial optimization unit specifically includes:
the dynamic optimization subunit is used for determining an optimal kernel function combination of the playback unit in the space resource dimension from the kernel function sequence of the playback unit in the space resource dimension by adopting a dynamic programming method according to a first target architecture characteristic value of the playback unit in the space resource dimension, and determining a third target architecture characteristic value of the playback unit in the CPU resource dimension according to a CPI value generated by the optimal kernel function combination of the playback unit in the space resource dimension during system operation and a second target architecture characteristic value of the playback unit in the CPU resource dimension; determining the optimal kernel function combination of the replay unit on the CPU resource dimension from the kernel function sequence of the replay unit on the CPU resource dimension by adopting a dynamic programming method according to the third target architecture characteristic value of the replay unit on the CPU resource dimension;
and the circulation subunit is used for instructing the dynamic optimization unit to respectively acquire the optimal kernel function combinations corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension.
In a possible design, the kernel function analysis obtaining module is specifically configured to determine a load behavior range according to load behavior information recorded in the public cloud log, and obtain, according to a load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
In a possible design, the evaluation requirement of the user includes at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
In a possible design, the replay log obtaining module specifically includes:
the log segment selecting unit is used for selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units, and each first playback unit comprises the resource utilization rate of each load in each current first playback unit in each resource dimension;
the processing unit is used for summing the resource utilization rate of each load in each first playback unit on each resource dimension to obtain the total resource utilization rate of each first playback unit on each resource dimension, and carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain the target architecture characteristic value of each first playback unit on each resource dimension;
and the replay log acquisition unit is used for acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
The beneficial effects of the load reduction device provided by the second aspect and the possible designs of the second aspect may refer to the beneficial effects brought by the first aspect and the possible designs of the first aspect, and are not described herein again.
In a third aspect, an embodiment of the present invention provides a load reduction device, including:
the processor is used for acquiring a first kernel function set from a kernel function library according to load behavior information recorded in the public cloud log, and acquiring a load replay log according to the evaluation requirement of a user and the public cloud log; acquiring a second kernel function set corresponding to each resource dimension of each replay unit from the first kernel function set according to the total resource utilization rate of each replay unit on each resource dimension and the load behavior of each kernel function in the first kernel function set, and acquiring a mixed load for evaluating the system performance according to the second kernel function sets of all replay units in the load replay log on each resource dimension; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions; each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively; the sum of the resource usage of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage in the resource dimension.
In one possible design, each replay unit further includes a target architecture feature value of all loads in the current replay unit in each resource dimension;
the processor is specifically configured to determine, according to a second kernel function set corresponding to each playback unit in each resource dimension, a kernel function sequence of each playback unit in each resource dimension, and determine, according to a target architecture feature value of each playback unit in each resource dimension, an optimal kernel function combination of each playback unit in each resource dimension from the kernel function sequence; acquiring a mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension; and the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum.
In a possible design, the processor is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
In one possible design, the processor is further configured to, after obtaining the optimal kernel function combination of each of the playback units in the spatial resource dimension, obtain load behavior information corresponding to the optimal kernel function combination of each of the playback units in the spatial resource dimension in the CPU resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
In one possible design, the set of spatial kernels is a set of memory kernels or a set of IO kernels;
the processor is specifically configured to determine a first space capacity of the playback unit according to a total resource utilization rate of the playback unit in a space resource dimension, determine a maximum number of used spatial kernel functions of each type in the first space capacity according to the first space capacity and a space resource utilization rate of each type of spatial kernel function in the spatial kernel function set, and determine a second kernel function set corresponding to the playback unit in the space resource dimension according to the maximum number of used spatial kernel functions of each type in the first space capacity; and determining a second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource utilization rate of the replay unit in the CPU resource dimension, determining the maximum use number of each CPU kernel function in the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU kernel function in the CPU kernel function set, determining a second kernel function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU kernel function in the second space capacity, and further obtaining the second kernel function set corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension respectively.
In one possible design, the processor is specifically configured to determine, from the sequence of kernels of the playback unit in the spatial resource dimension, an optimal kernel function combination of the playback unit in the spatial resource dimension by using a dynamic programming method according to a first target architecture feature value of the playback unit in the spatial resource dimension, determine a third target architecture feature value of the playback unit in the CPU resource dimension according to a value of CPI generated by the playback unit in the spatial resource dimension during system runtime and a second target architecture feature value of the playback unit in the CPU resource dimension, and determine, from the sequence of kernels of the playback unit in the CPU resource dimension, an optimal kernel function combination of the playback unit in the CPU resource dimension by using a dynamic programming method according to the third target architecture feature value of the playback unit in the CPU resource dimension, and further obtaining the optimal kernel function combination of all the replay units in the load replay log, which corresponds to the space resource dimension and the CPU resource dimension respectively.
In one possible design, the processor is specifically configured to determine a load behavior range according to load behavior information recorded in the public cloud log, and obtain, according to a load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
In a possible design, the evaluation requirement of the user includes at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
In a possible design, the processor is specifically configured to select a log segment from the public cloud log according to the evaluation requirement of the user, and perform a summation operation on the resource usage rate of each load in each first playback unit in each resource dimension to obtain a total resource usage rate of each first playback unit in each resource dimension; performing average calculation on the architecture characteristic value of each load in each first replay unit on each resource dimension to obtain a target architecture characteristic value of each first replay unit on each resource dimension, and acquiring a load replay log according to the total resource utilization rate of each first replay unit on each resource dimension and the target architecture characteristic value of each first replay unit on each resource dimension; the log segment includes a plurality of first replay units, and the first replay units include resource usage rates of each load in each resource dimension in the current first replay unit.
The beneficial effects of the load reduction device provided by the possible designs of the third aspect and the third aspect may refer to the beneficial effects brought by the possible designs of the first aspect and the first aspect, and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a first embodiment of a load reduction method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a second embodiment of a load reduction method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a dynamic programming method according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of a third embodiment of a load reduction method according to an embodiment of the present invention;
fig. 5 is a schematic flow chart of a fourth embodiment of the load reduction method according to the embodiment of the present invention;
fig. 6 is a schematic flow chart of a fifth embodiment of a load reduction method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a first load reduction device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a second load reduction device according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a third load reduction device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a fourth load reduction device according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a first load reduction device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
It should be understood that although the terms first, second, third, etc. may be used to describe XXX in embodiments of the present invention, these XXX should not be limited to these terms. These terms are only used to distinguish XXX from each other. For example, a first XXX may also be referred to as a second XXX, and similarly, a second XXX may also be referred to as a first XXX, without departing from the scope of embodiments of the present invention.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.
The load restoration method, the load restoration device and the load restoration equipment provided by the embodiment of the invention can be suitable for load restoration based on public cloud platform logs, and can evaluate the performance scene of the developed system through the restored load. Alternatively, the load restored here may be a hybrid load, which may include multiple loads (or multiple jobs), each load having its own load behavior. Optionally, the load behavior may include: load commit mode, run time, resource usage, architectural features, etc. Optionally, the resource may include at least one of a CPU resource, a memory resource, and an IO resource; the types and the architectural features of the resource utilization rates of the same load in different resource dimensions are different, optionally, for the same load, in the CPU resource dimension, the resource utilization rate is the utilization rate of the CPU resource, and the architectural feature is a load calculation characteristic (Cycles Per Instruction, CPI for short), which refers to the number of clock Cycles required by each Instruction of the CPU; in the dimension of Memory resources, the resource utilization rate is the utilization rate of the Memory resources, and the architecture of the dimension is characterized by load access per Instruction (MAI), wherein the MAI refers to the Memory access frequency of each Instruction of the CPU; in the dimension of the IO resource, the resource utilization rate is the utilization rate of the IO resource, and the architectural feature of the dimension is a load IO characteristic (Input/Output Operations Per Second, referred to as IOPS), where the IOPS refers to the number of times of reading and writing IO Per Second.
The embodiment of the invention relates to a load reduction method, a load reduction device and load reduction equipment, and aims to solve the technical problems that in the prior art, the load reduction is single in dimension and low in load reduction accuracy. Optionally, the method of this embodiment may be a load reduction device, and may also be a load reduction apparatus, and the load reduction device may be integrated in the load reduction apparatus. The following embodiments are described by taking as an example that the main execution body is a load reduction device.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 1 is a schematic flow chart of a load reduction method according to a first embodiment of the present invention. The embodiment relates to a specific process that load reduction accuracy is improved by reducing loads from different resource dimensions by load reduction equipment. As shown in fig. 1, the method includes:
s101: and acquiring a first kernel function set from a kernel function library according to the load behavior information recorded in the public cloud log.
The first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions.
Specifically, the public cloud log may record load behavior information of each load on each node under the public cloud platform, where the load behavior information may include information such as resource usage rates, architecture features, runtime, and submission modes of the load on different resource dimensions. The kernel function library may include a CPU kernel function set and a space kernel function set, where the CPU kernel function set may include a plurality of CPU kernel functions, and each CPU function represents a certain load behavior in a CPU resource dimension, such as a CPU resource usage rate and a CPI; the space kernel function set may include a memory kernel function set and/or an IO kernel function set, where the memory kernel function set includes a plurality of memory kernel functions, each memory kernel function also represents a certain load behavior in a memory resource dimension, such as a memory resource usage rate and an MAI, and the IO kernel function set may include a plurality of IO kernel functions, and each IO kernel function also represents a certain load behavior in an IO resource dimension, such as an IO resource usage rate and an IOPS.
Therefore, the load restoration device may analyze the load behavior (load characteristics) of each kernel function in the kernel function library, match the load behavior (load characteristics) with the load behavior information recorded in the public cloud log, and screen out the matched kernel functions to form a first kernel function set. Optionally, the matching operation here may be to screen out all kernel functions whose load behaviors are smaller than the maximum load behavior information recorded in the public cloud log to form a first kernel function set, may also be to screen out all kernel functions whose load behaviors are larger than the minimum load behavior information recorded in the public cloud log and smaller than the maximum load behavior information recorded in the public cloud log to form a first kernel function set, and may also be other matching manners.
S102: and acquiring a load replay log according to the evaluation requirement of the user and the public cloud log.
Wherein each replay unit in the load replay log comprises the total resource usage of all loads in the current replay unit in each resource dimension respectively.
Specifically, after the load restoring device obtains the first kernel function set, the load restoring device obtains the evaluation requirement of the user, optionally, the user may directly input the evaluation requirement of the system to be evaluated to the load restoring device, or the user may write the evaluation requirement of the system to be evaluated into a format file, and the load restoring device may obtain the evaluation requirement of the user by parsing the file. Optionally, the evaluation requirement is used to characterize a requirement of a user on the load replay log, and optionally, the evaluation requirement may include at least one of a machine type recorded in the load replay log, a number of machines corresponding to the machine type recorded in the load replay log, and a time period of the load replay log.
After the load restoring device obtains the evaluation requirement of the user, a load replay log meeting the evaluation requirement of the user is screened from the public cloud log according to the evaluation requirement of the user, the load replay log can be divided into a plurality of replay units according to a certain replay unit (for example, divided by seconds), each replay unit comprises the total resource utilization rate of all loads in the current replay unit on each resource dimension, and the total resource utilization rate comprises the total utilization rate of all loads in the current replay unit in the CPU resource dimension, the total utilization rate in the memory resource dimension and the total utilization rate in the IO resource dimension.
S103: and acquiring a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set according to the total resource utilization rate of each playback unit in each resource dimension and the load behavior of each kernel function in the first kernel function set.
Wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension.
For convenience of explanation of the technical solution of the present invention, one playback unit a is used for explanation, and operations of other playback units may be performed with reference to the playback unit, that is, operations of the embodiments of the present invention are the same for each playback unit.
Specifically, after the load reduction device obtains the total resource utilization of the playback unit a in each resource dimension, the load reduction device obtains, in combination with the load behavior of each kernel function in the kernel function library, for example, in combination with the resource utilization of each kernel function in different resource dimensions, a second kernel function set corresponding to the playback unit a in each resource dimension from the first kernel function set, where a sum of the resource utilizations of the same kernel function in the second kernel function set in the resource dimension does not exceed the total resource utilization in the resource dimension.
For example, it is assumed that the total utilization rate of CPU resources of all loads in the playback unit a in the CPU resource dimension is 30%, the total utilization rate of memory resources in the memory resource dimension is 40%, and the total utilization rate of IO resources in the IO resource dimension is 20%, where the first kernel function set includes a CPU kernel function set, a memory kernel function set, and an IO kernel function set, and the CPU kernel function set includes 3 CPU kernel functions, which are CPU kernel function 1, CPU kernel function 2, and CPU kernel function 3, respectively; the memory kernel function set comprises 2 memory kernel functions, namely a memory kernel function 1 and a memory kernel function 2; the IO kernel function set includes 2 IO kernel functions, which are IO kernel function 1 and IO kernel function 2, respectively. The CPU resource utilization rate of the CPU kernel function 1 is 8%, the CPU resource utilization rate of the CPU kernel function 2 is 12%, the CPU resource utilization rate of the CPU kernel function 3 is 10%, the memory resource utilization rate of the memory kernel function 1 is 15%, the memory resource utilization rate of the memory kernel function 2 is 20%, the memory resource utilization rate of the IO kernel function 1 is 5%, and the memory resource utilization rate of the IO kernel function 2 is 10%.
Based on the above data, the load restoration device may determine that the second function set a corresponding to the playback unit a in the CPU resource dimension includes 3 CPU core functions 1, 2 CPU core functions 2, and 3 CPU core functions 3, that is, for the CPU core function 1, the sum of the resource usage rates of the CPU core function 1 in the CPU resource dimension is 8% + 8% + 24% or less than 30%, the sum of the resource usage rates of the CPU core function 2 in the CPU resource dimension is 12% + 24% or less than 30%, and the sum of the resource usage rates of the CPU core function 3 in the CPU resource dimension is 10% + 10% + 10% + 30% or less than 30%; similarly, the second kernel function set B corresponding to the memory resource dimension of the playback unit a includes 2 memory kernel functions 1 and 2 memory kernel functions 2, that is, for the memory kernel function 1, the sum of the resource usage rates of the memory kernel function 1 in the memory resource dimension is 15% + 30% and less than 40%, and the sum of the resource usage rates of the memory kernel function 2 in the memory resource dimension is 20% + 40% and also does not exceed 40%; the second kernel function set C corresponding to the playback unit a in the IO resource dimension includes 4 IO kernel functions 1 and 2 IO kernel functions 2, that is, for the IO kernel function 1, the sum of the resource usage rates of the IO kernel function 1 in the IO resource dimension is 5% + 5% + 20%, which is not more than 20%, and the sum of the resource usage rates of the IO kernel function 2 in the IO resource dimension is 10% + 20%, which is not more than 20%.
In summary, the playback unit a corresponds to a second kernel function set a in the CPU resource dimension, a second kernel function set B in the memory resource dimension, and a second sum function set C in the IO resource dimension. According to the method, the load restoration device can obtain the second kernel function set corresponding to each playback unit in each resource dimension.
It should be noted that, when the first kernel function set only includes a CPU kernel function set and a memory kernel function set, the corresponding resource dimensions are the CPU resource dimensions and the memory resource dimensions, that is, the load reduction device needs to acquire the second kernel function set corresponding to each playback unit in the CPU resource dimensions and the memory resource dimensions; when the first kernel function set only includes a CPU kernel function set and an IO kernel function set, the corresponding resource dimensions are the CPU resource dimension and the IO resource dimension, that is, the load reduction device needs to acquire a second kernel function set corresponding to each playback unit in the CPU resource dimension and in the IO resource dimension; when the first kernel function set includes a CPU kernel function set, a memory kernel function set, and an IO kernel function set, the corresponding resource dimensions are the CPU resource dimension, the memory resource dimension, and the IO resource dimension, that is, the load reduction device needs to acquire the second kernel function set corresponding to each playback unit in the CPU resource dimension, the memory resource dimension, and the IO resource dimension.
S104: and acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension.
Specifically, after the load restoring device obtains the second kernel function set of each playback unit in each resource dimension, the load restoring device may perform load restoration based on any combination of kernel functions in the second kernel function set in each resource dimension, so as to obtain a mixed load for evaluating system performance. The mixed load restored by the method considers the load constraint of multiple resource dimensions, the restored mixed load is high in accuracy, and the kernel function is adopted to restore the load in the embodiment of the application, so that the time for system evaluation can be shortened by shortening the running time of the kernel function, and the evaluation efficiency of the system is greatly improved.
According to the load restoration method provided by the embodiment of the invention, the first kernel function set is obtained from the kernel function library according to the load behavior information recorded in the public cloud log, and the load replay log is obtained according to the evaluation requirement of the user and the public cloud log, so that the load restoration device obtains the second kernel function set corresponding to each resource dimension of each replay unit from the first kernel function set according to the total resource utilization rate of each replay unit in each resource dimension and the load behavior of each kernel function in the first kernel function set, and further obtains the mixed load for evaluating the system performance according to the second kernel function set of all replay units in the load replay log in each resource dimension. The method provided by the embodiment of the invention considers the load constraint of a plurality of resource dimensions during load reduction, the accuracy of the reduced mixed load is high, and the kernel function is adopted to reduce the load in the embodiment of the invention, so that the time for system evaluation can be shortened by shortening the running time of the kernel function, and the evaluation efficiency of the system is greatly improved.
Fig. 2 is a schematic flow chart of a second embodiment of the load reduction method according to the embodiment of the present invention. The embodiment relates to a specific process of obtaining a kernel function sequence through a second kernel function set and performing load reduction according to the kernel function sequence. On the basis of the foregoing embodiment, the foregoing S104 may specifically include:
s201: and determining a kernel function sequence of each playback unit on each resource dimension according to the second kernel function set corresponding to each playback unit on each resource dimension.
Specifically, after the load restoration device obtains the second kernel function set corresponding to each resource dimension of each playback unit, the kernel function sequence in the resource dimension may be determined according to the second kernel function set. Also taking the above-mentioned playback unit a as an example, the other playback units may perform the process with reference to the playback unit a, specifically:
the playback unit a corresponds to three second kernel function sets, which are a second kernel function set a of the CPU resource dimension, a second kernel function set B of the memory resource dimension, and a second kernel function set C of the IO resource dimension, where the second kernel function set a includes 3 CPU kernel functions 1 (set as C1), 2 CPU kernel functions 2 (set as C2), and 3 CPU kernel functions 3 (set as C3), the second kernel function set B includes 2 memory kernel functions 1 (set as m1) and 2 memory kernel functions 2 (set as m1), and the second kernel function set C includes 4 IO kernel functions 1 (set as x1) and 2 IO kernel functions 2 (set as x 2). The load reduction device may determine, according to a certain sequence composition rule, that the kernel function sequence corresponding to the second kernel function set a is (c1, c1, c1, c2, c2, c3, c3, c3), and optionally may also form other ordered sequences, which is not limited in this embodiment of the present invention; similarly, the load reduction device determines that the kernel function sequence corresponding to the second kernel function set B is (m1, m1, m2, m2), and determines that the kernel function sequence corresponding to the second kernel function set C is (x1, x1, x1, x1, x2, x 2).
In the same way, the load restoration device may obtain a sequence of kernel functions for each playback unit in each resource dimension.
S202: and determining the optimal kernel function combination of each replay unit in each resource dimension from the kernel function sequence according to the target architecture characteristic value of each replay unit in each resource dimension.
And the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value is minimum.
Specifically, in this embodiment of the present invention, each playback unit further includes target architecture feature values of all loads in the current playback unit in each resource dimension, and in addition to the total CPU resource usage rate, the total memory resource usage rate, and the total IO resource usage rate in the IO resource dimension of all loads in the current playback unit, the target architecture feature value in the CPU resource dimension (e.g., a target CPI value), the target architecture feature value in the memory resource dimension (e.g., a target MAI value), and the target architecture feature value in the IO resource dimension (e.g., a target IOPS value), taking the retransmission unit a as an example.
Taking the resource dimension of the playback unit A, CPU as an example, the kernel function sequence corresponding to the second kernel function set a is (c1, c1, c1, c2, c2, c3, c3, c3), and the corresponding kernel function combination may have a value of 28The load reduction equipment can be from 28And selecting an optimal kernel function combination (the optimal kernel function combination can comprise at least one kernel function) from the kernel function combinations, wherein the optimal kernel function combination is only required to ensure that the error between the architecture characteristic value (CPI) of the CPU resource dimension generated by the optimal kernel function combination in the system operation and the target architecture characteristic value of the CPU resource dimension is minimum.
Optionally, when the optimal kernel function combination is selected, each kernel function combination may be estimated and run one by one to check a CPI value generated in the CPU resource dimension, and then a kernel function combination with the minimum error from the target architecture feature value in the CPU resource dimension is selected.
Optionally, the load restoration device may also obtain the optimal kernel function combination by using a dynamic programming method, specifically refer to the flowchart shown in fig. 3, and refer to the description related to fig. 3 in the following embodiment for how to obtain the optimal kernel function combination of the playback unit a in the CPU resource dimension by using the dynamic programming method, which is not described herein again. The optimal kernel function combination is obtained through a dynamic programming method, the time for obtaining the optimal kernel function combination can be greatly shortened, and therefore the system evaluation efficiency is effectively improved.
According to the method described above, the load restoring device can obtain the optimal kernel function combination of the playback unit a in the memory resource dimension and the optimal kernel function combination in the IO resource dimension, and can also obtain the optimal kernel function combination of each playback unit in each resource dimension.
S203: and acquiring the mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension.
Specifically, the load restoration device may combine each of the obtained playback units at each optimal kernel function as each playback script, then combine the playback scripts, and perform load restoration based on the combined playback scripts, so that an error between the restored architecture feature value of the mixed load and a target architecture feature value in a corresponding resource dimension is minimized, and the restoration accuracy of the mixed load is greatly improved.
According to the load restoration method provided by the embodiment of the invention, the kernel function sequence of each replay unit in each resource dimension is determined according to the second kernel function set corresponding to each replay unit in each resource dimension, the optimal kernel function combination of each replay unit in each resource dimension is determined from the kernel function sequence according to the target architecture characteristic value of each replay unit in each resource dimension, and further, the mixed load for evaluating the system performance is obtained according to the optimal kernel function combination of all replay units in the load replay log in each resource dimension. By fully considering the system structure characteristics of the load during load restoration, the optimal kernel function combination with the minimum error of the target system structure characteristic value recorded in the load replay log is selected, so that the error between the system structure characteristic value generated when the restored mixed load runs on the system to be evaluated and the target system structure characteristic value on the corresponding resource dimension is minimum, and the restoration accuracy of the mixed load is greatly improved.
Fig. 4 is a schematic flow chart of a third embodiment of the load reduction method according to the embodiment of the present invention. The embodiment relates to a specific process of acquiring a first kernel function set from a kernel function library by load restoration equipment according to load behavior information recorded in a public cloud log. On the basis of the foregoing embodiment, further, the foregoing S101 may specifically include:
s301: and determining a load behavior range according to the load behavior information recorded in the public cloud log.
Specifically, the load restoration device may analyze load behavior information of all loads recorded in the public cloud log, and then determine a load behavior range corresponding to each resource dimension based on each resource dimension. Optionally, taking the CPU resource dimension as an example, the load behavior range may be [ P, Q ], where P may be the minimum CPU resource usage rate or the minimum CPI, and Q may be the corresponding maximum CPU resource usage rate or the maximum CPI, and of course, P and a may also be other range values as long as P is less than Q.
S302: and acquiring the kernel functions meeting the load behavior range from the kernel function library according to the load behavior and the load behavior range of each kernel function in the kernel function library to form the first kernel function set.
Specifically, the kernel function library may include three types of kernel function sets, which are a CPU kernel function set, a memory kernel function set, and an IO kernel function set, respectively, where the kernel functions of each set may include multiple kernel functions, for example, the CPU kernel function set may include different types of CPU kernel functions (for example, CPU kernel function 1 and CPU kernel function 2 … … CPU kernel function n), the memory kernel function set may include different types of memory kernel functions (for example, memory kernel function 1 and memory kernel function 2 … … memory kernel function n), and the IO kernel function set may include different types of IO kernel functions (for example, IO kernel function 1 and IO kernel function 2 … … IO kernel function n). The load reduction equipment needs to analyze the load behavior of each kernel function in the kernel function library, that is, the CPU resource utilization rate, the memory resource utilization rate, the IO resource utilization rate, the architecture characteristics CPI, MAI, the IOPS, and the like of each kernel function are analyzed, then the load reduction equipment judges which kernel functions have load behaviors within the load behavior range based on the determined load behavior range, and further screens out the kernel functions meeting the load behavior range to form a first kernel function set.
According to the load restoration method provided by the embodiment of the invention, a load behavior range is determined according to the load behavior information recorded in the public cloud log, and the kernel function meeting the load behavior range is obtained from the kernel function library according to the load behavior of each kernel function in the kernel function library and the load behavior range to form a first kernel function set, so that the kernel functions in the first kernel function set have certain typicality, the first kernel function set can provide better basis when the optimal kernel function combination on each resource dimension is determined subsequently, and the effectiveness and the accuracy of load restoration are ensured.
Fig. 5 is a schematic flow chart of a fourth embodiment of the load reduction method according to the embodiment of the present invention. The embodiment relates to a specific process of acquiring a load replay log by load restoration equipment according to the evaluation requirement of a user and the public cloud log. On the basis of the foregoing embodiment, further, the foregoing S102 may specifically include the following steps:
s401: selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units, and the first playback units comprise the resource utilization rate of each load in each resource dimension in the current first playback units.
Specifically, the user evaluation requirement is used to characterize what load replay log the user wants to obtain, that is, the user evaluation requirement may include at least one of a machine type that should be recorded in the load replay log, a number of machines corresponding to the machine type that should be recorded in the load replay log, and a time period of the load replay log. Then, the load restoration device selects a log segment meeting the evaluation demand from the public cloud logs according to the evaluation demand of the user, wherein the log segment comprises a plurality of first playback units, and each first playback unit comprises the resource utilization rate of each load in each current first playback unit in each resource dimension.
S402: and summing the resource utilization rate of each load in each first playback unit on each resource dimension to obtain the total resource utilization rate of each first playback unit on each resource dimension.
Specifically, after the load restoring device obtains the log segment that meets the evaluation requirement of the user, since each first playback unit of the log segment includes the resource usage rate of each load in each resource dimension in the current first playback unit, the load restoring device accumulates the resource usage rates in the same resource dimension in the first playback unit, and further obtains the total resource usage rate of all the loads in each resource dimension in the first playback unit. For example, suppose that 100 first playback units are included in a log segment, wherein one first playback unit includes 3 loads, the CPU resource usage rates of the 3 loads in the first retransmission unit are 20%, 15%, and 10%, respectively, and the memory resource usage rates in the first retransmission unit are 30%, 25%, and 20%, respectively; therefore, the load restoration device may superimpose the CPU resource usage rates of the three loads, and superimpose the memory resource usage rates of the three loads, so that the total CPU resource usage rate of the first playback unit in the CPU resource dimension is 45%, and the total memory resource usage rate in the memory resource dimension is 75%.
S403: and carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain a target architecture characteristic value of each first playback unit on each resource dimension.
Specifically, continuing with the above example, the load restoring apparatus may perform an average calculation on the architectural feature values of each load in the first playback unit in the same resource dimension, so as to obtain the target architectural feature value of the first playback unit in the resource dimension. Optionally, the average calculation may be arithmetic average, weighted average, or average in other manners, which is not limited in this embodiment of the present invention.
This is performed for each first playback unit, in the manner described above, resulting in a target architectural feature value for each first playback unit in each resource dimension.
Optionally, there is no limitation on the time sequence relationship between S402 and S403, both may be performed simultaneously, or S403 may be performed before S402.
S404: and acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
Specifically, after obtaining the total resource utilization rate of each first playback unit in each resource dimension and the target architecture characteristic value of each first playback unit in each resource dimension, the load restoration device takes the total resource utilization rate of the same first playback unit in each resource dimension and the target architecture characteristic value of each first playback unit in each resource dimension as the content of one playback unit in the load playback log, and so on, obtains a plurality of playback units of the load playback log, and further obtains the load playback log.
According to the load restoration method provided by the embodiment of the invention, the log segment is selected from the public cloud log according to the evaluation requirement of the user, the content of each first playback unit in the log segment is converted, the content of the playback unit in the load playback log is obtained, and the load playback log is further obtained, so that the load restoration device can restore based on the fine-grained playback unit when performing load restoration, and the restored load has higher precision.
To more specifically explain the technical solution of the embodiment of the present invention, the following embodiment is issued from the perspective of the playback unit a to describe the overall process of the mixed load restoration. The resource dimension of the load related to the playback unit A comprises a CPU resource dimension, a memory resource dimension and an IO resource dimension, and the first kernel function set comprises a CPU kernel function set, a memory kernel function set and an IO kernel function set. See in particular the following examples:
fig. 6 is a schematic flow chart of a fifth embodiment of the load reduction method according to the embodiment of the present invention. The present embodiment relates to a specific process of obtaining an optimal kernel function combination of the playback unit a in each resource dimension based on the playback unit a. As shown in fig. 6, the method includes:
s501: and determining a load behavior range according to the load behavior information recorded in the public cloud log.
S502: and acquiring the kernel functions meeting the load behavior range from the kernel function library according to the load behavior and the load behavior range of each kernel function in the kernel function library to form the first kernel function set.
S503: and selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units.
The first playback unit includes a resource usage rate of each load in the current first playback unit in each resource dimension.
S504: and summing the resource utilization rate of each load in each first playback unit on each resource dimension to obtain the total resource utilization rate of each first playback unit on each resource dimension.
S505: and carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain a target architecture characteristic value of each first playback unit on each resource dimension.
S506: and acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
Specifically, the specific processes of S501 to S506 may refer to the descriptions of the third embodiment and the fourth embodiment, and are not described herein again.
Based on this, the load restoration device obtains a load replay log, where each replay unit in the load replay log includes a total resource usage rate of all loads in the current replay unit in each resource dimension, respectively, and a target architecture feature value of all loads in the current replay unit in each resource dimension, respectively. In the following description, a playback unit a is taken as an example, and it is assumed that the first kernel function set includes M memory kernel functions, P IO kernel functions, and Q CPU kernel functions.
S507: determining a first space capacity of the playback unit A according to the total resource utilization rate of the playback unit A in the space resource dimension, and determining the maximum use number of each space kernel function under the first space capacity according to the first space capacity and the space resource utilization rate of each space kernel function in the space kernel function set.
Optionally, the space resource dimension may be a memory resource dimension, and may also be an IO resource dimension.
Specifically, in this embodiment, the spatial kernel function set may be a memory kernel function set or an IO kernel function set, and S507 may be executed for any kernel function set of the two sets. Since the first kernel function set in this embodiment includes the CPU kernel function set, the memory kernel function set, and the IO kernel function set, the load reduction device needs to execute the process of S507 with respect to the memory kernel function set and the IO kernel function set, specifically:
the load restoration device determines the first space capacity of the playback unit a according to the total resource usage of the playback unit a in the memory resource dimension (i.e., the total memory resource usage in the playback unit a). Alternatively, the first space capacity may be a conversion form of the total usage of the memory resources in the playback unit a, for example, assuming that the total usage of the memory resources of the playback unit a is 25%, the first space capacity may be 25, and of course, the embodiment of the present invention does not limit the conversion form between the total usage of the resources and the total space capacity in one resource dimension, as long as the conversion form between the total usage of the resources and the total space capacity is consistent with the following conversion form of the usage of the resources and the space capacity for each kernel function.
After the load restoring device obtains the first space capacity of the playback unit a, the load restoring device performs the same conversion for the memory resource utilization rate of each memory kernel function in the memory kernel function set, so as to determine the maximum number of each memory kernel function used under the first space capacity. For example, assuming that the first space capacity of the playback unit a is 25, the memory resource usage rate of the memory kernel function 1 in the memory kernel function set is 8% (the space occupied by the memory kernel function after the conversion is 8), the memory resource usage rate of the memory kernel function 2 is 9% (the space occupied by the memory kernel function after the conversion is 9), and the memory resource usage rate of the memory kernel function 3 is 7% (the space occupied by the memory kernel function after the conversion is 7), the load reduction device determines that the maximum usage number of the memory kernel functions 1 is 3, the maximum usage number of the memory kernel functions 2 is 2, and the maximum usage number of the memory kernel functions 3 is 3 at the first space capacity.
Similarly, the load reduction device may also obtain, according to a similar method, the maximum number of used IO kernel functions of the playback unit a in the first space capacity of the IO resource dimension, that is, the load reduction device determines, according to the total resource usage rate of the playback unit a in the IO resource dimension, the first space capacity of the playback unit a in the IO resource dimension, and then determines, according to the first space capacity and the IO resource usage rate of each IO kernel function in the IO kernel function set, the maximum number of used IO kernel functions of each IO kernel function in the first space capacity.
S508: and determining a second kernel function set corresponding to the replay unit A in the space resource dimension according to the maximum using number of each space kernel function under the first space capacity.
Specifically, the load reduction device may add the maximum number of memory kernel functions used for each type in the first space capacity of the memory resource dimension to obtain the number of memory kernel functions in the second kernel function set in the memory resource dimension (N1 is set, and N1 is smaller than M); optionally, the load reduction device may further add the maximum number of used IO kernel functions of each type in the first space capacity of the IO resource dimension to obtain the number of IO kernel functions in the second kernel function set in the IO resource dimension (set to be N2, where N2 is smaller than P). For example, if the load reduction device determines that the maximum number of memory kernel functions 1(k1) used by the playback unit a in the first space capacity of the memory resource dimension is 3, the maximum number of memory kernel functions 2(k2) used is 2, and the maximum number of memory kernel functions 3(k3) used is 3, the load reduction device determines that the total number N1 of memory kernel functions in the second kernel function set of the playback unit a in the memory resource dimension, which includes 3 k1, 2 k2, and 3 k3 (set as the second kernel function set a), is 8.
Similarly, according to the method, the load reduction device may also obtain a second kernel function set (set as the second kernel function set B) of the playback unit a in the IO resource dimension and a sum of the numbers of IO kernel functions in the second kernel function set.
S509: and determining the kernel function sequence of the playback unit A in the space resource dimension according to the second kernel function set corresponding to the playback unit A in the space resource dimension.
Specifically, the load reduction device needs to determine a kernel function sequence (set as kernel function sequence a) of the memory resource dimension of the playback unit a according to the second kernel function set corresponding to the memory resource dimension of the playback unit a, and also needs to determine a kernel function sequence (set as kernel function sequence b) of the IO resource dimension of the playback unit a according to the second kernel function set corresponding to the IO resource dimension of the playback unit a.
Optionally, continuing with the above example as an example, the sequence of kernel functions corresponding to the memory resource dimension of the playback unit determined by the load restore device may be (k1, k1, k1, k2, k2, k3, k3, k 3). Optionally, the load restoring device may also determine the kernel function sequence B of the playback unit a in the IO resource dimension according to the second kernel function set B and a certain sequence composition rule, which is not illustrated here.
S510: and determining the optimal kernel function combination of the playback unit A in the space resource dimension from the kernel function sequence of the playback unit A in the space resource dimension by adopting a dynamic programming method according to the first target architecture characteristic value of the playback unit A in the space resource dimension.
Specifically, the load reduction device needs to determine, by using a dynamic programming method, an optimal kernel function combination (set as an optimal kernel function combination X) of the playback unit a in the memory resource dimension from a kernel function sequence a of the playback unit a in the memory resource dimension according to a first target architecture feature value Vtarget1 of the playback unit a in the memory resource dimension; and determining the optimal kernel function combination (set as the optimal kernel function combination Y) of the playback unit a in the memory resource dimension from the kernel function sequence b of the playback unit a in the IO resource dimension by using a dynamic programming method according to the first target architecture feature value Vtarget2 of the playback unit a in the IO resource dimension.
Taking an example that the load reduction device determines the optimal kernel function combination X of the playback unit a in the memory resource dimension from the kernel function sequence a corresponding to the memory resource dimension by using a dynamic programming method, with reference to the flow diagram shown in fig. 3:
for convenience of explanation, it is assumed that the load reduction device determines, according to the load behavior range in S501 and the load behavior of each kernel function in the kernel function library analyzed in S502, that the first kernel function set includes three memory kernel functions kmem1, kmem2 and kmem3 (the memory kernel functions often have relatively small CPU resource usage rates and CPI values), and three CPU kernel functions kCPU1，kCPU2And kCPU3The CPU kernel often has a relatively large CPU utilization and CPI value, and does not have a memory resource utilization and MAI access. The load behavior of the three memory core functions and the three CPU core functions can be seen in table 1:
TABLE 1
Figure BDA0001005891740000271
Figure BDA0001005891740000281
Then, the load restoring apparatus may obtain a load replay log according to the evaluation requirement of the user, and assume that the content of the replay unit a in the load replay log is as shown in table 2:
TABLE 2
Figure BDA0001005891740000282
With reference to table 1 and table 2, the first space capacity of the playback unit a in the memory resource dimension determined by the load reduction device is 25, the space occupied by k1 in the first kernel function set is 8, the space occupied by k2 is 9, and the space occupied by k3 is 7, so that the load reduction device determines that the playback unit a includes 3 k1, 2 k2, and 3 k3 in the second kernel function set a in the memory resource dimension, that is, the total number of the second kernel function set a is 8, and optionally, the load reduction device determines that the kernel function order a determined according to the second kernel function set is (k1, k1, k1, k2, k2, k3, k3, k 3).
Based on this, in FIG. 3, kiFor the ith kernel function of the kernel function sequence of the replay unit A in the memory resource dimension, N1 is the total number of the memory kernel functions of the replay unit A in the kernel function sequence a of the memory resource dimension (N1 is 8), C1 is the first space capacity of the replay unit A in the memory resource dimension (25), Vtarget1 is the first target architecture feature value of the replay unit A in the memory resource dimension (0.034), W [ k ],i]as a memory kernel function kiSize of occupied space, Vki]As a memory kernel function kiArchitectural characteristic value, V [ i, x, in playback Unit A]When the capacity is x, the error between the architecture characteristic and the target architecture characteristic value is generated when the optimal kernel function combination consisting of the first i memory kernel functions runs in the system.
The specific process of fig. 3 is as follows:
when determining the optimal kernel function combination from the kernel function sequence, the load reduction device gradually judges which memory kernel functions should be put into the optimal kernel function combination when the capacity x is increased from 1 to 25 from 1.
S601: when x is 1 and V [0, x ] is 1, the error of the first target architecture feature value Vtarget1 generated by the optimal kernel function combination consisting of the first 0 memory kernel functions during system operation is equal to the value of Vtarget 1.
S602: when x is less than C1, S603 is performed; when x is greater than or equal to C1, x is incremented by +1, and execution returns to S601.
S603: judging W [ k ]i]Whether x is less than or equal to x; if not, executing S604; if so, 607 is performed.
S604: let V [ i, x ] ═ V [ i-1, x ], S605 is continued.
Specifically, for the 1 st memory kernel function k1, if the space (8) occupied by the memory kernel function k1 is greater than 1, it means that V [1, x ] ═ V [1-1, x ] ═ V [0, x ], i.e. the error between the architectural feature and Vtarget1 generated by the first 1 memory kernel functions during system operation is the same as the error between the architectural feature and Vtarget1 generated by the first 0 memory kernel functions during system operation, because the 1 st memory kernel function is not added to the optimal kernel function combination.
S605: and continuously judging whether the current x is smaller than C1, if so, executing S606, and if not, executing S608.
S606: and adding 1 to x, and returning to execute S603.
Specifically, the process of x ═ 1 is continued, when it is determined again that current x ═ 1 is still smaller than C1(25), x +1 is returned to execution loop S603, and when x ═ 8 is reached, i is still 1, it is determined again that space (8) occupied by the 1 st memory kernel function is equal to the value of current x (the value of current x is 8), then it is determined to place the 1 st memory kernel function in the optimal kernel function combination (i.e., it is determined to replay the first memory kernel function), and S607 is executed to determine the error between the 1 st memory kernel function in the optimal kernel function combination and Vtarget 1.
S607: an error from Vtarget1 when the first i memory kernels are placed in the optimal kernel combination is determined according to the formula V [ i, x ] ═ min { V [ i-1, x ], E (V [ i ], V [ i-1, x-W [ i ] ], Vtarget) }, and S605 is continued.
Where V [ i-1, x ] refers to the error from Vtarget1 when the ith kernel function combination is not replayed (i.e., the error from Vtarget1 and the optimal kernel function combination including the first i-1 kernel functions), E (V [ i ], V [ i-1, x-W [ i ] ], Vtarget1) refers to the error from Vtarget1 when the first i-1 memory kernel functions are placed after the ith memory kernel function is placed in the optimal kernel function combination.
S608: judging whether the current i is smaller than N1; if yes, executing S609; if not, go to S610.
S609: and (5) clearing the i +1 and the current x, and returning to execute the step S603.
S610: and outputting the optimal kernel function combination and the corresponding error.
According to the flow diagram of the dynamic programming shown in fig. 3, the optimal kernel function combination of the playback unit a in the memory resource dimension can be obtained. With continued reference to the above example, the optimal kernel combinations obtained by the dynamic programming method are (k1, k2, k3), and the optimal kernel combinations are used as playback scripts of the dimension of the memory resource of the playback unit a.
For obtaining the optimal kernel function combination of the playback unit a in the IO resource dimension by using the dynamic programming method, reference may be made to the process shown in fig. 3, which is not described herein again.
It should be noted that, the optimal kernel function combination of the playback unit a in the memory resource dimension or in the IO resource dimension is obtained by using a dynamic programming method, which can greatly shorten the acquisition time of the optimal kernel function combination, and further greatly improve the evaluation efficiency of the system.
S511: and acquiring load behavior information corresponding to the CPU resource dimension by the optimal kernel function combination of the playback unit A on the space resource dimension.
The space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
Specifically, because the memory kernel function or the IO kernel function may occupy a certain amount of CPU space capacity and may generate a certain CPI when the system operates, after the load reduction device determines the optimal kernel function combination X of the playback unit a in the memory resource dimension, the CPU space capacity that the optimal kernel function combination X needs to occupy and the generated CPI when the optimal kernel function combination X operates on the system need to be obtained; moreover, after the load restoration device determines the optimal kernel function combination Y of the playback unit a in the IO resource dimension, it also needs to obtain the CPU space capacity that the optimal kernel function combination Y needs to occupy and the generated CPI when running on the system.
S512: and determining the second space capacity of the playback unit A according to the CPU space capacity required to be occupied by the optimal kernel function combination of the playback unit A in the space resource dimension during the system operation and the total resource utilization rate of the playback unit A in the CPU resource dimension.
Specifically, the load restoration device first converts the total resource utilization rate of the playback unit a in the CPU resource dimension into the total CPU space capacity, and then subtracts the CPU space capacity that the optimal kernel function combination of the playback unit a in the space resource dimension needs to occupy when the system operates from the total CPU space capacity, that is, subtracts the CPU space capacity that the optimal kernel function combination X needs to occupy when the system operates and the CPU space capacity that the optimal kernel function combination Y needs to occupy when the system operates from the total CPU space capacity, thereby obtaining the second space capacity of the playback unit a.
S513: and determining the maximum using number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU core function in the CPU core function set.
After the load restoring device obtains the second space capacity of the playback unit a, the load restoring device also performs space capacity conversion on the CPU resource utilization rate of each CPU core function in the CPU core function set, thereby determining the maximum number of used CPU core functions of each type at the second space capacity. For example, assuming that the first space capacity of the playback unit a is 30, the CPU resource usage rate of the CPU core function 1 in the CPU core function set is 6% (the space occupied by the CPU core function after the conversion is 6), the CPU resource usage rate of the CPU core function 2 is 10% (the space occupied by the CPU core function after the conversion is 10), and the CPU resource usage rate of the CPU core function 3 is 15% (the space occupied by the CPU core function after the conversion is 15), the load reduction device determines that the maximum usage number of the CPU core functions 1 is 5, the maximum usage number of the CPU core functions 2 is 3, and the maximum usage number of the CPU core functions 3 is 2 at the second space capacity.
S514: and determining a second kernel function set corresponding to the CPU resource dimension of the playback unit according to the maximum using number of each CPU kernel function under the second space capacity.
Specifically, the load reduction device may add the maximum number of used CPU core functions of each type at the second space capacity of the CPU resource dimension to obtain the number of CPU core functions (set to N2) in the second set of core functions at the CPU resource dimension, and continuing with the above example as an example, if the load reduction device determines that the maximum number of used CPU core functions 1(k1) of the playback unit a at the first space capacity of the CPU resource dimension is 5, the maximum number of used CPU core functions 2(k2) is 3, and the maximum number of used CPU core functions 3(k3) is 2, then the load reduction device determines that the total number N2 of CPU core functions in the second set of core functions of the playback unit a at the CPU resource dimension is 10, and the second set of core functions (set to the second set of core functions C) at the CPU resource dimension includes 5 k1, 3 k2, and 2 k 3.
S515: and determining a third target architecture characteristic value of the replay unit in the CPU resource dimension according to the optimal kernel function combination of the replay unit A in the space resource dimension, wherein the CPI value generated in the system operation process is combined with the second target architecture characteristic value of the replay unit A in the CPU resource dimension.
Specifically, the memory kernel function or the IO kernel function may occupy not only a certain amount of CPU space capacity but also a certain CPI during system operation, so that the load reduction device may perform mean value calculation on the CPI value generated by the optimal kernel function combination X during system operation, the CPI value generated by the optimal kernel function combination Y during system operation, and the second target architecture feature value of the playback unit a in the CPU resource dimension, which is recorded by the playback unit a itself, to obtain a third target architecture feature value of the playback unit a in the CPU resource dimension, where the third target architecture feature value considers an actual operation logic of the load and is an accurate value, and it may ensure that the accuracy of the optimal kernel function combination of the playback unit a in the CPU resource dimension is obtained by subsequently adopting a dynamic programming method.
S516: and determining the optimal kernel function combination of the playback unit in the CPU resource dimension from the kernel function sequence of the playback unit A in the CPU resource dimension by adopting a dynamic programming method according to the third target architecture characteristic value of the playback unit A in the CPU resource dimension.
Specifically, the process may specifically refer to the flowchart shown in fig. 3, and only need to replace Vtarget1 in the flowchart with a third target architecture feature value, replace C1 with a second space capacity, and replace the memory kernel function with a CPU kernel function, so as to obtain the optimal kernel function combination T of the playback unit a in the CPU resource dimension.
Based on this, the load reduction device obtains the optimal kernel function combination X of the playback unit a in the memory resource dimension, the optimal kernel function combination Y in the IO resource dimension, and the optimal kernel function combination T in the CPU resource dimension, and then the load reduction device takes the optimal kernel function combination X as one playback script, the optimal kernel function combination Y as another playback script, and the optimal kernel function combination T as another playback script.
S517: s507-516 are respectively executed on each playback unit in the load playback log to obtain the optimal kernel function combination of each playback unit in each resource dimension, and the mixed load for evaluating the system is obtained according to the optimal kernel function combination of each playback unit in each resource dimension.
Specifically, the load restoring device may process each playback unit based on each resource dimension according to the methods in S507 to S516 described above, to obtain an optimal kernel function combination of each playback unit in each resource dimension, and then the load restoring device obtains a playback script of each playback unit in each resource dimension. Then, the load restoration device can restore the load based on the replay scripts to obtain the mixed load for evaluating the system. By adopting the method provided by the embodiment of the invention, the public cloud mixed load can be restored with high precision, and further, the rapid generation and evaluation of the mixed load can be supported by adjusting the running time of the kernel function, for example, the evaluation time can be shortened to one tenth of the original time by adjusting the running time of the kernel function to one tenth of the original time, and meanwhile, the high precision of restoration of the mixed load is ensured by considering a plurality of resource dimensions.
According to the load restoration method provided by the embodiment of the invention, different types of jobs (or loads) are abstracted by considering the information of public cloud tenants in combination with kernel functions in a function library, the distribution condition of the loads is analyzed from multiple resource dimensions, in combination with replay units in a load replay log, an optimal kernel function combination with different resource dimensions is obtained for each replay unit, load restoration is carried out based on the optimal kernel function combination, the restored mixed load can reproduce the real scene of a data center, the precision is high, and therefore, an evaluation basis with quantification and high reliability is provided for an evaluation system; meanwhile, the optimal kernel function combination of each replay unit in each resource dimension is obtained by adopting a dynamic programming method, the obtaining efficiency of the optimal kernel function combination is greatly improved, the evaluation time of the system can be shortened by adjusting the running time of the kernel function, and the evaluation efficiency of the system performance is improved to a certain extent.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Fig. 7 is a schematic structural diagram of a first embodiment of a load reduction device according to an embodiment of the present invention, where the device may be implemented by software, or may be implemented by hardware or a combination of hardware and software. As shown in fig. 7, the apparatus may include: the system comprises a kernel function analysis obtaining module 10, a replay log obtaining module 11, a kernel function matching module 12 and a load restoring module 13.
Specifically, the kernel function analysis obtaining module 10 is configured to obtain a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions;
the replay log obtaining module 11 is configured to obtain a load replay log according to the evaluation requirement of the user and the public cloud log; wherein each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively;
a kernel function matching module 12, configured to obtain, from the first kernel function set, a second kernel function set corresponding to each playback unit in each resource dimension according to a total resource usage rate of each playback unit in each resource dimension and a load behavior of each kernel function in the first kernel function set; wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension;
and the load restoring module 13 is configured to obtain a mixed load for evaluating system performance according to the second kernel function sets of all the playback units in the load playback log on each resource dimension.
The load reduction device provided by the embodiment of the invention can execute the method embodiment, the realization principle and the technical effect are similar, and the detailed description is omitted.
Fig. 8 is a schematic structural diagram of a second embodiment of the load reduction device provided in the embodiment of the present invention, where the device may be implemented by software, or may be implemented by hardware, or by a combination of hardware and software. On the basis of the above embodiment shown in fig. 7, in this embodiment, each of the playback units further includes target architecture characteristic values of all loads in the current playback unit in each resource dimension; as shown in fig. 8, the load reduction module 13 specifically includes: a sequence determination unit 131, a combinatorial optimization unit 132 and a load restoration unit 133.
Specifically, the sequence determining unit 131 is configured to determine a kernel function sequence of each playback unit in each resource dimension according to a second kernel function set corresponding to each playback unit in each resource dimension;
a combination optimization unit 132, configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension according to a target architecture feature value of each playback unit in each resource dimension; wherein the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum;
and a load restoring unit 133, configured to obtain a mixed load for evaluating system performance according to the optimal kernel function combination of all the playback units in the load playback log in each resource dimension.
Further, the combination optimization unit 132 is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
The load reduction device provided by the embodiment of the invention can execute the method embodiment, the realization principle and the technical effect are similar, and the detailed description is omitted.
Fig. 9 is a schematic structural diagram of a third embodiment of the load reduction device provided in the embodiment of the present invention, where the device may be implemented by software, or may be implemented by hardware, or by a combination of hardware and software. On the basis of the embodiment shown in fig. 8, as shown in fig. 9, the apparatus may further include an obtaining module 14;
the obtaining module 14 is configured to obtain load behavior information of the CPU resource dimension corresponding to the optimal kernel function combination of each playback unit in the space resource dimension after the combination optimization unit 132 obtains the optimal kernel function combination of each playback unit in the space resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
Further, with continued reference to fig. 9, the spatial kernel function set is a memory kernel function set or an IO kernel function set; the kernel function matching module 12 specifically includes:
a determining unit 121, configured to determine a first space capacity of the playback unit according to a total resource usage rate of the playback unit in a space resource dimension, and determine a maximum number of used space kernel functions in the first space capacity according to the first space capacity and a space resource usage rate of each space kernel function in the space kernel function set; determining a second kernel function set corresponding to the replay unit in the space resource dimension according to the maximum use number of each space kernel function under the first space capacity, and determining the second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource use rate of the replay unit in the CPU resource dimension; determining the maximum use number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource use rate of each CPU core function in the CPU core function set, and determining a second core function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU core function under the second space capacity;
a loop unit 122, configured to instruct the determining unit 121 to obtain second kernel function sets corresponding to all the playback units in the load playback log in the space resource dimension and the CPU resource dimension, respectively.
Further, with reference to fig. 9, the combination optimization unit 132 specifically includes:
a dynamic optimization subunit 1321, configured to determine, by using a dynamic programming method, an optimal kernel function combination of the playback unit in the spatial resource dimension from the kernel function sequence of the playback unit in the spatial resource dimension according to the first target architecture feature value of the playback unit in the spatial resource dimension, and determine a third target architecture feature value of the playback unit in the CPU resource dimension according to a CPI value generated by the optimal kernel function combination of the playback unit in the spatial resource dimension during system runtime and the second target architecture feature value of the playback unit in the CPU resource dimension; determining the optimal kernel function combination of the replay unit on the CPU resource dimension from the kernel function sequence of the replay unit on the CPU resource dimension by adopting a dynamic programming method according to the third target architecture characteristic value of the replay unit on the CPU resource dimension;
a loop sub-unit 1322, configured to instruct the dynamic optimization unit to respectively obtain optimal kernel function combinations corresponding to all the playback units in the load playback log in the space resource dimension and the CPU resource dimension.
The load reduction device provided by the embodiment of the invention can execute the method embodiment, the realization principle and the technical effect are similar, and the detailed description is omitted.
Further, the kernel function analysis obtaining module 10 is specifically configured to determine a load behavior range according to the load behavior information recorded in the public cloud log, and obtain, according to the load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
Optionally, the evaluation requirement of the user includes at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
Fig. 10 is a schematic structural diagram of a fourth embodiment of the load reduction device provided in the embodiment of the present invention, where the device may be implemented by software, or may be implemented by hardware, or by a combination of hardware and software. On the basis of the embodiment shown in fig. 9, as shown in fig. 10, the replay log obtaining module 11 specifically includes:
the log segment selecting unit 111 is configured to select a log segment from the public cloud log according to the evaluation requirement of the user, where the log segment includes a plurality of first playback units, and the first playback units include resource usage rates of each load in each resource dimension in the current first playback unit;
a processing unit 112, configured to perform a summation operation on the resource usage rate of each load in each first playback unit in each resource dimension to obtain a total resource usage rate of each first playback unit in each resource dimension, and perform an average calculation on the architectural feature value of each load in each first playback unit in each resource dimension to obtain a target architectural feature value of each first playback unit in each resource dimension;
a replay log obtaining unit 113, configured to obtain the load replay log according to a total resource usage rate of each of the first replay units in each of the resource dimensions and a target architecture feature value of each of the first replay units in each of the resource dimensions.
The load reduction device provided by the embodiment of the invention can execute the method embodiment, the realization principle and the technical effect are similar, and the detailed description is omitted.
Fig. 11 is a schematic structural diagram of a first load reduction device according to an embodiment of the present invention. As shown in fig. 11, the load reduction device may include a memory 21, a processor 22, and at least one communication bus 23. The communication bus 23 is used to realize communication connection between the elements. The memory 21 may comprise a high-speed RAM memory, and may also include a non-volatile memory NVM, such as at least one disk memory, in which various programs may be stored in the memory 21 for performing various processing functions and implementing the method steps of the present embodiment. The processor 22 may be, for example, a Central Processing Unit (CPU).
The communication bus 23 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus 23 may be one or more physical lines, and when a plurality of physical lines, may be divided into an address bus, a data bus, a control bus, and the like.
In the embodiment of the present invention, the processor 22 is configured to obtain a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log, and obtain a load replay log according to an evaluation requirement of a user and the public cloud log; acquiring a second kernel function set corresponding to each resource dimension of each replay unit from the first kernel function set according to the total resource utilization rate of each replay unit on each resource dimension and the load behavior of each kernel function in the first kernel function set, and acquiring a mixed load for evaluating the system performance according to the second kernel function sets of all replay units in the load replay log on each resource dimension; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions; each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively; the sum of the resource usage of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage in the resource dimension.
Optionally, each playback unit further includes a target architecture feature value of all loads in the current playback unit in each resource dimension;
the processor 22 is specifically configured to determine, according to a second kernel function set corresponding to each playback unit in each resource dimension, a kernel function sequence of each playback unit in each resource dimension, and determine, according to a target architecture feature value of each playback unit in each resource dimension, an optimal kernel function combination of each playback unit in each resource dimension from the kernel function sequence; acquiring a mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension; and the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum.
Further, the processor 22 is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
Optionally, the processor 22 is further configured to, after obtaining the optimal kernel function combination of each playback unit in the spatial resource dimension, obtain load behavior information corresponding to the optimal kernel function combination of each playback unit in the spatial resource dimension in the CPU resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
Optionally, the spatial kernel function set is a memory kernel function set or an IO kernel function set;
the processor 22 is specifically configured to determine a first space capacity of the playback unit according to a total resource utilization rate of the playback unit in a space resource dimension, determine a maximum number of used space kernel functions of each type in the first space capacity according to the first space capacity and a space resource utilization rate of each type of space kernel function in the space kernel function set, and determine a second kernel function set corresponding to the playback unit in the space resource dimension according to the maximum number of used space kernel functions of each type in the first space capacity; and determining a second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource utilization rate of the replay unit in the CPU resource dimension, determining the maximum use number of each CPU kernel function in the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU kernel function in the CPU kernel function set, determining a second kernel function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU kernel function in the second space capacity, and further obtaining the second kernel function set corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension respectively.
Optionally, the processor 22 is specifically configured to determine, according to the first target architectural feature value of the playback unit in the spatial resource dimension, an optimal kernel function combination of the playback unit in the spatial resource dimension from the kernel function sequence of the playback unit in the spatial resource dimension by using a dynamic programming method, determine, according to a CPI value generated by the optimal kernel function combination of the playback unit in the spatial resource dimension during system runtime and the second target architectural feature value of the playback unit in the CPU resource dimension, a third target architectural feature value of the playback unit in the CPU resource dimension, and determine, according to the third target architectural feature value of the playback unit in the CPU resource dimension, the optimal kernel function combination of the playback unit in the CPU resource dimension from the kernel function sequence of the playback unit in the CPU resource dimension by using a dynamic programming method, and further obtaining the optimal kernel function combination of all the replay units in the load replay log, which corresponds to the space resource dimension and the CPU resource dimension respectively.
Further, the processor 22 is specifically configured to determine a load behavior range according to the load behavior information recorded in the public cloud log, and obtain, according to the load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
Optionally, the evaluation requirement of the user includes at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
Further, the processor 22 is specifically configured to select a log segment from the public cloud log according to the evaluation requirement of the user, and perform a summation operation on the resource utilization rate of each load in each first playback unit in each resource dimension to obtain a total resource utilization rate of each first playback unit in each resource dimension; performing average calculation on the architecture characteristic value of each load in each first replay unit on each resource dimension to obtain a target architecture characteristic value of each first replay unit on each resource dimension, and acquiring a load replay log according to the total resource utilization rate of each first replay unit on each resource dimension and the target architecture characteristic value of each first replay unit on each resource dimension; the log segment includes a plurality of first replay units, and the first replay units include resource usage rates of each load in each resource dimension in the current first replay unit.
The load reduction device provided by the embodiment of the present invention may implement the method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (27)

1. A method of reducing a load, comprising:
acquiring a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions;
acquiring a load replay log according to the evaluation requirement of the user and the public cloud log; wherein each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively;
acquiring a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set according to the total resource utilization rate of each playback unit in each resource dimension and the load behavior of each kernel function in the first kernel function set; wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension;
and acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension.
2. The method according to claim 1, wherein each replay unit further comprises a target architectural feature value of all loads in the current replay unit in each resource dimension; the obtaining, according to the second kernel function sets of all the playback units in the load playback log in each resource dimension, a mixed load for evaluating system performance specifically includes:
determining a kernel function sequence of each playback unit on each resource dimension according to a second kernel function set corresponding to each playback unit on each resource dimension;
determining an optimal kernel function combination of each replay unit in each resource dimension from the kernel function sequence according to a target architecture characteristic value of each replay unit in each resource dimension; wherein the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum;
and acquiring the mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension.
3. The method according to claim 2, wherein the determining an optimal kernel function combination of each of the playback units in each of the resource dimensions from the kernel function sequence according to the target architecture feature value of each of the playback units in each of the resource dimensions specifically comprises:
and determining the optimal kernel function combination of each replay unit in each resource dimension from the kernel function sequence by adopting a dynamic programming method according to the target architecture characteristic value of each replay unit in each resource dimension.
4. The method of claim 3, wherein after obtaining the optimal kernel function combination for each of the playback units in the spatial resource dimension, the method further comprises:
acquiring load behavior information of each playback unit corresponding to the optimal kernel function combination in the space resource dimension in the CPU resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
5. The method of claim 4, wherein the set of spatial kernels is a set of memory kernels or a set of IO kernels;
the acquiring, according to a total resource usage rate of each playback unit in each resource dimension and a load behavior of each kernel function in the first kernel function set, a second kernel function set corresponding to each playback unit in each resource dimension from the first kernel function set specifically includes:
step A: determining a first space capacity of the playback unit according to the total resource utilization rate of the playback unit in space resource dimensions, and determining the maximum use number of each space kernel function under the first space capacity according to the first space capacity and the space resource utilization rate of each space kernel function in the space kernel function set;
and B: determining a second kernel function set corresponding to the space resource dimension of the playback unit according to the maximum using number of each space kernel function under the first space capacity;
and C: determining a second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource utilization rate of the replay unit in the CPU resource dimension;
step D: determining the maximum using number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU core function in the CPU core function set;
step E: determining a second kernel function set corresponding to the CPU resource dimension of the playback unit according to the maximum using number of each CPU kernel function under the second space capacity;
step F: and B, respectively executing the step A to the step E on each replay unit in the load replay log to obtain a second kernel function set corresponding to all replay units in the load replay log in a space resource dimension and a CPU resource dimension.
6. The method according to claim 4, wherein the determining an optimal kernel function combination of each playback unit in each resource dimension from the kernel function sequence by using a dynamic programming method according to the target architecture feature value of each playback unit in each resource dimension specifically comprises:
step G: determining the optimal kernel function combination of the replay unit on the space resource dimension from the kernel function sequence of the replay unit on the space resource dimension by adopting a dynamic programming method according to the first target architecture characteristic value of the replay unit on the space resource dimension;
step H: determining a third target architecture characteristic value of the replay unit on the CPU resource dimension according to the optimal kernel function combination of the replay unit on the space resource dimension, the CPI value generated during the system operation and the second target architecture characteristic value of the replay unit on the CPU resource dimension;
step I: according to a third target architecture characteristic value of the replay unit in the CPU resource dimension, determining an optimal kernel function combination of the replay unit in the CPU resource dimension from a kernel function sequence of the replay unit in the CPU resource dimension by adopting a dynamic programming method;
step J: and G to I are respectively executed for each replay unit in the load replay log, and the optimal kernel function combination corresponding to all replay units in the load replay log in the space resource dimension and the CPU resource dimension is obtained.
7. The method according to any one of claims 1 to 6, wherein the obtaining the first kernel function set from the kernel function library according to the load behavior information recorded in the public cloud log specifically includes:
determining a load behavior range according to the load behavior information recorded in the public cloud log;
and acquiring the kernel functions meeting the load behavior range from the kernel function library according to the load behavior and the load behavior range of each kernel function in the kernel function library to form the first kernel function set.
8. The method according to claim 7, wherein the user evaluation requirement comprises at least one of a machine type recorded in the log, the number of machines corresponding to the machine type recorded in the log, and a time period of the log.
9. The method according to claim 2, wherein the obtaining a load replay log according to the user's evaluation requirements and the public cloud log specifically comprises:
selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units, and each first playback unit comprises the resource utilization rate of each load in each current first playback unit in each resource dimension;
summing the resource utilization rate of each load in each first playback unit in each resource dimension to obtain the total resource utilization rate of each first playback unit in each resource dimension;
carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain a target architecture characteristic value of each first playback unit on each resource dimension;
and acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
10. A load reduction device, comprising:
the kernel function analysis and acquisition module is used for acquiring a first kernel function set from a kernel function library according to load behavior information recorded in a public cloud log; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions;
the replay log obtaining module is used for obtaining a load replay log according to the evaluation requirement of the user and the public cloud log; wherein each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively;
a kernel function matching module, configured to obtain, from the first kernel function set, a second kernel function set corresponding to each playback unit in each resource dimension according to a total resource usage rate of each playback unit in each resource dimension and a load behavior of each kernel function in the first kernel function set; wherein the sum of the resource usage rates of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage rate in the resource dimension;
and the load restoration module is used for acquiring the mixed load for evaluating the system performance according to the second kernel function set of all the replay units in the load replay log on each resource dimension.
11. The apparatus according to claim 10, wherein each of the replay units further comprises a target architectural feature value of all loads in the current replay unit in each resource dimension; the load reduction module specifically comprises:
a sequence determining unit, configured to determine, according to a second kernel function set corresponding to each of the playback units in each of the resource dimensions, a kernel function sequence of each of the playback units in each of the resource dimensions;
a combination optimization unit, configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension according to a target architecture feature value of each playback unit in each resource dimension; wherein the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum;
and the load restoration unit is used for acquiring the mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension.
12. The apparatus according to claim 11, wherein the combination optimization unit is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination of each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
13. The apparatus of claim 12, further comprising:
an obtaining module, configured to obtain load behavior information of the optimal kernel function combination of each playback unit in the space resource dimension, corresponding to the CPU resource dimension, after the combination optimization unit obtains the optimal kernel function combination of each playback unit in the space resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
14. The apparatus of claim 13, wherein the set of spatial kernels is a set of memory kernels or a set of IO kernels; the kernel function matching module specifically includes:
a determining unit, configured to determine a first space capacity of the playback unit according to a total resource usage rate of the playback unit in a space resource dimension, and determine a maximum number of used space kernel functions in the first space capacity according to the first space capacity and a space resource usage rate of each space kernel function in the space kernel function set; determining a second kernel function set corresponding to the replay unit in the space resource dimension according to the maximum use number of each space kernel function under the first space capacity, and determining the second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource use rate of the replay unit in the CPU resource dimension; determining the maximum use number of each CPU core function under the second space capacity according to the second space capacity and the CPU resource use rate of each CPU core function in the CPU core function set, and determining a second core function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU core function under the second space capacity;
and the circulation unit is used for instructing the determining unit to acquire the second kernel function sets corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension respectively.
15. The apparatus according to claim 13, wherein the combinatorial optimization unit specifically includes:
the dynamic optimization subunit is used for determining an optimal kernel function combination of the playback unit in the space resource dimension from the kernel function sequence of the playback unit in the space resource dimension by adopting a dynamic programming method according to a first target architecture characteristic value of the playback unit in the space resource dimension, and determining a third target architecture characteristic value of the playback unit in the CPU resource dimension according to a CPI value generated by the optimal kernel function combination of the playback unit in the space resource dimension during system operation and a second target architecture characteristic value of the playback unit in the CPU resource dimension; determining the optimal kernel function combination of the replay unit on the CPU resource dimension from the kernel function sequence of the replay unit on the CPU resource dimension by adopting a dynamic programming method according to the third target architecture characteristic value of the replay unit on the CPU resource dimension;
and the circulation subunit is used for indicating the dynamic optimization subunit to respectively acquire the optimal kernel function combinations corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension.
16. The apparatus according to any one of claims 10 to 15, wherein the kernel function analysis obtaining module is specifically configured to determine a load behavior range according to load behavior information recorded in the public cloud log, and obtain, according to a load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
17. The apparatus of claim 16,
the user evaluation requirement comprises at least one of the machine type recorded in the log, the number of machines corresponding to the machine type recorded in the log and the time period of the log.
18. The apparatus according to claim 11, wherein the replay log obtaining module specifically includes:
the log segment selecting unit is used for selecting a log segment from the public cloud log according to the evaluation requirement of the user, wherein the log segment comprises a plurality of first playback units, and each first playback unit comprises the resource utilization rate of each load in each current first playback unit in each resource dimension;
the processing unit is used for summing the resource utilization rate of each load in each first playback unit on each resource dimension to obtain the total resource utilization rate of each first playback unit on each resource dimension, and carrying out average calculation on the architecture characteristic value of each load in each first playback unit on each resource dimension to obtain the target architecture characteristic value of each first playback unit on each resource dimension;
and the replay log acquisition unit is used for acquiring the load replay log according to the total resource utilization rate of each first replay unit in each resource dimension and the target architecture characteristic value of each first replay unit in each resource dimension.
19. A load reduction device, comprising:
the processor is used for acquiring a first kernel function set from a kernel function library according to load behavior information recorded in the public cloud log, and acquiring a load replay log according to the evaluation requirement of a user and the public cloud log; acquiring a second kernel function set corresponding to each resource dimension of each replay unit from the first kernel function set according to the total resource utilization rate of each replay unit on each resource dimension and the load behavior of each kernel function in the first kernel function set, and acquiring a mixed load for evaluating the system performance according to the second kernel function sets of all replay units in the load replay log on each resource dimension; the first kernel function set comprises a CPU kernel function set and a space kernel function set; the CPU kernel function set comprises a plurality of CPU kernel functions, the space kernel function set comprises a memory kernel function set and/or an IO kernel function set, the memory kernel function set comprises a plurality of memory kernel functions, and the IO kernel function set comprises a plurality of IO kernel functions; each replay unit in the load replay log comprises the total resource utilization rate of all loads in the current replay unit in each resource dimension respectively; the sum of the resource usage of the same kernel function in the second set of kernel functions in the resource dimension does not exceed the total resource usage in the resource dimension.
20. The apparatus of claim 19, wherein each of the replay units further comprises a target architectural feature value of all loads in the current replay unit in each resource dimension;
the processor is specifically configured to determine, according to a second kernel function set corresponding to each playback unit in each resource dimension, a kernel function sequence of each playback unit in each resource dimension, and determine, according to a target architecture feature value of each playback unit in each resource dimension, an optimal kernel function combination of each playback unit in each resource dimension from the kernel function sequence; acquiring a mixed load for evaluating the system performance according to the optimal kernel function combination of all the replay units in the load replay log on each resource dimension; and the error between the architecture characteristic value generated by all the kernel functions in the optimal kernel function combination when the system runs and the target architecture characteristic value on the resource dimension is minimum.
21. The apparatus according to claim 20, wherein the processor is specifically configured to determine, from the kernel function sequence, an optimal kernel function combination for each playback unit in each resource dimension by using a dynamic programming method according to a target architecture feature value of each playback unit in each resource dimension.
22. The device of claim 21, wherein the processor is further configured to, after obtaining the optimal kernel function combination of each of the playback units in the spatial resource dimension, obtain load behavior information corresponding to the optimal kernel function combination of each of the playback units in the spatial resource dimension in the CPU resource dimension; the space resource dimension comprises a memory resource dimension and/or an IO resource dimension, and the load behavior information of the optimal kernel function combination of the playback unit on the space resource dimension, which corresponds to the CPU resource dimension, comprises the CPU space capacity required to be occupied and the value of the generated load calculation characteristic CPI when the optimal kernel function combination runs on the system.
23. The apparatus of claim 22, wherein the set of spatial kernels is a set of memory kernels or a set of IO kernels;
the processor is specifically configured to determine a first space capacity of the playback unit according to a total resource utilization rate of the playback unit in a space resource dimension, determine a maximum number of used spatial kernel functions of each type in the first space capacity according to the first space capacity and a space resource utilization rate of each type of spatial kernel function in the spatial kernel function set, and determine a second kernel function set corresponding to the playback unit in the space resource dimension according to the maximum number of used spatial kernel functions of each type in the first space capacity; and determining a second space capacity of the replay unit according to the CPU space capacity required to be occupied by the optimal kernel function combination of the replay unit in the space resource dimension during system operation and the total resource utilization rate of the replay unit in the CPU resource dimension, determining the maximum use number of each CPU kernel function in the second space capacity according to the second space capacity and the CPU resource utilization rate of each CPU kernel function in the CPU kernel function set, determining a second kernel function set corresponding to the replay unit in the CPU resource dimension according to the maximum use number of each CPU kernel function in the second space capacity, and further obtaining the second kernel function set corresponding to all the replay units in the load replay log in the space resource dimension and the CPU resource dimension respectively.
24. The apparatus of claim 22,
the processor is specifically configured to determine, according to a first target architectural feature value of the playback unit in a spatial resource dimension, an optimal kernel function combination of the playback unit in the spatial resource dimension from a kernel function sequence of the playback unit in the spatial resource dimension by using a dynamic programming method, determine, according to a CPI value generated by the optimal kernel function combination of the playback unit in the spatial resource dimension during system operation and a second target architectural feature value of the playback unit in the CPU resource dimension, a third target architectural feature value of the playback unit in the CPU resource dimension, and determine, according to the third target architectural feature value of the playback unit in the CPU resource dimension, an optimal kernel function combination of the playback unit in the CPU resource dimension from the kernel function sequence of the playback unit in the CPU resource dimension by using a dynamic programming method, and further obtaining the optimal kernel function combination of all the replay units in the load replay log, which corresponds to the space resource dimension and the CPU resource dimension respectively.
25. The device according to any one of claims 19 to 24, wherein the processor is specifically configured to determine a load behavior range according to load behavior information recorded in the public cloud log, and obtain, according to the load behavior of each kernel function in the kernel function library and the load behavior range, a kernel function that satisfies the load behavior range from the kernel function library to form the first kernel function set.
26. The apparatus according to claim 25, wherein the user's evaluation requirements include at least one of a machine type recorded in the log, a number of machines corresponding to the machine type recorded in the log, and a time period of the log.
27. The apparatus of claim 26,
the processor is specifically configured to select a log segment from the public cloud log according to the evaluation requirement of the user, and perform summation operation on the resource utilization rate of each load in each first playback unit in each resource dimension to obtain the total resource utilization rate of each first playback unit in each resource dimension; performing average calculation on the architecture characteristic value of each load in each first replay unit on each resource dimension to obtain a target architecture characteristic value of each first replay unit on each resource dimension, and acquiring a load replay log according to the total resource utilization rate of each first replay unit on each resource dimension and the target architecture characteristic value of each first replay unit on each resource dimension; the log segment includes a plurality of first replay units, and the first replay units include resource usage rates of each load in each resource dimension in the current first replay unit.
CN201610377787.2A 2016-05-31 2016-05-31 Load reduction method, device and equipment Active CN107450968B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610377787.2A CN107450968B (en) 2016-05-31 2016-05-31 Load reduction method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610377787.2A CN107450968B (en) 2016-05-31 2016-05-31 Load reduction method, device and equipment

Publications (2)

Publication Number Publication Date
CN107450968A CN107450968A (en) 2017-12-08
CN107450968B true CN107450968B (en) 2020-09-08

Family

ID=60485968

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610377787.2A Active CN107450968B (en) 2016-05-31 2016-05-31 Load reduction method, device and equipment

Country Status (1)

Country Link
CN (1) CN107450968B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112486738B (en) * 2019-09-12 2022-04-26 阿里巴巴集团控股有限公司 Load testing method and device, electronic equipment and computer readable storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101800771A (en) * 2010-01-29 2010-08-11 北京理工大学 Copy selection method based on kernel density estimation
CN102576314A (en) * 2009-07-27 2012-07-11 先进微装置公司 Mapping processing logic having data parallel threads across processors
CN102681889A (en) * 2012-04-27 2012-09-19 电子科技大学 Scheduling method of cloud computing open platform
CN104065663A (en) * 2014-07-01 2014-09-24 复旦大学 Auto-expanding/shrinking cost-optimized content distribution service method based on hybrid cloud scheduling model
CN105487930A (en) * 2015-12-01 2016-04-13 中国电子科技集团公司第二十八研究所 Task optimization scheduling method based on Hadoop
CN105487927A (en) * 2014-09-15 2016-04-13 华为技术有限公司 Resource management method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9715723B2 (en) * 2012-04-19 2017-07-25 Applied Materials Israel Ltd Optimization of unknown defect rejection for automatic defect classification

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102576314A (en) * 2009-07-27 2012-07-11 先进微装置公司 Mapping processing logic having data parallel threads across processors
CN101800771A (en) * 2010-01-29 2010-08-11 北京理工大学 Copy selection method based on kernel density estimation
CN102681889A (en) * 2012-04-27 2012-09-19 电子科技大学 Scheduling method of cloud computing open platform
CN104065663A (en) * 2014-07-01 2014-09-24 复旦大学 Auto-expanding/shrinking cost-optimized content distribution service method based on hybrid cloud scheduling model
CN105487927A (en) * 2014-09-15 2016-04-13 华为技术有限公司 Resource management method and device
CN105487930A (en) * 2015-12-01 2016-04-13 中国电子科技集团公司第二十八研究所 Task optimization scheduling method based on Hadoop

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Investigation on extended black-start schemes of power system considering reasonable load restoration;xuping gu等;《2014 IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC)》;20141210;第1-5页 *

Also Published As

Publication number Publication date
CN107450968A (en) 2017-12-08

Similar Documents

Publication Publication Date Title
US9684562B2 (en) Automatic serial starting of resource groups on failover based on the prediction of aggregate resource usage
WO2012033909A2 (en) Method and system of simulating a data center
US9460032B2 (en) Apparatus and method for processing an interrupt
CN103092751A (en) Web application performance test system based on customer behavior model in cloud environment
CN109359118B (en) Data writing method and device
JP2017506400A (en) Cloud release pipeline diagnosis and optimization
US11372594B2 (en) Method and apparatus for scheduling memory access request, device and storage medium
JP6382284B2 (en) Data flow programming of computing devices with graph partitioning based on vector estimation
US20110016455A1 (en) Power Profiling for Embedded System Design
KR20180011096A (en) System and method for determining concurrent execution arguments for dispatch sizes of parallel processor kernels
US11663505B2 (en) Estimating performance and required resources from shift-left analysis
CN111026493A (en) Interface rendering processing method and device
CN107450968B (en) Load reduction method, device and equipment
US20150052328A1 (en) User-controlled paging
JP2018508865A (en) Application event tracking
US10089151B2 (en) Apparatus, method, and program medium for parallel-processing parameter determination
US8332461B2 (en) Task migration system and method thereof
US9769025B2 (en) Predicting the performance of a multi-stage communications network under load from multiple communicating servers
US11340952B2 (en) Function performance trigger
CN110704222A (en) Dump file analysis method and device, storage medium and electronic equipment
CN115687159B (en) Debugging method, debugging device and computer readable storage medium
JP5545133B2 (en) Static analysis processing system, method, and program
CN111597047A (en) Service deployment method, device, electronic equipment and storage medium
KR20190071575A (en) Method for data center storage evaluation framework simulation
CN110795215A (en) Data processing method, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant