WO2021051529A1 - Method, apparatus and device for estimating cloud host resources, and storage medium - Google Patents

Method, apparatus and device for estimating cloud host resources, and storage medium Download PDF

Info

Publication number
WO2021051529A1
WO2021051529A1 PCT/CN2019/117048 CN2019117048W WO2021051529A1 WO 2021051529 A1 WO2021051529 A1 WO 2021051529A1 CN 2019117048 W CN2019117048 W CN 2019117048W WO 2021051529 A1 WO2021051529 A1 WO 2021051529A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
information
resource
sampling
evaluated
Prior art date
Application number
PCT/CN2019/117048
Other languages
French (fr)
Chinese (zh)
Inventor
徐锐杰
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2021051529A1 publication Critical patent/WO2021051529A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/22Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2205Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
    • G06F11/2236Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested to test CPU or processors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/22Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2268Logging of test results
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/22Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2273Test methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/22Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2289Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing by configuration test
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/22Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26Functional testing

Definitions

  • This application relates to the field of cloud monitoring, in particular to methods, devices, equipment, and storage media for evaluating cloud host resources.
  • cloud hosting integrates high-performance servers and high-quality network bandwidth, effectively solving the shortcomings of traditional hosting rental prices and uneven service quality, and can fully meet the needs of small and medium-sized enterprises and individual webmaster users.
  • Host rental services require low cost, high reliability, and easy management. Therefore, cloud hosting is widely used.
  • the unreasonable use and waste of cloud host resources such as the cloud host to which the resigned personnel belong, may not be involved in the resignation process due to possible project associations. It has not been released, resulting in many servers with extremely low resource usage but equipped with high-configuration cloud hosts, which in turn causes a waste of cloud host resources.
  • the resource usage information is analyzed and evaluated to obtain the evaluation information, the usage status of the cloud host is determined according to the evaluation information, and the usage status is determined according to the usage status Optimize the configuration plan.
  • the resource usage of the cloud host will increase with the increase in the number of users and usage time, the number of users and usage time will affect the resource usage of the cloud host.
  • the inventor realizes that in this case, if you continue to use the cloud host The current resource usage analysis and evaluation plan will result in the single analysis and evaluation of resource usage information, resulting in the determined optimization configuration strategy not being rigorous and comprehensive, resulting in low resource utilization of the cloud host.
  • This application provides a method, device, equipment, and storage medium for evaluating cloud host resources, which can solve the problem of low resource utilization of cloud hosts.
  • this application provides a method for evaluating cloud host resources, including: obtaining training data, inputting the training data to a neural network model, and training the neural network model to obtain a resource monitoring model, wherein ,
  • the training data includes resource sampling data of multiple cloud hosts for multiple periods; acquiring the input target project information, and analyzing the target project information to obtain the project data type and project operation of the target project information Demand information; obtain the resource type and resource usage of the cloud host, and determine the cloud host to be evaluated and the resource usage condition according to the resource usage type, the resource usage condition, the project data type, and the project operation requirement information
  • the resource to be evaluated of the cloud host to be evaluated; obtaining the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and obtaining the second sampling data of the resource to be evaluated in the second time period, Wherein, the start time of the first time period is later than the end time of the second time period; the cloud host to be evaluated is predicted according to the first sampling data and the second sampling data to obtain the first A prediction data
  • the present application provides an apparatus for evaluating cloud host resources, including: an input and output module for obtaining training data for obtaining input target item information; and a processing module for converting the input and output module
  • the acquired training data is input to a neural network model, and the neural network model is trained to obtain a resource monitoring model, wherein the training data includes resource sampling data of multiple cloud hosts for multiple periods; and
  • the input target project information obtained by the input and output module is analyzed to obtain the project data type and project operation requirement information of the target project information; the used resource type and the used resource situation of the cloud host are obtained, and according to the used resource type,
  • the resource usage, the project data type, and the project operation requirement information are used to determine the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed; and the resource to be assessed is acquired through the resource monitoring model.
  • the evaluation information and the optimal configuration strategy; the display module is used to display the evaluation information and the optimal configuration strategy.
  • the present application provides a computer device, which includes at least one connected processor, a memory, a display, and an input and output unit, wherein the memory is used to store program code, and the processor is used to call the memory To execute the method described in the first aspect above.
  • the present application provides a computer-readable storage medium having computer instructions stored in the computer-readable storage medium.
  • the computer instructions run on a computer, the computer executes the above-mentioned first aspect. method.
  • the resource monitoring model obtained through training analyzes the target project information, the types of resources used and the resource usage conditions of multiple cloud hosts to obtain the cloud host to be evaluated and the cloud host to be evaluated. Evaluate resources, obtain first sampling data and second sampling data of the resource to be evaluated, obtain prediction information according to the first sampling data and the second sampling data, and obtain prediction information according to the first sampling data and the first sampling data. Second, the sampling data and the prediction information obtain evaluation evaluation information, and based on the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated.
  • the cloud host to be evaluated and the resources to be evaluated of the cloud host to be evaluated are determined based on the target project information, so that the evaluation requirements for the target project information can be met on the basis of the resource usage evaluation of the cloud host, Thereby improving the resource utilization of the cloud host; by evaluating the first sampling data, second sampling data and forecast information of the cloud host, the evaluated data information is multi-faceted, so that the evaluation results are more rigorous and accurate.
  • FIG. 1 is a schematic flowchart of a method for evaluating cloud host resources in an embodiment of this application
  • FIG. 2 is a schematic structural diagram of an apparatus for evaluating cloud host resources in an embodiment of the application
  • FIG. 3 is a schematic structural diagram of a computer device in an embodiment of the application.
  • This application provides a method, device, device, and storage medium for evaluating cloud host resources, which can be used for the configuration and use of cloud hosts, for the release of cloud hosts that do not use resources, and for those who need to perform configuration reduction operations due to minimal resource usage.
  • Cloud hosting provides reference. Among them, for cloud computing platform vendors, it can be used to provide quantitative standards for resource recovery to provide reference support for reducing the waste of internal computing resources; for cloud computing platform tenants, it can be used to provide information on the resource usage and rational use of business cloud hosts reference.
  • this application mainly provides the following technical solutions:
  • the method is executed by a computer device.
  • the computer device may be a server or a terminal.
  • the terminal is a terminal on which the device 20 shown in FIG. 2 is installed. This application does not limit the type of execution subject.
  • the method includes:
  • the training data includes resource sampling data of multiple cloud hosts for multiple periods; the training data is data of multiple sampling points that are sampled at a preset sampling interval and arranged in a forward time sequence, including the first data in the first period And the second data in the second time period, the start time of the first time period is later than the end time of the second time period.
  • the existing monitoring model can be trained through a migration learning algorithm to obtain a resource monitoring model for monitoring, evaluating, and optimizing the configuration of the resources used by the cloud host.
  • the resource monitoring model in this embodiment is a backward propagation neural network model.
  • the neural network By combining the nonlinear mapping of the input and output of the backward propagation neural network model, performing gradient descent calculations, having a certain generalization ability and the characteristics of using different transfer functions, on the one hand, it enables it to monitor, evaluate and evaluate the use of cloud host resources.
  • Data mining such as optimized configuration for multi-dimensional feature construction, and when new data enters the neural network model network for training, the neural network can adjust the weight to adapt to more data; on the other hand, make its output value But any value, and reduce the error of the resource monitoring model, in order to obtain better evaluation information and optimize the allocation strategy.
  • the above-mentioned acquiring training data, inputting the training data to the neural network model, and training the neural network model to obtain the resource monitoring model include: acquiring training data, Perform data preprocessing; store the smoothed and abnormal data processed training data in the training database, and set up configuration files, where the configuration files include network structure, training duration, training and test ratio arrangements, output content, and optimized learning Rate setting, optimization parameters and archiving rules setting; according to the configuration file, the training data is smoothed to obtain the prediction information; the prediction information is evaluated according to the preset comprehensive evaluation rules to obtain the evaluation information, where the evaluation The information includes the scoring range and analysis information; according to the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated to obtain the resource monitoring model; the resource monitoring model is tested for accuracy and performance through the created test script; if the accuracy of the test results When the first preset threshold is reached and the result of the performance test reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model; if
  • Training the neural network model so that the acquired resource monitoring model has the ability to analyze the target project information, the resource type and resource usage of multiple cloud hosts, to obtain the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed , Obtain the first sampling data and the second sampling data of the resource to be evaluated, obtain the prediction information according to the first sampling data and the second sampling data, and obtain the prediction information according to the first sampling data and the second sampling data.
  • the data and the prediction information obtain the evaluation evaluation information, and the function of generating an optimized configuration strategy corresponding to the evaluation information according to the evaluation information, so as to better evaluate the resource utilization of the cloud host, thereby improving the resource utilization of the cloud host rate.
  • the resource monitoring model includes a data processing sub-model, a prediction sub-model, an evaluation sub-model, and a strategy generation sub-model.
  • the prediction sub-model and the data processing sub-model are connected in series, and the evaluation sub-model and prediction are connected in series.
  • the sub-models are connected in series, and the strategy generation sub-models are connected in series with the evaluation sub-models.
  • the data processing sub-model is used to smooth the sampled data obtained by the resource monitoring model and abnormal data processing; the prediction sub-model is used to predict the sampled data output by the data processing sub-model to obtain prediction information; the evaluation sub-model is used to predict The prediction information output by the sub-model is evaluated to obtain the evaluation information; the strategy generation sub-model is used to generate the corresponding optimal configuration strategy according to the evaluation information output by the evaluation sub-model.
  • the target project information is analyzed to obtain the data type of the operation required to complete the target project and the operation requirement information of the target project.
  • the project operation requirement information includes, but is not limited to, the planned time for project completion, deadlines, and project completion functions.
  • the target project is a management system, by analyzing the target project information input by the user, to obtain the planned time, deadlines, amount of cloud host resources required for the completion of the management system research and development, the performance of the management system and the requirements of the management system Function and so on.
  • the above-mentioned obtaining the input target project information and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information includes: creating a project operation The demand table, where the project operation demand table includes the preset completion time of the project, the host resource demand and the optimal host resource allocation corresponding to the host resource demand; obtain the target project information input by the user, and perform data on the target project information Preprocessing, where data preprocessing includes missing value filling processing, denoising processing, and data standardization processing; divide the target item information after data preprocessing into N groups, and perform multiple iterations on the target item information divided into N groups Regroup to obtain the optimal grouping plan; obtain the project data type of each group of target project information in the optimal grouping plan; analyze the target project information to obtain the fourth key information; traverse the project operation requirement table according to the fourth key information, To obtain the project operation requirement information corresponding to the fourth key information.
  • the resource monitoring model can intuitively and easily obtain the project data type and project operation requirement information of the target project.
  • the project operation requirement information can be obtained from multiple angles, so that the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed can be determined more quickly and accurately.
  • cloud hosts include all cloud hosts used by connections; resources to be evaluated include resource usage and remaining resources, and the remaining resources are the total resources of the cloud hosts to be evaluated minus used resources.
  • Analyze the resource types and resource usage conditions of all cloud hosts match the resource usage types corresponding to the project data type to obtain the first cloud host, and match the resource usage conditions corresponding to the project operation requirement information to obtain The second cloud host, judges the cloud host that meets the project data type and meets the project operation demand information among the first cloud host and the second cloud host, as the cloud host to be evaluated, and obtains the resource usage and the resource usage of the cloud host to be evaluated The remaining resources.
  • Both the first sampling data and the second sampling data are data of multiple sampling points that are sampled at a preset sampling interval and arranged in a forward time sequence.
  • Both the first sampling data and the second sampling data include the occupancy rate information of the central processing unit CPU , Memory occupancy information and input/output IO occupancy information.
  • the occupancy information includes the proportion of physical resources and total resources, the proportion of functionally described information resources and total resources, and the non-functionally described information resources and total resources.
  • the aforementioned resource to be evaluated includes a first task in a first period and a second task in a second period.
  • the aforementioned resource monitoring model is used to obtain the resource to be evaluated.
  • the first sampling data in the first time period and obtaining the second sampling data of the resource to be evaluated in the second time period include: obtaining the first occupancy rate information and the first priority of the first task through the resource monitoring model, And obtain the second occupancy rate information and the second priority of the second task; classify the first task according to the first priority, and identify the first category label, and perform the second task according to the second priority Classify and identify the second category label; according to the first occupancy rate information, classify the first task that identifies the first category label, and identify the third category label, and the second occupancy rate information, to identify the second category label Classify the second task and identify the fourth category label; determine whether the first priority meets the first preset sampling condition and/or whether the first occupancy rate information meets the second preset sampling condition, and determine whether the second priority Whether the third prese
  • the prediction includes the prediction of the resource usage status and the trend of each resource usage of the cloud host to be evaluated in the third period, and the start time of the third period is later than the end time of the first period.
  • the first prediction data includes the prediction data obtained by predicting the usage status of each resource and the trend of each resource usage of the cloud host in the third period based on the first sampling data
  • the second prediction data includes the prediction data to be evaluated based on the second sampling data.
  • the third prediction data includes the cloud host to be evaluated in the third period based on the first sampling data and the second sampling data.
  • the prediction data obtained by predicting the usage status of each resource and the trend of each resource usage, the start time of the third period is later than the end time of the first period.
  • the main information and auxiliary information refer to the processing of the main information first, and then combined with the auxiliary information for processing.
  • the method further includes: acquiring first time series data of the forecast information; performing sliding window processing on the first time series data to generate a preset number The time series subsequence with a preset length; analyze the statistical indicators of the time series subsequence to obtain statistical characteristic information, and use the statistical characteristic information as the updated forecast information, where the statistical characteristic information includes the maximum value, the minimum value, and the median , First quartile, third quartile, variance and standard deviation.
  • the data of the forecast information is more concentrated, more systematic, and more clearly reflecting the objective reality, so that the subsequent future forecast values are more biased towards the actual value, and the accuracy of the forecast is promoted.
  • the method before predicting the cloud host to be evaluated based on the first sampling data and the second sampling data, the method further includes: acquiring second time series data of the first sampling data, and Obtain the third time series data of the second sampling data; use the exponentially weighted moving average method EWMA in the data processing sub-model to evaluate the second time series data and the third time series data respectively to obtain the first smoothed data
  • the first smoothing data, the second time series data and the third time series data are evaluated as follows: Among them, x t is the actual second time series data or the actual third time series data at time t, the coefficient ⁇ is the rate of weighted decline, and V t is the EWMA value at time t; through the Mahalanobis distance in the data processing sub-model
  • the algorithm detects and identifies the extreme abnormal points in the first smoothing data and the second smoothing data respectively, and deletes the marked extreme abnormal points.
  • the calculation for detecting and identifying the extreme abnormal points is as follows: among them, Is
  • Smooth processing and abnormal data processing are performed on the first sampled data and the second sampled data to obtain relatively balanced and stable time series data, reduce errors, and provide further support for the subsequent accurate values of future predicted values.
  • the analysis includes the identification and acquisition of the type, use time, operation rate, and resource proportion of each resource of the first sampling data, the second sampling data, and the prediction information.
  • the first key information includes the type, usage time, operation rate, and resource proportion of each resource obtained by analyzing the first sampled data
  • the second key information includes the type of each resource obtained by analyzing the second sampled data, Use time, operation rate, and resource proportion.
  • the third key information includes the type, use time, operation rate, and resource proportion of each resource obtained by analyzing the forecast information.
  • the method includes a resource database.
  • the resource database includes collected resource data corresponding to multiple data types.
  • the first alternative resource of the resource to be evaluated in the first sampling data is obtained as described above.
  • acquiring the second alternative resource of the resource to be evaluated in the second sampling data including: acquiring first characteristic information of the resource to be evaluated in the first sampling data, and acquiring second characteristic information of the resource to be evaluated in the second sampling data , Where the first feature information and the second feature information both include the data type, the total capacity of the resource, the usage proportion of the resource corresponding to each data type, the performance and characteristics of the resource; the first feature information is obtained from the resource database according to the first feature information.
  • the first resource data corresponding to one feature information, and the second resource data corresponding to the second feature information is obtained in the resource database according to the second feature information; the first resource data is calculated and filtered by preset replacement conditions to obtain the first resource data A replaceable resource, and the second resource data is calculated and filtered through preset replacement conditions to obtain the second replaceable resource; based on the first replaceable resource, analysis and matching are performed in cloud hosts other than the cloud host to be evaluated , To obtain the first alternative cloud host, and obtain the first resource usage information of the first alternative cloud host, and mark the first alternative cloud host and the first resource usage information on the first alternative resource to obtain The final first alternative resource, and according to the second alternative resource, perform analysis and matching among cloud hosts other than the cloud host to be evaluated to obtain the second alternative cloud host, and obtain the second alternative cloud host Second resource usage information, and mark the second replaceable cloud host and the second resource usage information on the second replaceable resource to obtain the final second replaceable resource.
  • the current use resources of the cloud host to be evaluated and the estimated use resources for a certain period of time in the future are evaluated, and based on the first alternative resource and the second alternative resource
  • the evaluation can replace other cloud hosts of the cloud host to be evaluated in order to rationally use the cloud host and improve the resource utilization rate of the cloud host.
  • a cloud host is a cloud host to be evaluated
  • a cloud host B is a cloud host other than the cloud host to be evaluated.
  • the available resources of the A Cloud host are 10% left, and the indicators do not meet the requirements of continued use and the requirements of the project.
  • the resource type of B cloud host is the same and/or similar to A cloud host, which can be replaced, and all indicators meet the requirements of continued use and the requirements of the project, then the input goals are Project information, Cloud B host can be used as a replaceable cloud host for Cloud A host.
  • the process information and result information generated by the analysis process are used as evaluation information.
  • the evaluation information According to the evaluation information, generate an optimized configuration strategy corresponding to the evaluation information, and output the evaluation information and optimized configuration strategy.
  • the optimal configuration strategy includes the resource usage status of the cloud host, the occupancy rate information of each physical resource of the cloud host, whether the cloud host needs to be configured to upgrade or reduce or release resources, and the cloud host's preset resources for each time period in the future Usage, user behavior matching, and semantic matching.
  • the foregoing describes a method for evaluating cloud host resources in the present application, and the following describes an apparatus for executing the foregoing method for evaluating cloud host resources.
  • FIG. 2 is a schematic structural diagram of a device 20 for evaluating cloud host resources, which can be applied to the configuration and use of cloud hosts, for the release of cloud hosts that do not use resources, and the need to perform due to the minimal use of resources.
  • the cloud host that reduces the configuration operation provides a reference.
  • it can be used to provide quantitative standards for resource recovery to provide reference support for reducing the waste of internal computing resources; for cloud computing platform tenants, it can be used to provide information on the resource usage and rational use of business cloud hosts reference.
  • the apparatus 20 in the embodiment of the present application can implement the method corresponding to the embodiment corresponding to FIG. 1 or any one of the optional embodiments or the optional implementation manners in the embodiment corresponding to FIG. 1 for evaluating cloud host resources.
  • the functions implemented by the device 20 can be implemented by hardware, or can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the above-mentioned functions, and the modules may be software and/or hardware.
  • the device 20 may include an input/output module 201, a processing module 202, and a display module 203.
  • the functional realization of the input/output module 201, the processing module 202, and the display module 203 may refer to the embodiment corresponding to FIG. 1 or the implementation corresponding to FIG. 1
  • the operations performed in any optional embodiment or optional implementation manner in the example will not be repeated here.
  • the processing module 202 can be used to control the receiving and sending operations of the input/output module 201, and the display module 203 can be used to display the processing operations of the processing module 202.
  • the input and output module 201 is used to obtain training data and is used to obtain input target item information; the processing module 202 is used to input the training data obtained by the input and output module 201 into the neural network model, and the neural network model Train to obtain the resource monitoring model; analyze the target project information obtained by the input and output module 201 to obtain the project data type and project operation requirement information of the target project information; obtain the resource type and resource usage of the cloud host, and Determine the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed according to the resource usage type, the resource usage situation, the project data type, and the project operation requirement information; obtain through the resource monitoring model
  • the first sampling data of the resource to be evaluated in the first time period, and the second sampling data of the resource to be evaluated in the second time period are acquired;
  • the cloud host to be evaluated performs prediction to obtain first prediction data, second prediction data, and third prediction data, respectively, using the third prediction data as main information, and combining the first prediction data and the first prediction data
  • the second prediction data is used as auxiliary
  • Three key information and acquiring the first alternative resource of the resource to be evaluated in the first sampling data, and acquiring the second alternative resource of the resource to be evaluated in the second sampling data; according to the first key information, The second key information, the third key information, the first replaceable resource, and the second replaceable resource evaluate the cloud host to be evaluated to obtain evaluation information; generate and evaluate based on the evaluation information
  • the optimized configuration strategy corresponding to the information, the evaluation information and the optimized configuration strategy are sent to the display module 203, and the evaluation information and the optimized configuration strategy are output through the display module 203; the display module 203 is used to display the evaluation information and the optimized configuration strategy.
  • the training data includes resource sampling data of multiple cloud hosts in multiple periods; the start time of the first period is later than the end time of the second period; the prediction includes the resource usage status of the cloud host to be evaluated in the third period and For the prediction of the trend of each resource usage, the start time of the third time period is later than the end time of the first time period.
  • the aforementioned processing module 202 is further configured to: create a project operation demand table, where the project operation demand table includes the preset completion time of the project, the host resource demand, and the optimal host resource allocation corresponding to the host resource demand; Obtain the target project information input by the user, and perform data preprocessing on the target project information.
  • the data preprocessing includes missing value filling processing, denoising processing and data standardization processing; the target project information after data preprocessing is divided into N groups, The target project information divided into N groups is regrouped many times by iterative method to obtain the optimal grouping plan; the project data type of each group of target project information in the optimal grouping plan is obtained; the target project information is analyzed to obtain the first Four key information: traverse the project operation demand table according to the fourth key information to obtain the project operation demand information corresponding to the fourth key information.
  • the above-mentioned processing module 202 is further configured to: obtain the first occupancy rate information and the first priority in the first task, and obtain the second occupancy rate information and the second priority in the second task through the resource monitoring model According to the first priority, the first task is classified and the first category label is identified, and the second task is classified according to the second priority, and the second category label is identified; according to the first occupancy rate information, Classify the first task that identifies the first category label, identify the third category label, and second occupancy rate information, classify the second task that identifies the second category label, and identify the fourth category label; determine the first Whether the priority meets the first preset sampling condition and/or whether the first occupancy rate information meets the second preset sampling condition, and whether the second priority meets the third preset sampling condition and/or whether the second occupancy rate information Meet the fourth preset sampling condition; if the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling condition, then according to the preset first sampling frequency, the Set the sampling condition and/or the
  • processing module 202 is executed to obtain the prediction information, it is further used to: obtain the first time series data of the prediction information; perform sliding window processing on the first time series data to generate a preset number of preset lengths.
  • the time series subsequence analyze the statistical indicators of the time series subsequence to obtain statistical feature information, and use the statistical feature information as the updated forecast information, where the statistical feature information includes the maximum value, the minimum value, the median, and the first quartile Digits, third quartile, variance and standard deviation.
  • the processing module 202 described above is further configured to: obtain the second time series data of the first sampled data, and obtain the second sampled data before performing the prediction based on the first sampled data and the second sampled data of the cloud host to be evaluated.
  • the third time series data; the second time series data and the third time series data are evaluated by the exponentially weighted moving average method EWMA in the data processing sub-model to obtain the first smoothing data and the first smoothing Data
  • the evaluation of the second time series data and the third time series data is calculated as follows: Among them, x t is the actual second time series data or the actual third time series data at time t, the coefficient ⁇ is the rate of weighted decline, and V t is the EWMA value at time t; through the Mahalanobis distance in the data processing sub-model
  • the algorithm detects and identifies the extreme abnormal points in the first smoothing data and the second smoothing data respectively, and deletes the marked extreme abnormal points.
  • the calculation for detecting and identifying the extreme abnormal points is
  • the above-mentioned processing module 202 is further configured to: obtain first characteristic information of the resource to be assessed in the first sampled data, and obtain second characteristic information of the resource to be assessed in the second sampled data, where the first characteristic information and The second feature information includes the data type, the total resource capacity, the proportion of resource usage corresponding to each data type, the performance and characteristics of the resource; the first resource corresponding to the first feature information is obtained from the resource database according to the first feature information Data, and obtain the second resource data corresponding to the second characteristic information in the resource database according to the second characteristic information; calculate and filter the first resource data through preset replacement conditions to obtain the first alternative resource, and obtain the first alternative resource through preset replacement conditions; Set replacement conditions to calculate and filter the second resource data to obtain the second alternative resource; according to the first alternative resource, perform analysis and matching in the cloud host other than the cloud host to be evaluated to obtain the first alternative cloud Host, and obtain the first resource usage information of the first alternative cloud host, and mark the first alternative cloud host and the first resource usage information on the first alternative
  • the above-mentioned processing module 202 is further used to: obtain training data, perform data preprocessing on the training data; store the training data that has undergone smoothing and abnormal data processing in a training database, and set a configuration file, where the configuration file Including network structure, training duration, training and testing ratio arrangement, output content, optimization learning rate setting, optimization parameter and archiving rule setting; according to the configuration file, the training data is smoothed to obtain prediction information; The comprehensive evaluation rules set up evaluate the forecast information to obtain the evaluation information.
  • the evaluation information includes the scoring range and analysis information; according to the evaluation information, the optimized configuration strategy corresponding to the evaluation information is generated to obtain the resource monitoring model; and the created detection
  • the script performs accuracy detection and performance testing on the resource monitoring model; if the accuracy detection result reaches the first preset threshold and the performance test result reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model; if If the result of the accuracy detection does not reach the first preset threshold and/or the result of the performance test does not reach the second preset threshold, the training data is continuously updated and the preset comprehensive evaluation rules are modified, and the resource monitoring model is retrained , Until the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold.
  • multi-angle resource usage data is obtained so that users can quickly and comprehensively learn the resource usage of the cloud host to be evaluated; on the other hand, multi-angle data is evaluated to improve the evaluation Because of its rigor and accuracy, this application can improve the resource utilization of the cloud host.
  • the technical features mentioned in any embodiment or implementation of the method for evaluating cloud host resources are also applicable to the evaluation of cloud host resources in this application.
  • the similarities will not be repeated here.
  • the device 20 in the embodiment of the present application is described above from the perspective of modular functional entities.
  • the following describes a computer device from the perspective of hardware, as shown in FIG. 3, which includes: a processor, a memory, a display, and an input and output unit ( It may also be a transceiver (not identified in FIG. 3) and a computer program stored in the memory and running on the processor.
  • the computer program may be a program corresponding to the method for evaluating cloud host resources in the embodiment corresponding to FIG. 1 or any optional embodiment in the embodiment corresponding to FIG. 1 or the optional implementation manner.
  • a computer device implements the function of the device 20 shown in FIG.
  • the processor executes the computer program to implement the method for evaluating cloud host resources executed by the device 20 in the embodiment corresponding to FIG. 2
  • the processor executes the computer program, the function of each module in the apparatus 20 of the embodiment corresponding to FIG. 2 is realized.
  • the computer program may be a program corresponding to the method in the embodiment corresponding to FIG. 1 or any optional embodiment in the embodiment corresponding to FIG. 1 or the optional implementation manner.
  • the so-called processor can be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
  • the processor is the control center of the computer device, and various interfaces and lines are used to connect various parts of the entire computer device.
  • the memory may be used to store the computer program and/or module, and the processor implements the computer by running or executing the computer program and/or module stored in the memory and calling data stored in the memory.
  • the memory may mainly include a storage program area and a storage data area, where the storage program area can store an operating system and at least one application program required by a function (such as determining the cloud host to be evaluated and the resource to be evaluated of the cloud host to be evaluated, etc.) Etc.; the data storage area can store data created based on the use of the mobile phone (such as obtaining training data, etc.), etc.
  • the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, memory, plug-in hard disks, smart media cards (SMC), and secure digital (SD) cards.
  • non-volatile memory such as hard disks, memory, plug-in hard disks, smart media cards (SMC), and secure digital (SD) cards.
  • Flash Card at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the input and output units can also be replaced by receivers and transmitters, and they can be the same or different physical entities. When they are the same physical entity, they can be collectively referred to as input and output units.
  • the input and output unit may be a transceiver.
  • the memory may be integrated in the processor, or may be provided separately from the processor.
  • the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.
  • the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer executes the following steps:
  • training data input the training data into the neural network model, and train the neural network model to obtain a resource monitoring model, where the training data includes resource sampling data of multiple cloud hosts for multiple periods; obtain the input target item Information, and analyze the target project information to obtain the project data type and project operation requirement information of the target project information; obtain the resource type and resource usage of the cloud host, and according to the resource type, resource usage, and project data type And project operation requirements information, determine the cloud host to be evaluated and the resource to be evaluated; obtain the first sampling data of the resource to be evaluated in the first period through the resource monitoring model, and obtain the resource to be evaluated in the second period
  • the second sampling data of the first time period is later than the end time of the second time period; the cloud host to be evaluated is predicted based on the first sampling data and the second sampling data to obtain the first prediction data and the first prediction data, respectively.
  • the second prediction data and the third prediction data, and the third prediction data is used as the main information, and the first prediction data and the second prediction data are used as auxiliary information to obtain the prediction information, where the prediction includes the cloud host to be evaluated in the third To predict the usage status of each resource and the trend of each resource usage in a time period, the start time of the third time period is later than the end time of the first time period; feature extraction is performed on the first sampling data, the second sampling data and the prediction information respectively, To obtain the first key information, the second key information, and the third key information respectively, and obtain the first alternative resource of the resource to be evaluated in the first sampling data, and obtain the second alternative resource of the resource to be evaluated in the second sampling data Resources; evaluate the cloud host to be evaluated based on the first key information, second key information, third key information, first alternative resource, and second alternative resource to obtain the evaluation information; generate the corresponding evaluation information based on the evaluation information Optimize the configuration strategy, and output the evaluation information and optimize the configuration strategy.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The present application relates to the field of cloud services, and provides a method, apparatus and device for estimating cloud host resources, and a storage medium. The method comprises: analyzing target item information, and used resource types and resource usage situations of a plurality of cloud hosts by means of a resource monitoring model obtained through training, so as to acquire a cloud host to be subjected to estimation and a resource to be estimated of said cloud host; acquiring first sampling data and second sampling data of said resource; acquiring prediction information according to the first sampling data and the second sampling data; acquiring estimated information according to the first sampling data, the second sampling data and the prediction information; and generating, according to the estimated information, an optimization configuration policy corresponding to the estimated information. By means of the solution, the resource utilization rate of a cloud host can be increased.

Description

评估云主机资源的方法、装置、设备及存储介质Method, device, equipment and storage medium for evaluating cloud host resources
本申请要求于2019年9月19日提交中国专利局、申请号为201910885428.1,发明名称为“评估云主机资源的方法、装置、设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 19, 2019, the application number is 201910885428.1, and the invention title is "Methods, Apparatus, Equipment, and Storage Medium for Evaluating Cloud Host Resources". The entire content of the application is approved. The reference is incorporated in the application.
技术领域Technical field
本申请涉及云监控领域,尤其涉及评估云主机资源的方法、装置、设备及存储介质。This application relates to the field of cloud monitoring, in particular to methods, devices, equipment, and storage media for evaluating cloud host resources.
背景技术Background technique
云主机作为新一代的主机租用服务,它整合了高性能服务器与优质网络带宽,有效解决了传统主机租用价格偏高和服务品质参差不齐的缺点,能全面满足中小企业、个人站长用户对主机租用服务低成本、高可靠、易管理的需求,因而,云主机被广泛使用。但是,随着时间的发展、人员的变动或者项目的搁置等情况变化,导致云主机资源的不合理利用和浪费,诸如离职人员所归属的云主机由于可能存在项目的关联而在离职过程中并未被释放,从而导致许多服务器资源使用量极低却配备高配置云主机,进而造成云主机资源的浪费。As a new generation of hosting services, cloud hosting integrates high-performance servers and high-quality network bandwidth, effectively solving the shortcomings of traditional hosting rental prices and uneven service quality, and can fully meet the needs of small and medium-sized enterprises and individual webmaster users. Host rental services require low cost, high reliability, and easy management. Therefore, cloud hosting is widely used. However, with the development of time, personnel changes, or the shelving of projects, etc., the unreasonable use and waste of cloud host resources, such as the cloud host to which the resigned personnel belong, may not be involved in the resignation process due to possible project associations. It has not been released, resulting in many servers with extremely low resource usage but equipped with high-configuration cloud hosts, which in turn causes a waste of cloud host resources.
目前的云监控中,通过获取云主机中当前的资源使用信息,对所述资源使用信息进行分析与评估,以获得评估信息,根据评估信息确定云主机的使用状态,并根据所述使用状态确定优化配置方案。In the current cloud monitoring, by obtaining the current resource usage information in the cloud host, the resource usage information is analyzed and evaluated to obtain the evaluation information, the usage status of the cloud host is determined according to the evaluation information, and the usage status is determined according to the usage status Optimize the configuration plan.
由于云主机的资源使用量会随着用户人数和使用时间的增加而增长,用户人数和使用时间会影响到云主机的资源使用量,发明人意识到在该情况下,若继续采用对云主机当前的资源使用量进行分析和评估的方案,则会导致所分析和评估的资源使用信息单一,致使所确定的优化配置策略不严谨且不全面,从而,导致云主机的资源利用率低。Since the resource usage of the cloud host will increase with the increase in the number of users and usage time, the number of users and usage time will affect the resource usage of the cloud host. The inventor realizes that in this case, if you continue to use the cloud host The current resource usage analysis and evaluation plan will result in the single analysis and evaluation of resource usage information, resulting in the determined optimization configuration strategy not being rigorous and comprehensive, resulting in low resource utilization of the cloud host.
发明内容Summary of the invention
本申请提供了一种评估云主机资源的方法、装置、设备及存储介质,能够解决云主机的资源利用率低的问题。This application provides a method, device, equipment, and storage medium for evaluating cloud host resources, which can solve the problem of low resource utilization of cloud hosts.
第一方面,本申请提供一种评估云主机资源的方法,包括:获取训练数据,将所述训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅 助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;根据所述评估信息生成与所述评估信息对应的优化配置策略,并输出所述评估信息和所述优化配置策略。In a first aspect, this application provides a method for evaluating cloud host resources, including: obtaining training data, inputting the training data to a neural network model, and training the neural network model to obtain a resource monitoring model, wherein , The training data includes resource sampling data of multiple cloud hosts for multiple periods; acquiring the input target project information, and analyzing the target project information to obtain the project data type and project operation of the target project information Demand information; obtain the resource type and resource usage of the cloud host, and determine the cloud host to be evaluated and the resource usage condition according to the resource usage type, the resource usage condition, the project data type, and the project operation requirement information The resource to be evaluated of the cloud host to be evaluated; obtaining the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and obtaining the second sampling data of the resource to be evaluated in the second time period, Wherein, the start time of the first time period is later than the end time of the second time period; the cloud host to be evaluated is predicted according to the first sampling data and the second sampling data to obtain the first A prediction data, a second prediction data, and a third prediction data, and the third prediction data is used as main information, and the first prediction data and the second prediction data are used as auxiliary information to obtain prediction information, Wherein, the prediction includes a prediction of the resource usage status and the trend of each resource usage of the cloud host to be evaluated in a third period, and the start time of the third period is later than the end of the first period Time; feature extraction of the first sampling data, the second sampling data, and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and to obtain the first The first alternative resource of the resource to be evaluated in the sampling data, and the second alternative resource of the resource to be evaluated in the second sampling data is acquired; according to the first key information, the second key information, and the first Three key information, the first alternative resource and the second alternative resource evaluate the cloud host to be evaluated to obtain evaluation information; generate an optimized configuration strategy corresponding to the evaluation information according to the evaluation information , And output the evaluation information and the optimal configuration strategy.
第二方面,本申请提供一种用于评估云主机资源的装置,包括:输入输出模块,用于获取训练数据,用于获取输入的目标项目信息;处理模块,用于将所述输入输出模块获取的训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;并对所述输入输出模块获取的输入的目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;根据所述评估信息生成与所述评估信息对应的优化配置策略,将所述评估信息和所述优化配置策略至显示模块,并通过所述显示模块输出所述评估信息和所述优化配置策略;所述显示模块,用于显示所述评估信息和所述优化配置策略。In a second aspect, the present application provides an apparatus for evaluating cloud host resources, including: an input and output module for obtaining training data for obtaining input target item information; and a processing module for converting the input and output module The acquired training data is input to a neural network model, and the neural network model is trained to obtain a resource monitoring model, wherein the training data includes resource sampling data of multiple cloud hosts for multiple periods; and The input target project information obtained by the input and output module is analyzed to obtain the project data type and project operation requirement information of the target project information; the used resource type and the used resource situation of the cloud host are obtained, and according to the used resource type, The resource usage, the project data type, and the project operation requirement information are used to determine the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed; and the resource to be assessed is acquired through the resource monitoring model. The first sampling data in a period of time, and the second sampling data of acquiring the resource to be evaluated in a second period of time, wherein the start time of the first period is later than the end time of the second period; according to The first sampling data and the second sampling data predict the cloud host to be evaluated to obtain first prediction data, second prediction data, and third prediction data, respectively, and use the third prediction data as Main information, and use the first prediction data and the second prediction data as auxiliary information to obtain prediction information, where the prediction includes the use of each resource status of the cloud host to be evaluated in the third period and For the prediction of the trend of each resource usage, the start time of the third period is later than the end time of the first period; the first sampling data, the second sampling data, and the prediction information are respectively performed Feature extraction to obtain the first key information, the second key information, and the third key information respectively, and to obtain the first alternative resource of the resource to be evaluated in the first sampling data, and to obtain the resource to be evaluated in the second sampling data. Evaluate the second alternative resource of the resource; according to the first key information, the second key information, the third key information, the first alternative resource and the second alternative resource Evaluate the cloud host for evaluation to obtain evaluation information; generate an optimized configuration strategy corresponding to the evaluation information according to the evaluation information, send the evaluation information and the optimized configuration strategy to the display module, and output through the display module The evaluation information and the optimal configuration strategy; the display module is used to display the evaluation information and the optimal configuration strategy.
第三方面,本申请提供了一种计算机设备,其包括至少一个连接的处理器、存储器、显示器和输入输出单元,其中,所述存储器用于存储程序代码,所述处理器用于调用所述存储器中的程序代码来执行上述第一方面所述的方法。In a third aspect, the present application provides a computer device, which includes at least one connected processor, a memory, a display, and an input and output unit, wherein the memory is used to store program code, and the processor is used to call the memory To execute the method described in the first aspect above.
第四方面,本申请提供了一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机指令,当所述计算机指令在计算机上运行时,使得计算机执行上述第一方面所述的方法。In a fourth aspect, the present application provides a computer-readable storage medium having computer instructions stored in the computer-readable storage medium. When the computer instructions run on a computer, the computer executes the above-mentioned first aspect. method.
本申请提供的技术方案中,通过训练所得的资源监控模型对目标项目信息、多个云主 机的使用资源类型和使用资源情况进行分析,以获取待评估云主机以及所述待评估云主机的待评估资源,获取所述待评估资源的第一采样数据和和第二采样数据,根据所述第一采样数据和所述第二采样数据获取预测信息,根据所述第一采样数据、所述第二采样数据和所述预测信息获取评估评估信息,根据所述评估信息生成与所述评估信息对应的优化配置策略。由于是通过在根据目标项目信息确定待评估云主机和所述待评估云主机的待评估资源,以使在对云主机的资源使用评估的基础上,还能满足对目标项目信息的评估需求,从而提高云主机的资源利用率;通过对云主机的第一采样数据、第二采样数据和预测信息进行评估,所评估的数据信息多方位,以使评估结果更具严谨性和准确性,通过多角度获取预测数据以提高评估的准确性;综上所述,一方面,获取多角度的资源使用情况的数据,以便于用户快速而全面获悉待评估云主机的资源使用情况,另一方面,通过对多角度的数据进行评估以提高评估的严谨性和准确性,因而,本申请能够提高云主机的资源利用率。In the technical solution provided by this application, the resource monitoring model obtained through training analyzes the target project information, the types of resources used and the resource usage conditions of multiple cloud hosts to obtain the cloud host to be evaluated and the cloud host to be evaluated. Evaluate resources, obtain first sampling data and second sampling data of the resource to be evaluated, obtain prediction information according to the first sampling data and the second sampling data, and obtain prediction information according to the first sampling data and the first sampling data. Second, the sampling data and the prediction information obtain evaluation evaluation information, and based on the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated. Since the cloud host to be evaluated and the resources to be evaluated of the cloud host to be evaluated are determined based on the target project information, so that the evaluation requirements for the target project information can be met on the basis of the resource usage evaluation of the cloud host, Thereby improving the resource utilization of the cloud host; by evaluating the first sampling data, second sampling data and forecast information of the cloud host, the evaluated data information is multi-faceted, so that the evaluation results are more rigorous and accurate. Obtain forecast data from multiple angles to improve the accuracy of the assessment; in summary, on the one hand, obtain data on resource usage from multiple angles, so that users can quickly and comprehensively learn about the resource usage of the cloud host to be evaluated, on the other hand, By evaluating data from multiple angles to improve the rigor and accuracy of the evaluation, this application can improve the resource utilization of the cloud host.
附图说明Description of the drawings
图1为本申请实施例中评估云主机资源的方法的一种流程示意图;FIG. 1 is a schematic flowchart of a method for evaluating cloud host resources in an embodiment of this application;
图2为本申请实施例中用于评估云主机资源的装置的一种结构示意图;2 is a schematic structural diagram of an apparatus for evaluating cloud host resources in an embodiment of the application;
图3为本申请实施例中计算机装置的一种结构示意图。FIG. 3 is a schematic structural diagram of a computer device in an embodiment of the application.
具体实施方式detailed description
本申请提供一种评估云主机资源的方法、装置、设备及存储介质,可用于云主机的配置使用,为无使用资源的云主机的释放以及由于资源使用量极少而需要执行降低配置操作的云主机提供参考。其中,对于云计算平台厂商,可用于提供资源回收的量化标准,为减少内部计算资源的浪费提供参照支持;对于云计算平台租户,可用于为对业务云主机的资源使用情况和是否使用合理提供参考。This application provides a method, device, device, and storage medium for evaluating cloud host resources, which can be used for the configuration and use of cloud hosts, for the release of cloud hosts that do not use resources, and for those who need to perform configuration reduction operations due to minimal resource usage. Cloud hosting provides reference. Among them, for cloud computing platform vendors, it can be used to provide quantitative standards for resource recovery to provide reference support for reducing the waste of internal computing resources; for cloud computing platform tenants, it can be used to provide information on the resource usage and rational use of business cloud hosts reference.
为解决上述技术问题,本申请主要提供以下技术方案:In order to solve the above technical problems, this application mainly provides the following technical solutions:
请参照图1,以下对本申请提供一种评估云主机资源的方法进行举例说明,该方法由计算机设备执行,计算机设备可为服务器或者终端,当图2所示的装置20为应用或者执行程序时,终端为安装图2所示的装置20的终端,本申请不对执行主体的类型作限制,所述方法包括:Please refer to FIG. 1, the following is an example of a method for evaluating cloud host resources provided by the present application. The method is executed by a computer device. The computer device may be a server or a terminal. When the device 20 shown in FIG. 2 is an application or an executing program , The terminal is a terminal on which the device 20 shown in FIG. 2 is installed. This application does not limit the type of execution subject. The method includes:
101、获取训练数据,将训练数据输入至神经网络模型,对神经网络模型进行训练,以获得资源监控模型。101. Obtain training data, input the training data into a neural network model, and train the neural network model to obtain a resource monitoring model.
其中,训练数据包括多个云主机的多个时段的资源采样数据;训练数据为按照预设采样间隔采样的按照顺向时间顺序排列的多个采样点数据,包括第一时段内的第一数据和第二时段内的第二数据,第一时段的起始时刻晚于第二时段的结束时刻。本实施例中,可通过迁移学习算法对现有的监控模型进行训练,以获取用于对云主机的使用资源进行监控、评估和优化配置的资源监控模型。本实施例的资源监控模型为后向传播神经网络模型。通过结合后向传播神经网络模型的输入输出的非线性映射、进行梯度下降计算、具备一定泛化能力和使用不同传递函数的特点,一方面,使其对云主机的使用资源进行监控、评估和优化配置等的数据挖掘进行多维的特征构造,和新的数据进入神经网络模型的网络进行训 练的时候,神经网络能够在调整权值以适应更多的数据;另一方面,使其输出的值可是任意值,和减小资源监控模型的误差,以获取更优的评估信息和优化配置策略。Wherein, the training data includes resource sampling data of multiple cloud hosts for multiple periods; the training data is data of multiple sampling points that are sampled at a preset sampling interval and arranged in a forward time sequence, including the first data in the first period And the second data in the second time period, the start time of the first time period is later than the end time of the second time period. In this embodiment, the existing monitoring model can be trained through a migration learning algorithm to obtain a resource monitoring model for monitoring, evaluating, and optimizing the configuration of the resources used by the cloud host. The resource monitoring model in this embodiment is a backward propagation neural network model. By combining the nonlinear mapping of the input and output of the backward propagation neural network model, performing gradient descent calculations, having a certain generalization ability and the characteristics of using different transfer functions, on the one hand, it enables it to monitor, evaluate and evaluate the use of cloud host resources. Data mining such as optimized configuration for multi-dimensional feature construction, and when new data enters the neural network model network for training, the neural network can adjust the weight to adapt to more data; on the other hand, make its output value But any value, and reduce the error of the resource monitoring model, in order to obtain better evaluation information and optimize the allocation strategy.
可选的,在本申请的一些实施例中,上述的获取训练数据,将训练数据输入至神经网络模型,对神经网络模型进行训练,以获得资源监控模型,包括:获取训练数据,对训练数据进行数据预处理;将经过平滑处理和异常数据处理的训练数据存储在训练数据库中,并设置配置文件,其中,配置文件包括网络结构、训练时长、训练与测试的比例安排、输出内容、优化学习率的设定、优化参数和存档规则设定;根据配置文件,对训练数据进行平滑处理,以得到预测信息;根据预设的综合评估规则对预测信息进行评估,以获得评估信息,其中,评估信息包括评分范围以及分析信息;根据评估信息生成与评估信息对应的优化配置策略,获得资源监控模型;通过已创建的检测脚本对资源监控模型进行准确性检测与性能测试;若准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值,则将资源监控模型作为最终的资源监控模型;若准确性检测的结果未达到第一预设阈值和/或性能测试的结果未达到第二预设阈值,则通过不断更新训练数据和修改预设的综合评估规则,并对资源监控模型进行再训练,直至准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值。Optionally, in some embodiments of the present application, the above-mentioned acquiring training data, inputting the training data to the neural network model, and training the neural network model to obtain the resource monitoring model include: acquiring training data, Perform data preprocessing; store the smoothed and abnormal data processed training data in the training database, and set up configuration files, where the configuration files include network structure, training duration, training and test ratio arrangements, output content, and optimized learning Rate setting, optimization parameters and archiving rules setting; according to the configuration file, the training data is smoothed to obtain the prediction information; the prediction information is evaluated according to the preset comprehensive evaluation rules to obtain the evaluation information, where the evaluation The information includes the scoring range and analysis information; according to the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated to obtain the resource monitoring model; the resource monitoring model is tested for accuracy and performance through the created test script; if the accuracy of the test results When the first preset threshold is reached and the result of the performance test reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model; if the accuracy check result does not reach the first preset threshold and/or the result of the performance test If the second preset threshold is not reached, the training data is continuously updated and the preset comprehensive evaluation rules are modified, and the resource monitoring model is retrained until the accuracy detection result reaches the first preset threshold and the performance test result reaches The second preset threshold.
对神经网络模型训练以使获取的资源监控模型具备对目标项目信息、多个云主机的使用资源类型和使用资源情况进行分析,以获取待评估云主机以及所述待评估云主机的待评估资源,获取所述待评估资源的第一采样数据和和第二采样数据,根据所述第一采样数据和所述第二采样数据获取预测信息,根据所述第一采样数据、所述第二采样数据和所述预测信息获取评估评估信息,根据所述评估信息生成与所述评估信息对应的优化配置策略的功能,以更好地对云主机的资源利用进行评估,从而提高云主机的资源利用率。Training the neural network model so that the acquired resource monitoring model has the ability to analyze the target project information, the resource type and resource usage of multiple cloud hosts, to obtain the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed , Obtain the first sampling data and the second sampling data of the resource to be evaluated, obtain the prediction information according to the first sampling data and the second sampling data, and obtain the prediction information according to the first sampling data and the second sampling data The data and the prediction information obtain the evaluation evaluation information, and the function of generating an optimized configuration strategy corresponding to the evaluation information according to the evaluation information, so as to better evaluate the resource utilization of the cloud host, thereby improving the resource utilization of the cloud host rate.
可选的,在本申请的一些实施例中,资源监控模型包括数据处理子模型、预测子模型、评估子模型和策略生成子模型,预测子模型与数据处理子模型串联,评估子模型与预测子模型串联,策略生成子模型与评估子模型串联。数据处理子模型用于对资源监控模型获取的采样数据进行平滑处理和异常数据处理;预测子模型用于对数据处理子模型输出的采样数据进行预测,得到预测信息;评估子模型用于对预测子模型输出的预测信息进行评估,得到评估信息;策略生成子模型用于根据评估子模型输出的评估信息,生成对应的优化配置策略。Optionally, in some embodiments of the present application, the resource monitoring model includes a data processing sub-model, a prediction sub-model, an evaluation sub-model, and a strategy generation sub-model. The prediction sub-model and the data processing sub-model are connected in series, and the evaluation sub-model and prediction are connected in series. The sub-models are connected in series, and the strategy generation sub-models are connected in series with the evaluation sub-models. The data processing sub-model is used to smooth the sampled data obtained by the resource monitoring model and abnormal data processing; the prediction sub-model is used to predict the sampled data output by the data processing sub-model to obtain prediction information; the evaluation sub-model is used to predict The prediction information output by the sub-model is evaluated to obtain the evaluation information; the strategy generation sub-model is used to generate the corresponding optimal configuration strategy according to the evaluation information output by the evaluation sub-model.
102、获取输入的目标项目信息,并对目标项目信息进行分析,以获取目标项目信息的项目数据类型和项目操作需求信息。102. Obtain the input target project information, and analyze the target project information to obtain the project data type and project operation requirement information of the target project information.
获取用户输入的目标项目信息后,对目标项目信息进行分析以获取完成目标项目所需要操作的数据类型和目标项目的操作需求信息。其中,项目操作需求信息包括但不限于项目完成的计划时间、期限时间和项目完成的功能。例如:目标项目为管理系统,通过分析用户输入的目标项目信息,以获取对管理系统研发完成的计划时间、期限时间、所需的云主机资源量、管理系统具备的性能和管理系统需具备的功能等。通过获取目标项目信息中的项目数据类型和项目操作需求信息,以便于对云主机的使用资源和资源配置的评估提供更好的依据和分析基础。After obtaining the target project information input by the user, the target project information is analyzed to obtain the data type of the operation required to complete the target project and the operation requirement information of the target project. Among them, the project operation requirement information includes, but is not limited to, the planned time for project completion, deadlines, and project completion functions. For example: the target project is a management system, by analyzing the target project information input by the user, to obtain the planned time, deadlines, amount of cloud host resources required for the completion of the management system research and development, the performance of the management system and the requirements of the management system Function and so on. By obtaining the project data type and project operation requirement information in the target project information, in order to provide a better basis and analysis basis for the evaluation of the use of resources and resource allocation of the cloud host.
可选的,在本申请的一些实施例中,上述的获取输入的目标项目信息,并对目标项目信息进行分析,以获取目标项目信息的项目数据类型和项目操作需求信息,包括:创建项目操作需求表,其中,项目操作需求表包括项目的预设完成时间、主机资源需求量和主机资源需求量对应的最优主机资源分配量;获取用户输入的目标项目信息,并对目标项目信息进行数据预处理,其中,数据预处理包括缺失值填补处理、去噪处理和数据标准化处理;将经过数据预处理的目标项目信息分成N组,通过反复迭代法对分成N组的目标项目信息进行多次重新分组,以获取最优分组方案;获取最优分组方案中各组目标项目信息的项目数据类型;对目标项目信息进行分析以获取第四关键信息;根据第四关键信息遍历项目操作需求表,以获取第四关键信息对应的项目操作需求信息。Optionally, in some embodiments of the present application, the above-mentioned obtaining the input target project information and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information includes: creating a project operation The demand table, where the project operation demand table includes the preset completion time of the project, the host resource demand and the optimal host resource allocation corresponding to the host resource demand; obtain the target project information input by the user, and perform data on the target project information Preprocessing, where data preprocessing includes missing value filling processing, denoising processing, and data standardization processing; divide the target item information after data preprocessing into N groups, and perform multiple iterations on the target item information divided into N groups Regroup to obtain the optimal grouping plan; obtain the project data type of each group of target project information in the optimal grouping plan; analyze the target project information to obtain the fourth key information; traverse the project operation requirement table according to the fourth key information, To obtain the project operation requirement information corresponding to the fourth key information.
通过对目标项目信息进行聚类分析,以获得简明结论形式,从而以便于资源监控模型直观而简便地获取目标项目的项目数据类型和项目操作需求信息。通过遍历项目操作需求表以获取关键信息对应的项目操作需求信息,以使获取项目操作需求信息多角度,从而能够更快速而准确地确定待评估云主机以及待评估云主机的待评估资源。Through the cluster analysis of the target project information, a concise conclusion form can be obtained, so that the resource monitoring model can intuitively and easily obtain the project data type and project operation requirement information of the target project. By traversing the project operation requirement table to obtain the project operation requirement information corresponding to the key information, the project operation requirement information can be obtained from multiple angles, so that the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed can be determined more quickly and accurately.
103、分析云主机的使用资源类型和使用资源情况,并根据项目数据类型和项目操作需求信息,以确定待评估云主机以及待评估云主机的待评估资源。103. Analyze the resource types and usage conditions of the cloud host, and determine the cloud host to be assessed and the resource to be assessed based on the project data type and project operation requirement information.
其中,云主机包括连接使用的所有的云主机;待评估资源包括资源使用情况和剩余资源情况,剩余资源情况为待评估云主机中总资源减去已使用资源后的资源量。通过在根据目标项目信息确定待评估云主机和所述待评估云主机的待评估资源,一方面有目的性、针对性地对云主机资源进行评估,另一方面减少不必要的运行操作以减轻系统的压力和提高运行速度。例如:分析所有的云主机的使用资源类型和使用资源情况,匹配出与项目数据类型对应的使用资源类型,以获取第一云主机,以及匹配出与项目操作需求信息对应的使用资源情况以获取第二云主机,判断第一云主机与第二云主机中既符合项目数据类型又能满足项目操作需求信息的云主机,作为待评估云主机,并获取待评估云主机中的资源使用情况和剩余资源情况。Among them, cloud hosts include all cloud hosts used by connections; resources to be evaluated include resource usage and remaining resources, and the remaining resources are the total resources of the cloud hosts to be evaluated minus used resources. By determining the cloud host to be evaluated and the resource to be evaluated according to the target project information, on the one hand, the cloud host resources are evaluated in a purposeful and targeted manner, and on the other hand, unnecessary operation operations are reduced to reduce System pressure and increase operating speed. For example: Analyze the resource types and resource usage conditions of all cloud hosts, match the resource usage types corresponding to the project data type to obtain the first cloud host, and match the resource usage conditions corresponding to the project operation requirement information to obtain The second cloud host, judges the cloud host that meets the project data type and meets the project operation demand information among the first cloud host and the second cloud host, as the cloud host to be evaluated, and obtains the resource usage and the resource usage of the cloud host to be evaluated The remaining resources.
104、通过资源监控模型获取待评估资源在第一时段内的第一采样数据,以及获取待评估资源在第二时段内的第二采样数据。104. Obtain the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and acquire the second sampling data of the resource to be evaluated in the second time period.
其中,第一时段的起始时刻晚于所述第二时段的结束时刻。第一采样数据和第二采样数据均是按照预设采样间隔采样的按照顺向时间顺序排列的多个采样点数据,第一采样数据和第二采样数据均包括中央处理器CPU的占用率信息、内存占用率信息以及输入/输出IO占用率信息,占用率信息包括物理资源与总资源的占比、功能性描述的信息资源与总资源的占比、非功能性描述的信息资源与总资源的占比、网络资源与总资源的占比、已使用的资源与总资源的占比和未使用的资源与总资源的占比。通过采集两个不同时段的采样数据,增加评估资源的多样性,以提高对云主机资源评估的可参考性和准确性。Wherein, the start time of the first time period is later than the end time of the second time period. Both the first sampling data and the second sampling data are data of multiple sampling points that are sampled at a preset sampling interval and arranged in a forward time sequence. Both the first sampling data and the second sampling data include the occupancy rate information of the central processing unit CPU , Memory occupancy information and input/output IO occupancy information. The occupancy information includes the proportion of physical resources and total resources, the proportion of functionally described information resources and total resources, and the non-functionally described information resources and total resources. The proportion of network resources and the proportion of total resources, the proportion of used resources and the total resources, and the proportion of unused resources and the total resources. By collecting sampling data in two different time periods, the diversity of evaluation resources is increased, so as to improve the reference and accuracy of cloud host resource evaluation.
可选的,在本申请的一些实施例中,上述的待评估资源包括在第一时段内的第一任务和在第二时段内的第二任务,上述的通过资源监控模型获取待评估资源在第一时段内的第一采样数据,以及获取待评估资源在第二时段内的第二采样数据,包括:通过资源监控模型,获取第一任务中的第一占用率信息和第一优先级,以及获取第二任务中的第二占用率 信息和第二优先级;根据第一优先级,对第一任务进行分类,并标识第一类别标签,以及根据第二优先级,对第二任务进行分类,并标识第二类别标签;根据第一占用率信息,对标识第一类别标签的第一任务进行分类,并标识第三类别标签,以及第二占用率信息,对标识第二类别标签的第二任务进行分类,并标识第四类别标签;判断第一优先级是否符合第一预设采样条件和/或第一占用率信息是否符合第二预设采样条件,以及判断第二优先级是否符合第三预设采样条件和/或第二占用率信息是否符合第四预设采样条件;若第一优先级符合第一预设采样条件和/或第一占用率信息符合第二预设采样条件,则按照预设第一采样频率,对符合第一预设采样条件和/或第二预设采样条件的同一类别标签的任务进行采样,以获得第一采样数据;若第二优先级符合第三预设采样条件和/或第二占用率信息符合第四预设采样条件,则按照预设第二采样频率,对符合第三预设采样条件和/或第四预设采样条件的同一类别标签的任务进行采样,以获得第二采样数据。Optionally, in some embodiments of the present application, the aforementioned resource to be evaluated includes a first task in a first period and a second task in a second period. The aforementioned resource monitoring model is used to obtain the resource to be evaluated. The first sampling data in the first time period and obtaining the second sampling data of the resource to be evaluated in the second time period include: obtaining the first occupancy rate information and the first priority of the first task through the resource monitoring model, And obtain the second occupancy rate information and the second priority of the second task; classify the first task according to the first priority, and identify the first category label, and perform the second task according to the second priority Classify and identify the second category label; according to the first occupancy rate information, classify the first task that identifies the first category label, and identify the third category label, and the second occupancy rate information, to identify the second category label Classify the second task and identify the fourth category label; determine whether the first priority meets the first preset sampling condition and/or whether the first occupancy rate information meets the second preset sampling condition, and determine whether the second priority Whether the third preset sampling condition and/or the second occupancy rate information meets the fourth preset sampling condition; if the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling Condition, according to the preset first sampling frequency, the tasks of the same category label that meet the first preset sampling condition and/or the second preset sampling condition are sampled to obtain the first sampling data; if the second priority meets If the third preset sampling condition and/or the second occupancy rate information meets the fourth preset sampling condition, then according to the preset second sampling frequency, the same sampling condition that meets the third preset sampling condition and/or the fourth preset sampling condition is The task of the category label performs sampling to obtain the second sampling data.
通过上述实施方式,能够实现便于获悉云主机中资源的使用状况和对资源占比的影响程度,从而对后续得到的评估信息的准确性和严谨性起到一定的支持作用的有益效果。由于用户提交的每一个任务(即为应用请求)在提交时都带有特定的优先级,而任何一个相对高优先级的任务可以抢占低优先级使用的资源,且处于不同档次的高优先级的任务具有对应的特征,例如:正常生产档的高优先级的任务对延迟敏感,一般不会因为过度使用资源而被剔除、挂起或暂停,因而为了能够便于清晰了解各占用率信息的使用状况和对资源占比的影响程度,任务为用户发起的应用请求,可采用上述操作。Through the foregoing implementation manners, it is possible to facilitate the understanding of the use status of resources in the cloud host and the degree of influence on the proportion of resources, thereby supporting the accuracy and rigor of the subsequent evaluation information to a certain extent. Since each task submitted by the user (that is, an application request) is submitted with a specific priority, any relatively high priority task can preempt resources used by low priority, and is at a different level of high priority The tasks have corresponding characteristics. For example, high-priority tasks in the normal production stage are sensitive to delay, and generally will not be rejected, suspended or suspended due to excessive use of resources. Therefore, in order to facilitate a clear understanding of the use of each occupancy rate information The status and the degree of influence on the proportion of resources, the task is an application request initiated by the user, and the above operations can be used.
105、根据第一采样数据和第二采样数据对待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将第三预测数据作为主信息,以及将第一预测数据和第二预测数据作为辅助信息,以获得预测信息。105. Perform predictions on the cloud host to be evaluated based on the first sampling data and the second sampling data to obtain the first prediction data, the second prediction data, and the third prediction data respectively, using the third prediction data as the main information, and the first The first prediction data and the second prediction data are used as auxiliary information to obtain prediction information.
其中,预测包括对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,第三时段的起始时刻晚于第一时段的结束时刻。第一预测数据包括根据第一采样数据对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势进行预测而得的预测数据,第二预测数据包括根据第二采样数据对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势进行预测而得的预测数据,第三预测数据包括根据第一采样数据和第二采样数据对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势进行预测而得的预测数据,第三时段的起始时刻晚于第一时段的结束时刻。主信息和辅助信息指的是首先对主信息进行处理后,再结合辅助信息进行处理。通过采用多中预测数据和将第三预测数据作为主信息,以及将第一预测数据和第二预测数据作为辅助信息,以提高评估的准确性。Wherein, the prediction includes the prediction of the resource usage status and the trend of each resource usage of the cloud host to be evaluated in the third period, and the start time of the third period is later than the end time of the first period. The first prediction data includes the prediction data obtained by predicting the usage status of each resource and the trend of each resource usage of the cloud host in the third period based on the first sampling data, and the second prediction data includes the prediction data to be evaluated based on the second sampling data. The prediction data obtained by predicting the usage status of each resource and the trend of each resource usage of the cloud host in the third period. The third prediction data includes the cloud host to be evaluated in the third period based on the first sampling data and the second sampling data. The prediction data obtained by predicting the usage status of each resource and the trend of each resource usage, the start time of the third period is later than the end time of the first period. The main information and auxiliary information refer to the processing of the main information first, and then combined with the auxiliary information for processing. By using multiple prediction data and using the third prediction data as the main information, and using the first prediction data and the second prediction data as auxiliary information, the accuracy of the evaluation can be improved.
可选的,在本申请的一些实施例中,上述的获得预测信息之后,方法还包括:获取预测信息的第一时间序列数据;对第一时间序列数据进行滑动窗口处理,以生成预设数量的具备预设长度的时序子序列;分析时序子序列的统计指标以获取统计特征信息,并以统计特征信息作为更新后的预测信息,其中,统计特征信息包括最大值、最小值、中位数、第一四分位数、第三四分位数、方差和标准差。Optionally, in some embodiments of the present application, after the foregoing forecast information is obtained, the method further includes: acquiring first time series data of the forecast information; performing sliding window processing on the first time series data to generate a preset number The time series subsequence with a preset length; analyze the statistical indicators of the time series subsequence to obtain statistical characteristic information, and use the statistical characteristic information as the updated forecast information, where the statistical characteristic information includes the maximum value, the minimum value, and the median , First quartile, third quartile, variance and standard deviation.
通过对预测信息进行统计特征提取,使得预测信息的数据更集中、更系统、更清楚地 反映客观实际,以使后续的未来预测值更偏向现实值,促进预测的准确性。By extracting the statistical features of the forecast information, the data of the forecast information is more concentrated, more systematic, and more clearly reflecting the objective reality, so that the subsequent future forecast values are more biased towards the actual value, and the accuracy of the forecast is promoted.
可选的,在本申请的一些实施例中,上述的根据第一采样数据和第二采样数据对待评估云主机进行预测之前,方法还包括:获取第一采样数据的第二时间序列数据,以及获取第二采样数据的第三时间序列数据;通过数据处理子模型中的指数加权移动平均法EWMA,分别对第二时间序列数据和第三时间序列数据进行求值,以获得第一平滑处理数据和第一平滑处理数据,对第二时间序列数据和第三时间序列数据的求值计算如下:
Figure PCTCN2019117048-appb-000001
其中,x t为时刻t的实际第二时间序列数据或实际第三时间序列数据,系数α为加权下降的速率,V t为t时刻的EWMA值;通过数据处理子模型中的马氏距离Mahalanobis算法,分别对第一平滑处理数据和第二平滑处理数据中的极度异常点进行检测与标识,并删除所标识的极度异常点,检测与标识极度异常点的计算如下:
Figure PCTCN2019117048-appb-000002
其中,
Figure PCTCN2019117048-appb-000003
是b与
Figure PCTCN2019117048-appb-000004
的距离,
Figure PCTCN2019117048-appb-000005
是平滑处理数据的均值向量,b是为平滑处理数据中的其他对象,S是协方差矩阵。
Optionally, in some embodiments of the present application, before predicting the cloud host to be evaluated based on the first sampling data and the second sampling data, the method further includes: acquiring second time series data of the first sampling data, and Obtain the third time series data of the second sampling data; use the exponentially weighted moving average method EWMA in the data processing sub-model to evaluate the second time series data and the third time series data respectively to obtain the first smoothed data And the first smoothing data, the second time series data and the third time series data are evaluated as follows:
Figure PCTCN2019117048-appb-000001
Among them, x t is the actual second time series data or the actual third time series data at time t, the coefficient α is the rate of weighted decline, and V t is the EWMA value at time t; through the Mahalanobis distance in the data processing sub-model The algorithm detects and identifies the extreme abnormal points in the first smoothing data and the second smoothing data respectively, and deletes the marked extreme abnormal points. The calculation for detecting and identifying the extreme abnormal points is as follows:
Figure PCTCN2019117048-appb-000002
among them,
Figure PCTCN2019117048-appb-000003
Is b and
Figure PCTCN2019117048-appb-000004
the distance,
Figure PCTCN2019117048-appb-000005
Is the mean vector of the smoothed data, b is the other objects in the smoothed data, and S is the covariance matrix.
通过对第一采样数据和第二采样数据进行平滑处理和异常数据处理,以获得相对均衡、稳定的时间序列数据,减少误差,为后续的未来预测值的准确值提供进一步的支持。Smooth processing and abnormal data processing are performed on the first sampled data and the second sampled data to obtain relatively balanced and stable time series data, reduce errors, and provide further support for the subsequent accurate values of future predicted values.
106、分别对第一采样数据、第二采样数据和预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取第一采样数据中待评估资源的第一可替代资源,和获取第二采样数据中待评估资源的第二可替代资源。106. Perform feature extraction on the first sampling data, the second sampling data, and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and obtain the first sampling data of the resource to be evaluated. An alternative resource, and a second alternative resource for obtaining the resource to be evaluated in the second sampling data.
其中,分析包括对第一采样数据、第二采样数据和预测信息的各资源的类型、使用时间、操作速率和资源占比的识别和获取。第一关键信息包括对第一采样数据进行分析而获取的各资源的类型、使用时间、操作速率和资源占比,第二关键信息包括对第二采样数据进行分析而获取的各资源的类型、使用时间、操作速率和资源占比,第三关键信息包括对预测信息进行分析而获取的各资源的类型、使用时间、操作速率和资源占比。一方面,通过获取多角度的资源使用情况的数据,以便于用户快速而全面获悉待评估云主机的资源使用,另一方面,通过结合多样化的数据进行评估,以提高评估的准确性。Among them, the analysis includes the identification and acquisition of the type, use time, operation rate, and resource proportion of each resource of the first sampling data, the second sampling data, and the prediction information. The first key information includes the type, usage time, operation rate, and resource proportion of each resource obtained by analyzing the first sampled data, and the second key information includes the type of each resource obtained by analyzing the second sampled data, Use time, operation rate, and resource proportion. The third key information includes the type, use time, operation rate, and resource proportion of each resource obtained by analyzing the forecast information. On the one hand, by obtaining multi-angle resource usage data, users can quickly and comprehensively learn the resource usage of the cloud host to be evaluated. On the other hand, by combining diversified data for evaluation, the accuracy of the evaluation can be improved.
可选的,在本申请的一些实施例中,本方法包括资源数据库,资源数据库包括收集的多种数据类型对应的资源数据,上述的获取第一采样数据中待评估资源的第一可替代资源,和获取第二采样数据中待评估资源的第二可替代资源,包括:获取第一采样数据中待评估资源的第一特征信息,以及获取第二采样数据中待评估资源的第二特征信息,其中,第一特征信息和第二特征信息均包括数据类型、资源总容量、各数据类型对应的资源的使用占比、资源的性能和特性;根据第一特征信息在资源数据库中获取与第一特征信息对应的第一资源数据,以及根据第二特征信息在资源数据库中获取与第二特征信息对应的第二资源数据;通过预设替换条件对第一资源数据进行计算和筛选以获取第一可替代资源,以及通过预设替换条件对第二资源数据进行计算和筛选以获取第二可替代资源;根据第一可替换资源,在待评估云主机之外的云主机中进行分析和匹配,以获取第一可替代云主机,并获取第一可替代云主机的第一资源使用信息,并将第一可替代云主机和第一资源使用信息标记在第一可替换资源上,以获取最终的第一可替换资源,以及根据第二可替换资源,在待 评估云主机之外的云主机中进行分析和匹配,以获取第二可替代云主机,并获取第二可替代云主机的第二资源使用信息,并将第二可替代云主机和第二资源使用信息标记在第二可替换资源上,以获取最终的第二可替换资源。Optionally, in some embodiments of the present application, the method includes a resource database. The resource database includes collected resource data corresponding to multiple data types. The first alternative resource of the resource to be evaluated in the first sampling data is obtained as described above. , And acquiring the second alternative resource of the resource to be evaluated in the second sampling data, including: acquiring first characteristic information of the resource to be evaluated in the first sampling data, and acquiring second characteristic information of the resource to be evaluated in the second sampling data , Where the first feature information and the second feature information both include the data type, the total capacity of the resource, the usage proportion of the resource corresponding to each data type, the performance and characteristics of the resource; the first feature information is obtained from the resource database according to the first feature information. The first resource data corresponding to one feature information, and the second resource data corresponding to the second feature information is obtained in the resource database according to the second feature information; the first resource data is calculated and filtered by preset replacement conditions to obtain the first resource data A replaceable resource, and the second resource data is calculated and filtered through preset replacement conditions to obtain the second replaceable resource; based on the first replaceable resource, analysis and matching are performed in cloud hosts other than the cloud host to be evaluated , To obtain the first alternative cloud host, and obtain the first resource usage information of the first alternative cloud host, and mark the first alternative cloud host and the first resource usage information on the first alternative resource to obtain The final first alternative resource, and according to the second alternative resource, perform analysis and matching among cloud hosts other than the cloud host to be evaluated to obtain the second alternative cloud host, and obtain the second alternative cloud host Second resource usage information, and mark the second replaceable cloud host and the second resource usage information on the second replaceable resource to obtain the final second replaceable resource.
通过标记可替代云主机和可替代云主机的资源使用信息,一方面,以便于用户获知可替代使用的云主机,以达到节省资源的效果,另一方面,以为评估提供多角度多方面的数据,从而提高评估的准确性和可行性。By marking the resource usage information of alternative cloud hosts and alternative cloud hosts, on the one hand, users can learn about alternative cloud hosts to save resources, and on the other hand, provide multi-angle and multi-faceted data for evaluation , Thereby improving the accuracy and feasibility of the assessment.
107、根据第一关键信息、第二关键信息、第三关键信息、第一可替代资源和第二可替代资源对待评估云主机进行评估,以获取评估信息。107. Evaluate the cloud host to be evaluated based on the first key information, the second key information, the third key information, the first replaceable resource, and the second replaceable resource to obtain evaluation information.
根据第一关键信息、第二关键信息和第三关键信息对待评估云主机的当前的使用资源和未来某一段时间的预估使用资源进行评估,并根据第一可替代资源和第二可替代资源评估可替换待评估云主机的其他云主机,以合理使用云主机和提高云主机的资源利用率。例如:甲云主机为待评估云主机,乙云主机为待评估云主机之外的云主机。根据第一关键信息、第二关键信息和第三关键信息对甲云主机进行评估,所得甲云主机的可使用资源已剩10%,且各项指标不满足继续使用要求和满足项目的要求,而乙云主机的可使用资源还有90%,和已空置较长一段时间,根据第一关键信息、第二关键信息、第三关键信息、第一可替代资源和第二可替代资源对甲云主机和乙云主机分析,所得乙云主机的资源类型与甲云主机相同和/或相似,可进行替换使用,且各项指标均满足继续使用要求和满足项目的要求,则对于输入的目标项目信息,乙云主机可作为甲云主机的可替换云主机。其中,分析过程所产生的过程信息和结果信息,作为评估信息。According to the first key information, the second key information, and the third key information, the current use resources of the cloud host to be evaluated and the estimated use resources for a certain period of time in the future are evaluated, and based on the first alternative resource and the second alternative resource The evaluation can replace other cloud hosts of the cloud host to be evaluated in order to rationally use the cloud host and improve the resource utilization rate of the cloud host. For example: A cloud host is a cloud host to be evaluated, and a cloud host B is a cloud host other than the cloud host to be evaluated. According to the evaluation of the first key information, the second key information and the third key information, the available resources of the A Cloud host are 10% left, and the indicators do not meet the requirements of continued use and the requirements of the project. On the other hand, there are still 90% of the available resources of Cloud B, and it has been vacant for a long time. According to the first key information, the second key information, the third key information, the first alternative resource and the second alternative resource, Cloud host and B cloud host analysis, the resource type of B cloud host is the same and/or similar to A cloud host, which can be replaced, and all indicators meet the requirements of continued use and the requirements of the project, then the input goals are Project information, Cloud B host can be used as a replaceable cloud host for Cloud A host. Among them, the process information and result information generated by the analysis process are used as evaluation information.
108、根据评估信息,生成与评估信息对应的优化配置策略,输出评估信息和优化配置策略。108. According to the evaluation information, generate an optimized configuration strategy corresponding to the evaluation information, and output the evaluation information and optimized configuration strategy.
其中,优化配置策略包括云主机的资源使用状态、云主机的各物理资源占用率信息、是否需要对云主机进行配置提升或配置降低或资源释放,以及云主机的预设未来各时间段的资源使用情况、用户行为匹配和语义匹配。Among them, the optimal configuration strategy includes the resource usage status of the cloud host, the occupancy rate information of each physical resource of the cloud host, whether the cloud host needs to be configured to upgrade or reduce or release resources, and the cloud host's preset resources for each time period in the future Usage, user behavior matching, and semantic matching.
与现有机制相比,本申请实施例中,一方面,获取多角度的资源使用情况的数据,以便于用户快速而全面获悉待评估云主机的资源使用情况,另一方面,通过对多角度的数据进行评估以提高评估的严谨性和准确性,因而,本申请能够提高云主机的资源利用率。Compared with the existing mechanism, in the embodiments of the present application, on the one hand, data on resource usage from multiple perspectives is obtained, so that users can quickly and comprehensively learn about the resource usage of the cloud host to be evaluated. On the other hand, through multiple perspectives The data is evaluated to improve the rigor and accuracy of the evaluation. Therefore, this application can improve the resource utilization rate of the cloud host.
上述图1对应的实施例或图1对应的实施例中的任一可选实施例或可选实施方式中所提及的技术特征也同样适用于本申请中的图2和图3所对应的实施例,后续类似之处不再赘述。The technical features mentioned in any optional embodiment or optional implementation in the embodiment corresponding to FIG. 1 or the embodiment corresponding to FIG. 1 are also applicable to those corresponding to FIG. 2 and FIG. 3 in this application. In the embodiment, the similarities will not be repeated in the following.
以上对本申请中一种评估云主机资源的方法进行说明,以下对执行上述评估云主机资源的方法的装置进行描述。The foregoing describes a method for evaluating cloud host resources in the present application, and the following describes an apparatus for executing the foregoing method for evaluating cloud host resources.
如图2所示的一种用于评估云主机资源的装置20的结构示意图,其可应用于云主机的配置使用,为无使用资源的云主机的释放以及由于资源使用量极少而需要执行降低配置操作的云主机提供参考。其中,对于云计算平台厂商,可用于提供资源回收的量化标准,为减少内部计算资源的浪费提供参照支持;对于云计算平台租户,可用于为对业务云主机的 资源使用情况和是否使用合理提供参考。本申请实施例中的装置20能够实现对应于上述图1对应的实施例或图1对应的实施例中的任一可选实施例或可选实施方式中所执行的评估云主机资源的方法的步骤。装置20实现的功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。硬件或软件包括一个或多个与上述功能相对应的模块,所述模块可以是软件和/或硬件。所述装置20可包括输入输出模块201、处理模块202和显示模块203,所述输入输出模块201、处理模块202和显示模块203的功能实现可参考图1对应的实施例或图1对应的实施例中的任一可选实施例或可选实施方式中所执行的操作,此处不作赘述。所述处理模块202可用于控制所述输入输出模块201的收发操作,显示模块203可用于显示处理模块202的处理操作。As shown in FIG. 2 is a schematic structural diagram of a device 20 for evaluating cloud host resources, which can be applied to the configuration and use of cloud hosts, for the release of cloud hosts that do not use resources, and the need to perform due to the minimal use of resources. The cloud host that reduces the configuration operation provides a reference. Among them, for cloud computing platform vendors, it can be used to provide quantitative standards for resource recovery to provide reference support for reducing the waste of internal computing resources; for cloud computing platform tenants, it can be used to provide information on the resource usage and rational use of business cloud hosts reference. The apparatus 20 in the embodiment of the present application can implement the method corresponding to the embodiment corresponding to FIG. 1 or any one of the optional embodiments or the optional implementation manners in the embodiment corresponding to FIG. 1 for evaluating cloud host resources. step. The functions implemented by the device 20 can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions, and the modules may be software and/or hardware. The device 20 may include an input/output module 201, a processing module 202, and a display module 203. The functional realization of the input/output module 201, the processing module 202, and the display module 203 may refer to the embodiment corresponding to FIG. 1 or the implementation corresponding to FIG. 1 The operations performed in any optional embodiment or optional implementation manner in the example will not be repeated here. The processing module 202 can be used to control the receiving and sending operations of the input/output module 201, and the display module 203 can be used to display the processing operations of the processing module 202.
一些实施方式中,输入输出模块201,用于获取训练数据,用于获取输入的目标项目信息;处理模块202,用于将输入输出模块201获取的训练数据输入至神经网络模型,对神经网络模型进行训练,以获得资源监控模型;对输入输出模块201获取的目标项目信息进行分析,以获取目标项目信息的项目数据类型和项目操作需求信息;获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据;根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息;分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;根据评估信息生成与评估信息对应的优化配置策略,将评估信息和优化配置策略至显示模块203,并通过显示模块203输出评估信息和优化配置策略;显示模块203,用于显示评估信息和优化配置策略。In some embodiments, the input and output module 201 is used to obtain training data and is used to obtain input target item information; the processing module 202 is used to input the training data obtained by the input and output module 201 into the neural network model, and the neural network model Train to obtain the resource monitoring model; analyze the target project information obtained by the input and output module 201 to obtain the project data type and project operation requirement information of the target project information; obtain the resource type and resource usage of the cloud host, and Determine the cloud host to be assessed and the resource to be assessed of the cloud host to be assessed according to the resource usage type, the resource usage situation, the project data type, and the project operation requirement information; obtain through the resource monitoring model The first sampling data of the resource to be evaluated in the first time period, and the second sampling data of the resource to be evaluated in the second time period are acquired; The cloud host to be evaluated performs prediction to obtain first prediction data, second prediction data, and third prediction data, respectively, using the third prediction data as main information, and combining the first prediction data and the first prediction data The second prediction data is used as auxiliary information to obtain prediction information; feature extraction is performed on the first sampling data, the second sampling data, and the prediction information, respectively, to obtain the first key information, the second key information, and the first key information, respectively. Three key information, and acquiring the first alternative resource of the resource to be evaluated in the first sampling data, and acquiring the second alternative resource of the resource to be evaluated in the second sampling data; according to the first key information, The second key information, the third key information, the first replaceable resource, and the second replaceable resource evaluate the cloud host to be evaluated to obtain evaluation information; generate and evaluate based on the evaluation information The optimized configuration strategy corresponding to the information, the evaluation information and the optimized configuration strategy are sent to the display module 203, and the evaluation information and the optimized configuration strategy are output through the display module 203; the display module 203 is used to display the evaluation information and the optimized configuration strategy.
其中,训练数据包括多个云主机的多个时段的资源采样数据;第一时段的起始时刻晚于第二时段的结束时刻;预测包括对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,第三时段的起始时刻晚于所述第一时段的结束时刻。Among them, the training data includes resource sampling data of multiple cloud hosts in multiple periods; the start time of the first period is later than the end time of the second period; the prediction includes the resource usage status of the cloud host to be evaluated in the third period and For the prediction of the trend of each resource usage, the start time of the third time period is later than the end time of the first time period.
可选的,上述处理模块202还用于:创建项目操作需求表,其中,项目操作需求表包括项目的预设完成时间、主机资源需求量和主机资源需求量对应的最优主机资源分配量;获取用户输入的目标项目信息,并对目标项目信息进行数据预处理,其中,数据预处理包括缺失值填补处理、去噪处理和数据标准化处理;将经过数据预处理的目标项目信息分成N组,通过反复迭代法对分成N组的目标项目信息进行多次重新分组,以获取最优分组方案;获取最优分组方案中各组目标项目信息的项目数据类型;对目标项目信息进行分析以获取第四关键信息;根据第四关键信息遍历项目操作需求表,以获取第四关键信息对应的 项目操作需求信息。Optionally, the aforementioned processing module 202 is further configured to: create a project operation demand table, where the project operation demand table includes the preset completion time of the project, the host resource demand, and the optimal host resource allocation corresponding to the host resource demand; Obtain the target project information input by the user, and perform data preprocessing on the target project information. The data preprocessing includes missing value filling processing, denoising processing and data standardization processing; the target project information after data preprocessing is divided into N groups, The target project information divided into N groups is regrouped many times by iterative method to obtain the optimal grouping plan; the project data type of each group of target project information in the optimal grouping plan is obtained; the target project information is analyzed to obtain the first Four key information: traverse the project operation demand table according to the fourth key information to obtain the project operation demand information corresponding to the fourth key information.
可选的,上述处理模块202还用于:通过资源监控模型,获取第一任务中的第一占用率信息和第一优先级,以及获取第二任务中的第二占用率信息和第二优先级;根据第一优先级,对第一任务进行分类,并标识第一类别标签,以及根据第二优先级,对第二任务进行分类,并标识第二类别标签;根据第一占用率信息,对标识第一类别标签的第一任务进行分类,并标识第三类别标签,以及第二占用率信息,对标识第二类别标签的第二任务进行分类,并标识第四类别标签;判断第一优先级是否符合第一预设采样条件和/或第一占用率信息是否符合第二预设采样条件,以及判断第二优先级是否符合第三预设采样条件和/或第二占用率信息是否符合第四预设采样条件;若第一优先级符合第一预设采样条件和/或第一占用率信息符合第二预设采样条件,则按照预设第一采样频率,对符合第一预设采样条件和/或第二预设采样条件的同一类别标签的任务进行采样,以获得第一采样数据;若第二优先级符合第三预设采样条件和/或第二占用率信息符合第四预设采样条件,则按照预设第二采样频率,对符合第三预设采样条件和/或第四预设采样条件的同一类别标签的任务进行采样,以获得第二采样数据。Optionally, the above-mentioned processing module 202 is further configured to: obtain the first occupancy rate information and the first priority in the first task, and obtain the second occupancy rate information and the second priority in the second task through the resource monitoring model According to the first priority, the first task is classified and the first category label is identified, and the second task is classified according to the second priority, and the second category label is identified; according to the first occupancy rate information, Classify the first task that identifies the first category label, identify the third category label, and second occupancy rate information, classify the second task that identifies the second category label, and identify the fourth category label; determine the first Whether the priority meets the first preset sampling condition and/or whether the first occupancy rate information meets the second preset sampling condition, and whether the second priority meets the third preset sampling condition and/or whether the second occupancy rate information Meet the fourth preset sampling condition; if the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling condition, then according to the preset first sampling frequency, the Set the sampling condition and/or the task of the same category label under the second preset sampling condition to sample to obtain the first sampling data; if the second priority meets the third preset sampling condition and/or the second occupancy rate information meets the first Four preset sampling conditions, according to the preset second sampling frequency, sampling tasks of the same category label that meet the third preset sampling condition and/or the fourth preset sampling condition are sampled to obtain second sampling data.
可选的,上述处理模块202在执行获得预测信息之后,还用于:获取预测信息的第一时间序列数据;对第一时间序列数据进行滑动窗口处理,以生成预设数量的具备预设长度的时序子序列;分析时序子序列的统计指标以获取统计特征信息,并以统计特征信息作为更新后的预测信息,其中,统计特征信息包括最大值、最小值、中位数、第一四分位数、第三四分位数、方差和标准差。Optionally, after the above-mentioned processing module 202 is executed to obtain the prediction information, it is further used to: obtain the first time series data of the prediction information; perform sliding window processing on the first time series data to generate a preset number of preset lengths. The time series subsequence; analyze the statistical indicators of the time series subsequence to obtain statistical feature information, and use the statistical feature information as the updated forecast information, where the statistical feature information includes the maximum value, the minimum value, the median, and the first quartile Digits, third quartile, variance and standard deviation.
可选的,上述处理模块202在执行根据第一采样数据和第二采样数据对待评估云主机进行预测之前,还用于:获取第一采样数据的第二时间序列数据,以及获取第二采样数据的第三时间序列数据;通过数据处理子模型中的指数加权移动平均法EWMA,分别对第二时间序列数据和第三时间序列数据进行求值,以获得第一平滑处理数据和第一平滑处理数据,对第二时间序列数据和第三时间序列数据的求值计算如下:
Figure PCTCN2019117048-appb-000006
其中,x t为时刻t的实际第二时间序列数据或实际第三时间序列数据,系数α为加权下降的速率,V t为t时刻的EWMA值;通过数据处理子模型中的马氏距离Mahalanobis算法,分别对第一平滑处理数据和第二平滑处理数据中的极度异常点进行检测与标识,并删除所标识的极度异常点,检测与标识极度异常点的计算如下:
Figure PCTCN2019117048-appb-000007
其中,
Figure PCTCN2019117048-appb-000008
是b与
Figure PCTCN2019117048-appb-000009
的距离,
Figure PCTCN2019117048-appb-000010
是平滑处理数据的均值向量,b是为平滑处理数据中的其他对象,S是协方差矩阵。
Optionally, the processing module 202 described above is further configured to: obtain the second time series data of the first sampled data, and obtain the second sampled data before performing the prediction based on the first sampled data and the second sampled data of the cloud host to be evaluated. The third time series data; the second time series data and the third time series data are evaluated by the exponentially weighted moving average method EWMA in the data processing sub-model to obtain the first smoothing data and the first smoothing Data, the evaluation of the second time series data and the third time series data is calculated as follows:
Figure PCTCN2019117048-appb-000006
Among them, x t is the actual second time series data or the actual third time series data at time t, the coefficient α is the rate of weighted decline, and V t is the EWMA value at time t; through the Mahalanobis distance in the data processing sub-model The algorithm detects and identifies the extreme abnormal points in the first smoothing data and the second smoothing data respectively, and deletes the marked extreme abnormal points. The calculation for detecting and identifying the extreme abnormal points is as follows:
Figure PCTCN2019117048-appb-000007
among them,
Figure PCTCN2019117048-appb-000008
Is b and
Figure PCTCN2019117048-appb-000009
the distance,
Figure PCTCN2019117048-appb-000010
Is the mean vector of the smoothed data, b is the other objects in the smoothed data, and S is the covariance matrix.
可选的,上述处理模块202还用于:获取第一采样数据中待评估资源的第一特征信息,以及获取第二采样数据中待评估资源的第二特征信息,其中,第一特征信息和第二特征信息均包括数据类型、资源总容量、各数据类型对应的资源的使用占比、资源的性能和特性;根据第一特征信息在资源数据库中获取与第一特征信息对应的第一资源数据,以及根据第二特征信息在资源数据库中获取与第二特征信息对应的第二资源数据;通过预设替换条件对第一资源数据进行计算和筛选以获取第一可替代资源,以及通过预设替换条件对第二资源数据进行计算和筛选以获取第二可替代资源;根据第一可替换资源,在待评估云主机之外的云主机中进行分析和匹配,以获取第一可替代云主机,并获取第一可替代云主机的第 一资源使用信息,并将第一可替代云主机和第一资源使用信息标记在第一可替换资源上,以获取最终的第一可替换资源,以及根据第二可替换资源,在待评估云主机之外的云主机中进行分析和匹配,以获取第二可替代云主机,并获取第二可替代云主机的第二资源使用信息,并将第二可替代云主机和第二资源使用信息标记在第二可替换资源上,以获取最终的第二可替换资源。Optionally, the above-mentioned processing module 202 is further configured to: obtain first characteristic information of the resource to be assessed in the first sampled data, and obtain second characteristic information of the resource to be assessed in the second sampled data, where the first characteristic information and The second feature information includes the data type, the total resource capacity, the proportion of resource usage corresponding to each data type, the performance and characteristics of the resource; the first resource corresponding to the first feature information is obtained from the resource database according to the first feature information Data, and obtain the second resource data corresponding to the second characteristic information in the resource database according to the second characteristic information; calculate and filter the first resource data through preset replacement conditions to obtain the first alternative resource, and obtain the first alternative resource through preset replacement conditions; Set replacement conditions to calculate and filter the second resource data to obtain the second alternative resource; according to the first alternative resource, perform analysis and matching in the cloud host other than the cloud host to be evaluated to obtain the first alternative cloud Host, and obtain the first resource usage information of the first alternative cloud host, and mark the first alternative cloud host and the first resource usage information on the first alternative resource to obtain the final first alternative resource, And according to the second alternative resource, perform analysis and matching among cloud hosts other than the cloud host to be evaluated to obtain the second alternative cloud host, and obtain the second resource usage information of the second alternative cloud host, and The second replaceable cloud host and the second resource usage information are marked on the second replaceable resource to obtain the final second replaceable resource.
可选的,上述处理模块202还用于:获取训练数据,对训练数据进行数据预处理;将经过平滑处理和异常数据处理的训练数据存储在训练数据库中,并设置配置文件,其中,配置文件包括网络结构、训练时长、训练与测试的比例安排、输出内容、优化学习率的设定、优化参数和存档规则设定;根据配置文件,对训练数据进行平滑处理,以得到预测信息;根据预设的综合评估规则对预测信息进行评估,以获得评估信息,其中,评估信息包括评分范围以及分析信息;根据评估信息生成与评估信息对应的优化配置策略,获得资源监控模型;通过已创建的检测脚本对资源监控模型进行准确性检测与性能测试;若准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值,则将资源监控模型作为最终的资源监控模型;若准确性检测的结果未达到第一预设阈值和/或性能测试的结果未达到第二预设阈值,则通过不断更新训练数据和修改预设的综合评估规则,并对资源监控模型进行再训练,直至准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值。Optionally, the above-mentioned processing module 202 is further used to: obtain training data, perform data preprocessing on the training data; store the training data that has undergone smoothing and abnormal data processing in a training database, and set a configuration file, where the configuration file Including network structure, training duration, training and testing ratio arrangement, output content, optimization learning rate setting, optimization parameter and archiving rule setting; according to the configuration file, the training data is smoothed to obtain prediction information; The comprehensive evaluation rules set up evaluate the forecast information to obtain the evaluation information. The evaluation information includes the scoring range and analysis information; according to the evaluation information, the optimized configuration strategy corresponding to the evaluation information is generated to obtain the resource monitoring model; and the created detection The script performs accuracy detection and performance testing on the resource monitoring model; if the accuracy detection result reaches the first preset threshold and the performance test result reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model; if If the result of the accuracy detection does not reach the first preset threshold and/or the result of the performance test does not reach the second preset threshold, the training data is continuously updated and the preset comprehensive evaluation rules are modified, and the resource monitoring model is retrained , Until the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold.
本申请实施例中,一方面,获取多角度的资源使用情况的数据,以便于用户快速而全面获悉待评估云主机的资源使用情况,另一方面,通过对多角度的数据进行评估以提高评估的严谨性和准确性,因而,本申请能够提高云主机的资源利用率。In the embodiments of this application, on the one hand, multi-angle resource usage data is obtained so that users can quickly and comprehensively learn the resource usage of the cloud host to be evaluated; on the other hand, multi-angle data is evaluated to improve the evaluation Because of its rigor and accuracy, this application can improve the resource utilization of the cloud host.
可选的,在本申请的一些实施方式中,上述评估云主机资源的方法的任一实施例或实施方式中所提及的技术特征也同样适用于本申请中的对执行上述评估云主机资源的方法的装置20,后续类似之处不再赘述。Optionally, in some embodiments of this application, the technical features mentioned in any embodiment or implementation of the method for evaluating cloud host resources are also applicable to the evaluation of cloud host resources in this application. For the device 20 of the method, the similarities will not be repeated here.
上面从模块化功能实体的角度分别介绍了本申请实施例中的装置20,以下从硬件角度介绍一种计算机装置,如图3所示,其包括:处理器、存储器、显示器、输入输出单元(也可以是收发器,图3中未标识出)以及存储在所述存储器中并可在所述处理器上运行的计算机程序。例如,该计算机程序可以为图1对应的实施例或图1对应的实施例中的任一可选实施例或可选实施方式中评估云主机资源的方法对应的程序。例如,当计算机装置实现如图2所示的装置20的功能时,所述处理器执行所述计算机程序时实现上述图2所对应的实施例中由装置20执行的评估云主机资源的方法中的各步骤;或者,所述处理器执行所述计算机程序时实现上述图2所对应的实施例的装置20中各模块的功能。又例如,该计算机程序可以为图1对应的实施例或图1对应的实施例中的任一可选实施例或可选实施方式的方法对应的程序。The device 20 in the embodiment of the present application is described above from the perspective of modular functional entities. The following describes a computer device from the perspective of hardware, as shown in FIG. 3, which includes: a processor, a memory, a display, and an input and output unit ( It may also be a transceiver (not identified in FIG. 3) and a computer program stored in the memory and running on the processor. For example, the computer program may be a program corresponding to the method for evaluating cloud host resources in the embodiment corresponding to FIG. 1 or any optional embodiment in the embodiment corresponding to FIG. 1 or the optional implementation manner. For example, when a computer device implements the function of the device 20 shown in FIG. 2, the processor executes the computer program to implement the method for evaluating cloud host resources executed by the device 20 in the embodiment corresponding to FIG. 2 Or, when the processor executes the computer program, the function of each module in the apparatus 20 of the embodiment corresponding to FIG. 2 is realized. For another example, the computer program may be a program corresponding to the method in the embodiment corresponding to FIG. 1 or any optional embodiment in the embodiment corresponding to FIG. 1 or the optional implementation manner.
所称处理器可以是中央处理单元(Central Processing Unit,CPU),还可以是其他通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等,所述处理器是所述计算机装置的控制中心,利用各种接口和线路连接整个计 算机装置的各个部分。The so-called processor can be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the computer device, and various interfaces and lines are used to connect various parts of the entire computer device.
所述存储器可用于存储所述计算机程序和/或模块,所述处理器通过运行或执行存储在所述存储器内的计算机程序和/或模块,以及调用存储在存储器内的数据,实现所述计算机装置的各种功能。所述存储器可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需的应用程序(比如确定待评估云主机以及待评估云主机的待评估资源等)等;存储数据区可存储根据手机的使用所创建的数据(比如获取训练数据等)等。此外,存储器可以包括高速随机存取存储器,还可以包括非易失性存储器,例如硬盘、内存、插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。The memory may be used to store the computer program and/or module, and the processor implements the computer by running or executing the computer program and/or module stored in the memory and calling data stored in the memory. Various functions of the device. The memory may mainly include a storage program area and a storage data area, where the storage program area can store an operating system and at least one application program required by a function (such as determining the cloud host to be evaluated and the resource to be evaluated of the cloud host to be evaluated, etc.) Etc.; the data storage area can store data created based on the use of the mobile phone (such as obtaining training data, etc.), etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, memory, plug-in hard disks, smart media cards (SMC), and secure digital (SD) cards. , Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
所述输入输出单元也可以用接收器和发送器代替,可以为相同或者不同的物理实体。为相同的物理实体时,可以统称为输入输出单元。该输入输出单元可以为收发器。The input and output units can also be replaced by receivers and transmitters, and they can be the same or different physical entities. When they are the same physical entity, they can be collectively referred to as input and output units. The input and output unit may be a transceiver.
所述存储器可以集成在所述处理器中,也可以与所述处理器分开设置。The memory may be integrated in the processor, or may be provided separately from the processor.
本申请还提供一种计算机可读存储介质,该计算机可读存储介质可以为非易失性计算机可读存储介质,也可以为易失性计算机可读存储介质。计算机可读存储介质存储有计算机指令,当所述计算机指令在计算机上运行时,使得计算机执行如下步骤:The present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer executes the following steps:
获取训练数据,将训练数据输入至神经网络模型,并对神经网络模型进行训练,以获得资源监控模型,其中,训练数据包括多个云主机的多个时段的资源采样数据;获取输入的目标项目信息,并对目标项目信息进行分析,以获取目标项目信息的项目数据类型和项目操作需求信息;获取云主机的使用资源类型和使用资源情况,并根据使用资源类型、使用资源情况、项目数据类型和项目操作需求信息,确定待评估云主机以及待评估云主机的待评估资源;通过资源监控模型获取待评估资源在第一时段内的第一采样数据,以及获取待评估资源在第二时段内的第二采样数据,其中,第一时段的起始时刻晚于第二时段的结束时刻;根据第一采样数据和第二采样数据对待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将第三预测数据作为主信息,以及将第一预测数据和第二预测数据作为辅助信息,以获取预测信息,其中,预测包括对待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,第三时段的起始时刻晚于第一时段的结束时刻;分别对第一采样数据、第二采样数据和预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取第一采样数据中待评估资源的第一可替代资源,和获取第二采样数据中待评估资源的第二可替代资源;根据第一关键信息、第二关键信息、第三关键信息、第一可替代资源和第二可替代资源对待评估云主机进行评估,以获取评估信息;根据评估信息生成与评估信息对应的优化配置策略,并输出评估信息和优化配置策略。Obtain training data, input the training data into the neural network model, and train the neural network model to obtain a resource monitoring model, where the training data includes resource sampling data of multiple cloud hosts for multiple periods; obtain the input target item Information, and analyze the target project information to obtain the project data type and project operation requirement information of the target project information; obtain the resource type and resource usage of the cloud host, and according to the resource type, resource usage, and project data type And project operation requirements information, determine the cloud host to be evaluated and the resource to be evaluated; obtain the first sampling data of the resource to be evaluated in the first period through the resource monitoring model, and obtain the resource to be evaluated in the second period The second sampling data of the first time period is later than the end time of the second time period; the cloud host to be evaluated is predicted based on the first sampling data and the second sampling data to obtain the first prediction data and the first prediction data, respectively. The second prediction data and the third prediction data, and the third prediction data is used as the main information, and the first prediction data and the second prediction data are used as auxiliary information to obtain the prediction information, where the prediction includes the cloud host to be evaluated in the third To predict the usage status of each resource and the trend of each resource usage in a time period, the start time of the third time period is later than the end time of the first time period; feature extraction is performed on the first sampling data, the second sampling data and the prediction information respectively, To obtain the first key information, the second key information, and the third key information respectively, and obtain the first alternative resource of the resource to be evaluated in the first sampling data, and obtain the second alternative resource of the resource to be evaluated in the second sampling data Resources; evaluate the cloud host to be evaluated based on the first key information, second key information, third key information, first alternative resource, and second alternative resource to obtain the evaluation information; generate the corresponding evaluation information based on the evaluation information Optimize the configuration strategy, and output the evaluation information and optimize the configuration strategy.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

Claims (20)

  1. 一种评估云主机资源的方法,所述方法包括:A method for evaluating cloud host resources, the method comprising:
    获取训练数据,将所述训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;Obtain training data, input the training data into a neural network model, and train the neural network model to obtain a resource monitoring model, wherein the training data includes resource sampling data of multiple cloud hosts for multiple periods ;
    获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;Obtaining the input target project information, and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information;
    获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;Obtain the used resource type and used resource situation of the cloud host, and determine the cloud host to be assessed and the cloud to be assessed based on the used resource type, the used resource situation, the project data type, and the project operation requirement information The host's resources to be evaluated;
    通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;Obtain the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and acquire the second sampling data of the resource to be evaluated in the second time period, where the beginning of the first time period The start time is later than the end time of the second time period;
    根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;According to the first sampling data and the second sampling data, the cloud host to be evaluated is predicted to obtain the first prediction data, the second prediction data, and the third prediction data, respectively, and the third prediction data As main information, and using the first prediction data and the second prediction data as auxiliary information to obtain prediction information, wherein the prediction includes the status of each resource usage of the cloud host to be evaluated in the third period And prediction of the trend of each resource usage, the start time of the third time period is later than the end time of the first time period;
    分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;Perform feature extraction on the first sampling data, the second sampling data, and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and obtain the first sampling data The first alternative resource of the resource to be evaluated in the second sample data, and the second alternative resource of the resource to be evaluated in the second sampling data;
    根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;Evaluate the cloud host to be evaluated according to the first key information, the second key information, the third key information, the first replaceable resource, and the second replaceable resource to obtain the evaluation information;
    根据所述评估信息生成与所述评估信息对应的优化配置策略,并输出所述评估信息和所述优化配置策略。According to the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated, and the evaluation information and the optimized configuration strategy are output.
  2. 根据权利要求1所述的方法,所述获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息,包括:The method according to claim 1, said obtaining the input target project information and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information, comprising:
    创建项目操作需求表,其中,所述项目操作需求表包括项目的预设完成时间、主机资源需求量和所述主机资源需求量对应的最优主机资源分配量;Creating a project operation requirement table, where the project operation requirement table includes the preset completion time of the project, the host resource demand, and the optimal host resource allocation corresponding to the host resource demand;
    获取用户输入的目标项目信息,并对所述目标项目信息进行数据预处理,其中,所述数据预处理包括缺失值填补处理、去噪处理和数据标准化处理;Acquiring target item information input by the user, and performing data preprocessing on the target item information, wherein the data preprocessing includes missing value filling processing, denoising processing, and data standardization processing;
    将经过数据预处理的目标项目信息分成N组,通过反复迭代法对分成N组的目标项目信息进行多次重新分组,以获取最优分组方案;Divide the target item information that has undergone data preprocessing into N groups, and regroup the target item information divided into N groups through repeated iterations to obtain the optimal grouping scheme;
    获取所述最优分组方案中各组目标项目信息的项目数据类型;Acquiring the project data type of each group of target project information in the optimal grouping scheme;
    对所述目标项目信息进行分析以获取第四关键信息;Analyze the target item information to obtain fourth key information;
    根据所述第四关键信息遍历所述项目操作需求表,以获取所述第四关键信息对应的项目操作需求信息。Traverse the project operation requirement table according to the fourth key information to obtain project operation requirement information corresponding to the fourth key information.
  3. 根据权利要求1所述的方法,所述待评估资源包括在第一时段内的第一任务和在第二时段内的第二任务,所述通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,包括:The method according to claim 1, wherein the resource to be evaluated includes a first task in a first time period and a second task in a second time period, and the resource to be evaluated is acquired through the resource monitoring model. The first sampling data in the first time period and obtaining the second sampling data of the resource to be evaluated in the second time period include:
    通过所述资源监控模型,获取所述第一任务中的第一占用率信息和第一优先级,以及获取所述第二任务中的第二占用率信息和第二优先级;Obtaining the first occupancy rate information and the first priority in the first task, and obtaining the second occupancy rate information and the second priority in the second task through the resource monitoring model;
    根据所述第一优先级,对所述第一任务进行分类,并标识第一类别标签,以及根据所述第二优先级,对所述第二任务进行分类,并标识第二类别标签;Classify the first task according to the first priority, and identify a first category label, and classify the second task according to the second priority, and identify a second category label;
    根据所述第一占用率信息,对标识所述第一类别标签的第一任务进行分类,并标识第三类别标签,以及所述第二占用率信息,对标识所述第二类别标签的第二任务进行分类,并标识第四类别标签;According to the first occupancy rate information, the first task that identifies the first category label is classified, and the third category label is identified, and the second occupancy rate information is used to identify the first task that identifies the second category label. Two tasks are classified and the fourth category label is identified;
    判断所述第一优先级是否符合第一预设采样条件和/或所述第一占用率信息是否符合第二预设采样条件,以及判断所述第二优先级是否符合第三预设采样条件和/或所述第二占用率信息是否符合第四预设采样条件;Determine whether the first priority meets a first preset sampling condition and/or whether the first occupancy rate information meets a second preset sampling condition, and determine whether the second priority meets a third preset sampling condition And/or whether the second occupancy rate information meets the fourth preset sampling condition;
    若所述第一优先级符合第一预设采样条件和/或所述第一占用率信息符合第二预设采样条件,则按照预设第一采样频率,对符合第一预设采样条件和/或第二预设采样条件的同一类别标签的任务进行采样,以获得第一采样数据;If the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling condition, then according to the preset first sampling frequency, the pair meets the first preset sampling condition and / Or the tasks of the same category label under the second preset sampling conditions are sampled to obtain the first sampling data;
    若所述第二优先级符合第三预设采样条件和/或所述第二占用率信息符合第四预设采样条件,则按照预设第二采样频率,对符合第三预设采样条件和/或第四预设采样条件的同一类别标签的任务进行采样,以获得第二采样数据。If the second priority level meets the third preset sampling condition and/or the second occupancy rate information meets the fourth preset sampling condition, then according to the preset second sampling frequency, the pair meets the third preset sampling condition and /Or the tasks of the same category label under the fourth preset sampling condition are sampled to obtain the second sampling data.
  4. 根据权利要求1所述的方法,所述获取预测信息之后,所述方法还包括:The method according to claim 1, after said obtaining the prediction information, the method further comprises:
    获取所述预测信息的第一时间序列数据;Acquiring the first time series data of the forecast information;
    对所述第一时间序列数据进行滑动窗口处理,以生成预设数量的具备预设长度的时序子序列;Performing sliding window processing on the first time series data to generate a preset number of time series subsequences with a preset length;
    分析所述时序子序列的统计指标以获取统计特征信息,并以所述统计特征信息作为更新后的预测信息,其中,所述统计特征信息包括最大值、最小值、中位数、第一四分位数、第三四分位数、方差和标准差。Analyze the statistical indicators of the time series subsequence to obtain statistical feature information, and use the statistical feature information as updated forecast information, where the statistical feature information includes the maximum value, the minimum value, the median, the first four Quantile, third quartile, variance and standard deviation.
  5. 根据权利要求1所述的方法,所述根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测之前,所述方法还包括:The method according to claim 1, before predicting the cloud host to be evaluated based on the first sampling data and the second sampling data, the method further comprises:
    获取所述第一采样数据的第二时间序列数据,以及获取所述第二采样数据的第三时间序列数据;Acquiring second time series data of the first sampling data, and acquiring third time series data of the second sampling data;
    通过所述数据处理子模型中的指数加权移动平均法EWMA,分别对所述第二时间序列数据和所述第三时间序列数据进行求值,以获得第一平滑处理数据和第一平滑处理数据,对所述第二时间序列数据和所述第三时间序列数据的求值计算如下:Through the exponentially weighted moving average method EWMA in the data processing sub-model, the second time series data and the third time series data are evaluated respectively to obtain the first smoothed data and the first smoothed data , The evaluation of the second time series data and the third time series data is calculated as follows:
    Figure PCTCN2019117048-appb-100001
    Figure PCTCN2019117048-appb-100001
    其中,x t为时刻t的实际第二时间序列数据或实际第三时间序列数据,系数α为加权下降的速率,V t为t时刻的EWMA值; Where, x t is the actual second time series data or the actual third time series data at time t, the coefficient α is the rate of weighted decline, and V t is the EWMA value at time t;
    通过所述数据处理子模型中的马氏距离Mahalanobis算法,分别对所述第一平滑处理 数据和所述第二平滑处理数据中的极度异常点进行检测与标识,并删除所标识的极度异常点,检测与标识所述极度异常点的计算如下:Through the Mahalanobis algorithm in the data processing sub-model, the extreme abnormal points in the first smoothing data and the second smoothing data are detected and identified, and the identified extreme abnormal points are deleted , The calculation for detecting and identifying the extreme abnormal point is as follows:
    Figure PCTCN2019117048-appb-100002
    Figure PCTCN2019117048-appb-100002
    其中,
    Figure PCTCN2019117048-appb-100003
    是b与
    Figure PCTCN2019117048-appb-100004
    的距离,
    Figure PCTCN2019117048-appb-100005
    是所述平滑处理数据的均值向量,b是为所述平滑处理数据中的其他对象,S是协方差矩阵。
    among them,
    Figure PCTCN2019117048-appb-100003
    Is b and
    Figure PCTCN2019117048-appb-100004
    the distance,
    Figure PCTCN2019117048-appb-100005
    Is the mean vector of the smoothed data, b is other objects in the smoothed data, and S is the covariance matrix.
  6. 根据权利要求1所述的方法,所述方法包括资源数据库,所述资源数据库包括收集的多种数据类型对应的资源数据,所述获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源,包括:The method according to claim 1, wherein the method includes a resource database, the resource database includes resource data corresponding to multiple data types collected, and the acquiring of the first alternative of the resource to be evaluated in the first sampling data The resource, and the second alternative resource that obtains the resource to be evaluated in the second sampling data includes:
    获取所述第一采样数据中待评估资源的第一特征信息,以及获取所述第二采样数据中待评估资源的第二特征信息,其中,所述第一特征信息和所述第二特征信息均包括数据类型、资源总容量、各数据类型对应的资源的使用占比、资源的性能和特性;Acquire first characteristic information of the resource to be evaluated in the first sampled data, and acquire second characteristic information of the resource to be evaluated in the second sampled data, wherein the first characteristic information and the second characteristic information All include data type, total resource capacity, usage proportion of resources corresponding to each data type, performance and characteristics of resources;
    根据所述第一特征信息在所述资源数据库中获取与所述第一特征信息对应的第一资源数据,以及根据所述第二特征信息在所述资源数据库中获取与所述第二特征信息对应的第二资源数据;Acquire first resource data corresponding to the first feature information in the resource database according to the first feature information, and obtain the second feature information from the resource database according to the second feature information Corresponding second resource data;
    通过预设替换条件对所述第一资源数据进行计算和筛选以获取第一可替代资源,以及通过所述预设替换条件对所述第二资源数据进行计算和筛选以获取第二可替代资源;Calculate and filter the first resource data based on a preset replacement condition to obtain a first alternative resource, and perform calculation and screening on the second resource data based on the preset replacement condition to obtain a second alternative resource ;
    根据所述第一可替换资源,在所述待评估云主机之外的云主机中进行分析和匹配,以获取第一可替代云主机,并获取所述第一可替代云主机的第一资源使用信息,并将所述第一可替代云主机和所述第一资源使用信息标记在所述第一可替换资源上,以获取最终的第一可替换资源,以及根据所述第二可替换资源,在所述待评估云主机之外的云主机中进行分析和匹配,以获取第二可替代云主机,并获取所述第二可替代云主机的第二资源使用信息,并将所述第二可替代云主机和所述第二资源使用信息标记在所述第二可替换资源上,以获取最终的第二可替换资源。According to the first replaceable resource, perform analysis and matching among cloud hosts other than the cloud host to be evaluated, to obtain a first replaceable cloud host, and obtain the first resource of the first replaceable cloud host Use information, and mark the first replaceable cloud host and the first resource use information on the first replaceable resource to obtain the final first replaceable resource, and according to the second replaceable resource Resources, perform analysis and matching in cloud hosts other than the cloud host to be evaluated to obtain a second alternative cloud host, and obtain second resource usage information of the second alternative cloud host, and compare the The second replaceable cloud host and the second resource usage information are marked on the second replaceable resource to obtain the final second replaceable resource.
  7. 根据权利要求1所述的方法,所述获取训练数据,将所述训练数据输入至神经网络模型,对所述神经网络模型进行训练,以获得资源监控模型,包括:The method according to claim 1, wherein the acquiring training data, inputting the training data to a neural network model, and training the neural network model to obtain a resource monitoring model includes:
    获取训练数据,对所述训练数据进行数据预处理;Acquiring training data, and performing data preprocessing on the training data;
    将经过平滑处理和异常数据处理的训练数据存储在训练数据库中,并设置配置文件,其中,所述配置文件包括网络结构、训练时长、训练与测试的比例安排、输出内容、优化学习率的设定、优化参数和存档规则设定;The training data that has been smoothed and processed with abnormal data is stored in the training database, and a configuration file is set. The configuration file includes the network structure, the training duration, the ratio of training and testing, the output content, and the settings for optimizing the learning rate. Setting and optimizing parameters and archiving rule settings;
    根据所述配置文件,对所述训练数据进行平滑处理,以得到预测信息;Smoothing the training data according to the configuration file to obtain prediction information;
    根据预设的综合评估规则对所述预测信息进行评估,以获得评估信息,其中,所述评估信息包括评分范围以及分析信息;Evaluate the prediction information according to a preset comprehensive evaluation rule to obtain evaluation information, where the evaluation information includes a scoring range and analysis information;
    根据所述评估信息生成与所述评估信息对应的优化配置策略,获得资源监控模型;Generating an optimized configuration strategy corresponding to the evaluation information according to the evaluation information to obtain a resource monitoring model;
    通过已创建的检测脚本对所述资源监控模型进行准确性检测与性能测试;Perform accuracy detection and performance testing on the resource monitoring model through the created detection script;
    若准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值,则将所述资源监控模型作为最终的资源监控模型;If the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model;
    若准确性检测的结果未达到第一预设阈值和/或性能测试的结果未达到第二预设阈值, 则通过不断更新所述训练数据和修改所述预设的综合评估规则,并对所述资源监控模型进行再训练,直至准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值。If the result of the accuracy detection does not reach the first preset threshold and/or the result of the performance test does not reach the second preset threshold, then by continuously updating the training data and modifying the preset comprehensive evaluation rules, The resource monitoring model is retrained until the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold.
  8. 一种评估云主机资源的装置,所述装置包括:A device for evaluating resources of a cloud host, the device comprising:
    输入输出模块,用于获取训练数据,用于获取输入的目标项目信息;Input and output module, used to obtain training data, used to obtain input target item information;
    处理模块,用于将所述输入输出模块获取的训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;并对所述输入输出模块获取的输入的目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;根据所述评估信息生成与所述评估信息对应的优化配置策略,将所述评估信息和所述优化配置策略至显示模块,并通过所述显示模块输出所述评估信息和所述优化配置策略;The processing module is used to input the training data obtained by the input and output module into a neural network model, and train the neural network model to obtain a resource monitoring model, wherein the training data includes multiple cloud hosts Resource sampling data for a period of time; analyze the input target project information obtained by the input and output module to obtain the project data type and project operation requirement information of the target project information; obtain the resource type and usage of the cloud host Resource situation, and determine the cloud host to be evaluated and the resource to be evaluated of the cloud host to be evaluated according to the resource usage type, the resource usage situation, the project data type, and the project operation requirement information; The resource monitoring model acquires the first sampling data of the resource to be evaluated in the first time period, and acquires the second sampling data of the resource to be evaluated in the second time period, wherein the start time of the first time period is later At the end of the second time period; predict the cloud host to be evaluated based on the first sampling data and the second sampling data to obtain the first prediction data, the second prediction data, and the third prediction, respectively Data, the third prediction data is used as the main information, and the first prediction data and the second prediction data are used as auxiliary information to obtain prediction information, wherein the prediction includes The host uses the state of each resource and the forecast of the trend of each resource usage in the third time period, the start time of the third time period is later than the end time of the first time period; Perform feature extraction on the second sampling data and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and obtain the first alternative resource of the resource to be evaluated in the first sampling data , And acquiring the second alternative resource of the resource to be evaluated in the second sampling data; according to the first key information, the second key information, the third key information, the first alternative resource and The second alternative resource evaluates the cloud host to be evaluated to obtain evaluation information; generates an optimized configuration strategy corresponding to the evaluation information according to the evaluation information, and compares the evaluation information with the optimized configuration strategy To a display module, and output the evaluation information and the optimal configuration strategy through the display module;
    所述显示模块,用于显示所述评估信息和所述优化配置策略。The display module is used to display the evaluation information and the optimal configuration strategy.
  9. 根据权利要求8所述的装置,所述处理模块还用于:According to the device of claim 8, the processing module is further configured to:
    创建项目操作需求表,其中,所述项目操作需求表包括项目的预设完成时间、主机资源需求量和所述主机资源需求量对应的最优主机资源分配量;Creating a project operation requirement table, where the project operation requirement table includes the preset completion time of the project, the host resource demand, and the optimal host resource allocation corresponding to the host resource demand;
    获取用户输入的目标项目信息,并对所述目标项目信息进行数据预处理,其中,所述数据预处理包括缺失值填补处理、去噪处理和数据标准化处理;Acquiring target item information input by the user, and performing data preprocessing on the target item information, wherein the data preprocessing includes missing value filling processing, denoising processing, and data standardization processing;
    将经过数据预处理的目标项目信息分成N组,通过反复迭代法对分成N组的目标项目信息进行多次重新分组,以获取最优分组方案;Divide the target item information that has undergone data preprocessing into N groups, and regroup the target item information divided into N groups through repeated iterations to obtain the optimal grouping scheme;
    获取所述最优分组方案中各组目标项目信息的项目数据类型;Acquiring the project data type of each group of target project information in the optimal grouping scheme;
    对所述目标项目信息进行分析以获取第四关键信息;Analyze the target item information to obtain fourth key information;
    根据所述第四关键信息遍历所述项目操作需求表,以获取所述第四关键信息对应的项目操作需求信息。Traverse the project operation requirement table according to the fourth key information to obtain project operation requirement information corresponding to the fourth key information.
  10. 根据权利要求8所述的装置,所述处理模块还用于:According to the device of claim 8, the processing module is further configured to:
    通过所述资源监控模型,获取所述第一任务中的第一占用率信息和第一优先级,以及获取所述第二任务中的第二占用率信息和第二优先级;Obtaining the first occupancy rate information and the first priority in the first task, and obtaining the second occupancy rate information and the second priority in the second task through the resource monitoring model;
    根据所述第一优先级,对所述第一任务进行分类,并标识第一类别标签,以及根据所述第二优先级,对所述第二任务进行分类,并标识第二类别标签;Classify the first task according to the first priority, and identify a first category label, and classify the second task according to the second priority, and identify a second category label;
    根据所述第一占用率信息,对标识所述第一类别标签的第一任务进行分类,并标识第三类别标签,以及所述第二占用率信息,对标识所述第二类别标签的第二任务进行分类,并标识第四类别标签;According to the first occupancy rate information, the first task that identifies the first category label is classified, and the third category label is identified, and the second occupancy rate information is used to identify the first task that identifies the second category label. Two tasks are classified and the fourth category label is identified;
    判断所述第一优先级是否符合第一预设采样条件和/或所述第一占用率信息是否符合第二预设采样条件,以及判断所述第二优先级是否符合第三预设采样条件和/或所述第二占用率信息是否符合第四预设采样条件;Determine whether the first priority meets a first preset sampling condition and/or whether the first occupancy rate information meets a second preset sampling condition, and determine whether the second priority meets a third preset sampling condition And/or whether the second occupancy rate information meets the fourth preset sampling condition;
    若所述第一优先级符合第一预设采样条件和/或所述第一占用率信息符合第二预设采样条件,则按照预设第一采样频率,对符合第一预设采样条件和/或第二预设采样条件的同一类别标签的任务进行采样,以获得第一采样数据;If the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling condition, then according to the preset first sampling frequency, the pair meets the first preset sampling condition and / Or the tasks of the same category label under the second preset sampling conditions are sampled to obtain the first sampling data;
    若所述第二优先级符合第三预设采样条件和/或所述第二占用率信息符合第四预设采样条件,则按照预设第二采样频率,对符合第三预设采样条件和/或第四预设采样条件的同一类别标签的任务进行采样,以获得第二采样数据。If the second priority level meets the third preset sampling condition and/or the second occupancy rate information meets the fourth preset sampling condition, then according to the preset second sampling frequency, the pair meets the third preset sampling condition and /Or the tasks of the same category label under the fourth preset sampling condition are sampled to obtain the second sampling data.
  11. 根据权利要求8所述的装置,所述处理模块在执行所述获得预测信息之后,还用于:The device according to claim 8, after the processing module executes the obtaining prediction information, it is further configured to:
    获取所述预测信息的第一时间序列数据;Acquiring the first time series data of the forecast information;
    对所述第一时间序列数据进行滑动窗口处理,以生成预设数量的具备预设长度的时序子序列;Performing sliding window processing on the first time series data to generate a preset number of time series subsequences with a preset length;
    分析所述时序子序列的统计指标以获取统计特征信息,并以所述统计特征信息作为更新后的预测信息,其中,所述统计特征信息包括最大值、最小值、中位数、第一四分位数、第三四分位数、方差和标准差。Analyze the statistical indicators of the time series subsequence to obtain statistical feature information, and use the statistical feature information as updated forecast information, where the statistical feature information includes the maximum value, the minimum value, the median, the first four Quantile, third quartile, variance and standard deviation.
  12. 根据权利要求8所述的装置,所述处理模块在执行所述根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测之前,还用于:The device according to claim 8, before executing the prediction of the cloud host to be evaluated based on the first sampling data and the second sampling data, the processing module is further configured to:
    获取所述第一采样数据的第二时间序列数据,以及获取所述第二采样数据的第三时间序列数据;Acquiring second time series data of the first sampling data, and acquiring third time series data of the second sampling data;
    通过所述数据处理子模型中的指数加权移动平均法EWMA,分别对所述第二时间序列数据和所述第三时间序列数据进行求值,以获得第一平滑处理数据和第一平滑处理数据,对所述第二时间序列数据和所述第三时间序列数据的求值计算如下:Through the exponentially weighted moving average method EWMA in the data processing sub-model, the second time series data and the third time series data are evaluated respectively to obtain the first smoothed data and the first smoothed data , The evaluation of the second time series data and the third time series data is calculated as follows:
    Figure PCTCN2019117048-appb-100006
    Figure PCTCN2019117048-appb-100006
    其中,x t为时刻t的实际第二时间序列数据或实际第三时间序列数据,系数α为加权下降的速率,V t为t时刻的EWMA值; Where, x t is the actual second time series data or the actual third time series data at time t, the coefficient α is the rate of weighted decline, and V t is the EWMA value at time t;
    通过所述数据处理子模型中的马氏距离Mahalanobis算法,分别对所述第一平滑处理数据和所述第二平滑处理数据中的极度异常点进行检测与标识,并删除所标识的极度异常 点,检测与标识所述极度异常点的计算如下:Through the Mahalanobis algorithm in the data processing sub-model, the extreme abnormal points in the first smoothing data and the second smoothing data are detected and identified, and the identified extreme abnormal points are deleted , The calculation for detecting and identifying the extreme abnormal point is as follows:
    Figure PCTCN2019117048-appb-100007
    Figure PCTCN2019117048-appb-100007
    其中,
    Figure PCTCN2019117048-appb-100008
    是b与
    Figure PCTCN2019117048-appb-100009
    的距离,
    Figure PCTCN2019117048-appb-100010
    是所述平滑处理数据的均值向量,b是为所述平滑处理数据中的其他对象,S是协方差矩阵。
    among them,
    Figure PCTCN2019117048-appb-100008
    Is b and
    Figure PCTCN2019117048-appb-100009
    the distance,
    Figure PCTCN2019117048-appb-100010
    Is the mean vector of the smoothed data, b is other objects in the smoothed data, and S is the covariance matrix.
  13. 根据权利要求8所述的装置,所述处理模块还用于:According to the device of claim 8, the processing module is further configured to:
    获取所述第一采样数据中待评估资源的第一特征信息,以及获取所述第二采样数据中待评估资源的第二特征信息,其中,所述第一特征信息和所述第二特征信息均包括数据类型、资源总容量、各数据类型对应的资源的使用占比、资源的性能和特性;Acquire first characteristic information of the resource to be evaluated in the first sampled data, and acquire second characteristic information of the resource to be evaluated in the second sampled data, wherein the first characteristic information and the second characteristic information All include data type, total resource capacity, usage proportion of resources corresponding to each data type, performance and characteristics of resources;
    根据所述第一特征信息在所述资源数据库中获取与所述第一特征信息对应的第一资源数据,以及根据所述第二特征信息在所述资源数据库中获取与所述第二特征信息对应的第二资源数据;Acquire first resource data corresponding to the first feature information in the resource database according to the first feature information, and obtain the second feature information from the resource database according to the second feature information Corresponding second resource data;
    通过预设替换条件对所述第一资源数据进行计算和筛选以获取第一可替代资源,以及通过所述预设替换条件对所述第二资源数据进行计算和筛选以获取第二可替代资源;Calculate and filter the first resource data based on a preset replacement condition to obtain a first alternative resource, and perform calculation and screening on the second resource data based on the preset replacement condition to obtain a second alternative resource ;
    根据所述第一可替换资源,在所述待评估云主机之外的云主机中进行分析和匹配,以获取第一可替代云主机,并获取所述第一可替代云主机的第一资源使用信息,并将所述第一可替代云主机和所述第一资源使用信息标记在所述第一可替换资源上,以获取最终的第一可替换资源,以及根据所述第二可替换资源,在所述待评估云主机之外的云主机中进行分析和匹配,以获取第二可替代云主机,并获取所述第二可替代云主机的第二资源使用信息,并将所述第二可替代云主机和所述第二资源使用信息标记在所述第二可替换资源上,以获取最终的第二可替换资源。According to the first replaceable resource, perform analysis and matching among cloud hosts other than the cloud host to be evaluated, to obtain a first replaceable cloud host, and obtain the first resource of the first replaceable cloud host Use information, and mark the first replaceable cloud host and the first resource use information on the first replaceable resource to obtain the final first replaceable resource, and according to the second replaceable resource Resources, perform analysis and matching in cloud hosts other than the cloud host to be evaluated to obtain a second alternative cloud host, and obtain second resource usage information of the second alternative cloud host, and compare the The second replaceable cloud host and the second resource usage information are marked on the second replaceable resource to obtain the final second replaceable resource.
  14. 根据权利要求8所述的装置,所述处理模块还用于:According to the device of claim 8, the processing module is further configured to:
    获取训练数据,对所述训练数据进行数据预处理;Acquiring training data, and performing data preprocessing on the training data;
    将经过平滑处理和异常数据处理的训练数据存储在训练数据库中,并设置配置文件,其中,所述配置文件包括网络结构、训练时长、训练与测试的比例安排、输出内容、优化学习率的设定、优化参数和存档规则设定;The training data that has been smoothed and processed with abnormal data is stored in the training database, and a configuration file is set. The configuration file includes the network structure, the training duration, the ratio of training and testing, the output content, and the settings for optimizing the learning rate. Setting and optimizing parameters and archiving rule settings;
    根据所述配置文件,对所述训练数据进行平滑处理,以得到预测信息;Smoothing the training data according to the configuration file to obtain prediction information;
    根据预设的综合评估规则对所述预测信息进行评估,以获得评估信息,其中,所述评估信息包括评分范围以及分析信息;Evaluate the prediction information according to a preset comprehensive evaluation rule to obtain evaluation information, where the evaluation information includes a scoring range and analysis information;
    根据所述评估信息生成与所述评估信息对应的优化配置策略,获得资源监控模型;Generating an optimized configuration strategy corresponding to the evaluation information according to the evaluation information to obtain a resource monitoring model;
    通过已创建的检测脚本对所述资源监控模型进行准确性检测与性能测试;Perform accuracy detection and performance testing on the resource monitoring model through the created detection script;
    若准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值,则将所述资源监控模型作为最终的资源监控模型;If the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold, the resource monitoring model is used as the final resource monitoring model;
    若准确性检测的结果未达到第一预设阈值和/或性能测试的结果未达到第二预设阈值,则通过不断更新所述训练数据和修改所述预设的综合评估规则,并对所述资源监控模型进行再训练,直至准确性检测的结果达到第一预设阈值以及性能测试的结果达到第二预设阈值。If the result of the accuracy detection does not reach the first preset threshold and/or the result of the performance test does not reach the second preset threshold, then by continuously updating the training data and modifying the preset comprehensive evaluation rules, The resource monitoring model is retrained until the result of the accuracy detection reaches the first preset threshold and the result of the performance test reaches the second preset threshold.
  15. 一种评估云主机资源的设备,包括存储器、处理器及存储在所述存储器上并可在 所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如下步骤:A device for evaluating cloud host resources includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor implements the following steps when the processor executes the computer program:
    获取训练数据,将所述训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;Obtain training data, input the training data into a neural network model, and train the neural network model to obtain a resource monitoring model, wherein the training data includes resource sampling data of multiple cloud hosts for multiple periods ;
    获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;Obtaining the input target project information, and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information;
    获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;Obtain the used resource type and used resource situation of the cloud host, and determine the cloud host to be assessed and the cloud to be assessed based on the used resource type, the used resource situation, the project data type, and the project operation requirement information The host's resources to be evaluated;
    通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;Obtain the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and acquire the second sampling data of the resource to be evaluated in the second time period, where the beginning of the first time period The start time is later than the end time of the second time period;
    根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;According to the first sampling data and the second sampling data, the cloud host to be evaluated is predicted to obtain the first prediction data, the second prediction data, and the third prediction data, respectively, and the third prediction data As main information, and using the first prediction data and the second prediction data as auxiliary information to obtain prediction information, wherein the prediction includes the status of each resource usage of the cloud host to be evaluated in the third period And prediction of the trend of each resource usage, the start time of the third time period is later than the end time of the first time period;
    分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;Perform feature extraction on the first sampling data, the second sampling data, and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and obtain the first sampling data The first alternative resource of the resource to be evaluated in the second sample data, and the second alternative resource of the resource to be evaluated in the second sampling data;
    根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;Evaluate the cloud host to be evaluated according to the first key information, the second key information, the third key information, the first replaceable resource, and the second replaceable resource to obtain the evaluation information;
    根据所述评估信息生成与所述评估信息对应的优化配置策略,并输出所述评估信息和所述优化配置策略。According to the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated, and the evaluation information and the optimized configuration strategy are output.
  16. 根据权利要求15所述的设备,所述处理器执行所述计算机程序实现所述获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息时,包括以下步骤:The device according to claim 15, wherein the processor executes the computer program to realize the acquisition of the input target item information, and analyzes the target item information to obtain the item data type and the item data type of the target item information. When project operation requires information, it includes the following steps:
    创建项目操作需求表,其中,所述项目操作需求表包括项目的预设完成时间、主机资源需求量和所述主机资源需求量对应的最优主机资源分配量;Creating a project operation requirement table, where the project operation requirement table includes the preset completion time of the project, the host resource demand, and the optimal host resource allocation corresponding to the host resource demand;
    获取用户输入的目标项目信息,并对所述目标项目信息进行数据预处理,其中,所述数据预处理包括缺失值填补处理、去噪处理和数据标准化处理;Acquiring target item information input by the user, and performing data preprocessing on the target item information, wherein the data preprocessing includes missing value filling processing, denoising processing, and data standardization processing;
    将经过数据预处理的目标项目信息分成N组,通过反复迭代法对分成N组的目标项目信息进行多次重新分组,以获取最优分组方案;Divide the target item information that has undergone data preprocessing into N groups, and regroup the target item information divided into N groups through repeated iterations to obtain the optimal grouping scheme;
    获取所述最优分组方案中各组目标项目信息的项目数据类型;Acquiring the project data type of each group of target project information in the optimal grouping scheme;
    对所述目标项目信息进行分析以获取第四关键信息;Analyze the target item information to obtain fourth key information;
    根据所述第四关键信息遍历所述项目操作需求表,以获取所述第四关键信息对应的项目操作需求信息。Traverse the project operation requirement table according to the fourth key information to obtain project operation requirement information corresponding to the fourth key information.
  17. 根据权利要求15所述的设备,所述处理器执行所述计算机程序实现所述待评估资源包括在第一时段内的第一任务和在第二时段内的第二任务,所述通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据时,包括以下步骤:The device according to claim 15, wherein the processor executes the computer program to realize that the resource to be evaluated includes a first task in a first time period and a second task in a second time period. When the resource monitoring model acquires the first sampling data of the resource to be evaluated in the first time period and acquiring the second sampling data of the resource to be evaluated in the second time period, it includes the following steps:
    通过所述资源监控模型,获取所述第一任务中的第一占用率信息和第一优先级,以及获取所述第二任务中的第二占用率信息和第二优先级;Obtaining the first occupancy rate information and the first priority in the first task, and obtaining the second occupancy rate information and the second priority in the second task through the resource monitoring model;
    根据所述第一优先级,对所述第一任务进行分类,并标识第一类别标签,以及根据所述第二优先级,对所述第二任务进行分类,并标识第二类别标签;Classify the first task according to the first priority, and identify a first category label, and classify the second task according to the second priority, and identify a second category label;
    根据所述第一占用率信息,对标识所述第一类别标签的第一任务进行分类,并标识第三类别标签,以及所述第二占用率信息,对标识所述第二类别标签的第二任务进行分类,并标识第四类别标签;According to the first occupancy rate information, the first task that identifies the first category label is classified, and the third category label is identified, and the second occupancy rate information is used to identify the first task that identifies the second category label. Two tasks are classified and the fourth category label is identified;
    判断所述第一优先级是否符合第一预设采样条件和/或所述第一占用率信息是否符合第二预设采样条件,以及判断所述第二优先级是否符合第三预设采样条件和/或所述第二占用率信息是否符合第四预设采样条件;Determine whether the first priority meets a first preset sampling condition and/or whether the first occupancy rate information meets a second preset sampling condition, and determine whether the second priority meets a third preset sampling condition And/or whether the second occupancy rate information meets the fourth preset sampling condition;
    若所述第一优先级符合第一预设采样条件和/或所述第一占用率信息符合第二预设采样条件,则按照预设第一采样频率,对符合第一预设采样条件和/或第二预设采样条件的同一类别标签的任务进行采样,以获得第一采样数据;If the first priority meets the first preset sampling condition and/or the first occupancy rate information meets the second preset sampling condition, then according to the preset first sampling frequency, the pair meets the first preset sampling condition and / Or the tasks of the same category label under the second preset sampling conditions are sampled to obtain the first sampling data;
    若所述第二优先级符合第三预设采样条件和/或所述第二占用率信息符合第四预设采样条件,则按照预设第二采样频率,对符合第三预设采样条件和/或第四预设采样条件的同一类别标签的任务进行采样,以获得第二采样数据。If the second priority level meets the third preset sampling condition and/or the second occupancy rate information meets the fourth preset sampling condition, then according to the preset second sampling frequency, the pair meets the third preset sampling condition and /Or the tasks of the same category label under the fourth preset sampling condition are sampled to obtain the second sampling data.
  18. 根据权利要求15所述的设备,所述处理器执行所述计算机程序实现所述获取预测信息之后,还包括以下步骤:The device according to claim 15, after the processor executes the computer program to realize the acquisition of prediction information, further comprising the following steps:
    获取所述预测信息的第一时间序列数据;Acquiring the first time series data of the forecast information;
    对所述第一时间序列数据进行滑动窗口处理,以生成预设数量的具备预设长度的时序子序列;Performing sliding window processing on the first time series data to generate a preset number of time series subsequences with a preset length;
    分析所述时序子序列的统计指标以获取统计特征信息,并以所述统计特征信息作为更新后的预测信息,其中,所述统计特征信息包括最大值、最小值、中位数、第一四分位数、第三四分位数、方差和标准差。Analyze the statistical indicators of the time series subsequence to obtain statistical feature information, and use the statistical feature information as updated forecast information, where the statistical feature information includes the maximum value, the minimum value, the median, the first four Quantile, third quartile, variance and standard deviation.
  19. 根据权利要求15所述的设备,所述处理器执行所述计算机程序实现所述根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测之前,还包括以下步骤:The device according to claim 15, before the processor executes the computer program to implement the prediction of the cloud host to be evaluated based on the first sampling data and the second sampling data, further comprising the following steps :
    获取所述第一采样数据的第二时间序列数据,以及获取所述第二采样数据的第三时间序列数据;Acquiring second time series data of the first sampling data, and acquiring third time series data of the second sampling data;
    通过所述数据处理子模型中的指数加权移动平均法EWMA,分别对所述第二时间序列数据和所述第三时间序列数据进行求值,以获得第一平滑处理数据和第一平滑处理数据,对所述第二时间序列数据和所述第三时间序列数据的求值计算如下:Through the exponentially weighted moving average method EWMA in the data processing sub-model, the second time series data and the third time series data are evaluated respectively to obtain the first smoothed data and the first smoothed data , The evaluation of the second time series data and the third time series data is calculated as follows:
    Figure PCTCN2019117048-appb-100011
    Figure PCTCN2019117048-appb-100011
    其中,x t为时刻t的实际第二时间序列数据或实际第三时间序列数据,系数α为加权 下降的速率,V t为t时刻的EWMA值; Where, x t is the actual second time series data or the actual third time series data at time t, the coefficient α is the rate of weighted decline, and V t is the EWMA value at time t;
    通过所述数据处理子模型中的马氏距离Mahalanobis算法,分别对所述第一平滑处理数据和所述第二平滑处理数据中的极度异常点进行检测与标识,并删除所标识的极度异常点,检测与标识所述极度异常点的计算如下:Through the Mahalanobis algorithm in the data processing sub-model, the extreme abnormal points in the first smoothing data and the second smoothing data are detected and identified, and the identified extreme abnormal points are deleted , The calculation for detecting and identifying the extreme abnormal point is as follows:
    Figure PCTCN2019117048-appb-100012
    Figure PCTCN2019117048-appb-100012
    其中,
    Figure PCTCN2019117048-appb-100013
    是b与
    Figure PCTCN2019117048-appb-100014
    的距离,
    Figure PCTCN2019117048-appb-100015
    是所述平滑处理数据的均值向量,b是为所述平滑处理数据中的其他对象,S是协方差矩阵。
    among them,
    Figure PCTCN2019117048-appb-100013
    Is b and
    Figure PCTCN2019117048-appb-100014
    the distance,
    Figure PCTCN2019117048-appb-100015
    Is the mean vector of the smoothed data, b is other objects in the smoothed data, and S is the covariance matrix.
  20. 一种计算机可读存储介质,所述计算机可读存储介质中存储计算机指令,当所述计算机指令在计算机上运行时,使得计算机执行如下步骤:A computer-readable storage medium stores computer instructions in the computer-readable storage medium, and when the computer instructions are executed on a computer, the computer executes the following steps:
    获取训练数据,将所述训练数据输入至神经网络模型,并对所述神经网络模型进行训练,以获得资源监控模型,其中,所述训练数据包括多个云主机的多个时段的资源采样数据;Obtain training data, input the training data into a neural network model, and train the neural network model to obtain a resource monitoring model, wherein the training data includes resource sampling data of multiple cloud hosts for multiple periods ;
    获取输入的目标项目信息,并对所述目标项目信息进行分析,以获取所述目标项目信息的项目数据类型和项目操作需求信息;Obtaining the input target project information, and analyzing the target project information to obtain the project data type and project operation requirement information of the target project information;
    获取云主机的使用资源类型和使用资源情况,并根据所述使用资源类型、所述使用资源情况、所述项目数据类型和所述项目操作需求信息,确定待评估云主机以及所述待评估云主机的待评估资源;Obtain the used resource type and used resource situation of the cloud host, and determine the cloud host to be assessed and the cloud to be assessed based on the used resource type, the used resource situation, the project data type, and the project operation requirement information The host's resources to be evaluated;
    通过所述资源监控模型获取所述待评估资源在第一时段内的第一采样数据,以及获取所述待评估资源在第二时段内的第二采样数据,其中,所述第一时段的起始时刻晚于所述第二时段的结束时刻;Obtain the first sampling data of the resource to be evaluated in the first time period through the resource monitoring model, and acquire the second sampling data of the resource to be evaluated in the second time period, where the beginning of the first time period The start time is later than the end time of the second time period;
    根据所述第一采样数据和所述第二采样数据对所述待评估云主机进行预测,以分别获取第一预测数据、第二预测数据和第三预测数据,并将所述第三预测数据作为主信息,以及将所述第一预测数据和所述第二预测数据作为辅助信息,以获取预测信息,其中,所述预测包括对所述待评估云主机在第三时段的各资源使用状态和各资源使用量的趋势的预测,所述第三时段的起始时刻晚于所述第一时段的结束时刻;According to the first sampling data and the second sampling data, the cloud host to be evaluated is predicted to obtain the first prediction data, the second prediction data, and the third prediction data, respectively, and the third prediction data As main information, and using the first prediction data and the second prediction data as auxiliary information to obtain prediction information, wherein the prediction includes the status of each resource usage of the cloud host to be evaluated in the third period And prediction of the trend of each resource usage, the start time of the third time period is later than the end time of the first time period;
    分别对所述第一采样数据、所述第二采样数据和所述预测信息进行特征提取,以分别获取第一关键信息、第二关键信息和第三关键信息,并获取所述第一采样数据中待评估资源的第一可替代资源,和获取所述第二采样数据中待评估资源的第二可替代资源;Perform feature extraction on the first sampling data, the second sampling data, and the prediction information to obtain the first key information, the second key information, and the third key information, respectively, and obtain the first sampling data The first alternative resource of the resource to be evaluated in the second sample data, and the second alternative resource of the resource to be evaluated in the second sampling data;
    根据所述第一关键信息、所述第二关键信息、所述第三关键信息、所述第一可替代资源和所述第二可替代资源对所述待评估云主机进行评估,以获取评估信息;Evaluate the cloud host to be evaluated according to the first key information, the second key information, the third key information, the first replaceable resource, and the second replaceable resource to obtain the evaluation information;
    根据所述评估信息生成与所述评估信息对应的优化配置策略,并输出所述评估信息和所述优化配置策略。According to the evaluation information, an optimized configuration strategy corresponding to the evaluation information is generated, and the evaluation information and the optimized configuration strategy are output.
PCT/CN2019/117048 2019-09-19 2019-11-11 Method, apparatus and device for estimating cloud host resources, and storage medium WO2021051529A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910885428.1 2019-09-19
CN201910885428.1A CN110806954B (en) 2019-09-19 2019-09-19 Method, device, equipment and storage medium for evaluating cloud host resources

Publications (1)

Publication Number Publication Date
WO2021051529A1 true WO2021051529A1 (en) 2021-03-25

Family

ID=69487681

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/117048 WO2021051529A1 (en) 2019-09-19 2019-11-11 Method, apparatus and device for estimating cloud host resources, and storage medium

Country Status (2)

Country Link
CN (1) CN110806954B (en)
WO (1) WO2021051529A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116383014A (en) * 2023-06-02 2023-07-04 北京集度科技有限公司 Monitoring computer, software debugging method, software debugging system, medium and product
CN117076141A (en) * 2023-10-17 2023-11-17 深圳迅策科技有限公司 High-applicability off-line data processing task issuing method and system
CN117155864A (en) * 2023-11-01 2023-12-01 南京市微驰数字科技有限公司 Flow management system and method based on Internet

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270436A (en) * 2020-10-26 2021-01-26 北京明略昭辉科技有限公司 Resource delivery effect evaluation method, device and system
CN112351105B (en) * 2020-11-12 2022-09-16 中国信息通信研究院 Cloud service evaluation method and device
TWI767530B (en) * 2021-01-22 2022-06-11 緯穎科技服務股份有限公司 Electronic device and method for generating reference configuration of computing device
CN114240212A (en) * 2021-12-22 2022-03-25 中国地质大学(北京) Method and equipment for determining influence weight of geological parameters on resource quantity
CN114219377B (en) * 2022-02-22 2022-06-07 云智慧(北京)科技有限公司 Service resource allocation method, device and equipment
CN114741133B (en) * 2022-04-21 2023-10-27 中国航空无线电电子研究所 Comprehensive modularized avionics system resource allocation and assessment method based on model
CN114840348B (en) * 2022-07-01 2022-10-18 石家庄学院 Resource grade determination method and system for computer

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104981783A (en) * 2013-03-07 2015-10-14 思杰系统有限公司 Dynamic configuration in cloud computing environments
CN107241421A (en) * 2017-06-21 2017-10-10 北京云联万维技术有限公司 A kind of cloud host resource method for obligating and device
US20180254998A1 (en) * 2017-03-02 2018-09-06 Alcatel Lucent Resource allocation in a cloud environment
CN109492774A (en) * 2018-11-06 2019-03-19 北京工业大学 A kind of cloud resource dispatching method based on deep learning

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106447077A (en) * 2016-08-30 2017-02-22 新奥泛能网络科技股份有限公司 Resource evaluation method and resource evaluation system
US10931595B2 (en) * 2017-05-31 2021-02-23 Futurewei Technologies, Inc. Cloud quality of service management
CN109684074A (en) * 2018-11-12 2019-04-26 平安科技(深圳)有限公司 Physical machine resource allocation methods and terminal device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104981783A (en) * 2013-03-07 2015-10-14 思杰系统有限公司 Dynamic configuration in cloud computing environments
US20180254998A1 (en) * 2017-03-02 2018-09-06 Alcatel Lucent Resource allocation in a cloud environment
CN107241421A (en) * 2017-06-21 2017-10-10 北京云联万维技术有限公司 A kind of cloud host resource method for obligating and device
CN109492774A (en) * 2018-11-06 2019-03-19 北京工业大学 A kind of cloud resource dispatching method based on deep learning

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116383014A (en) * 2023-06-02 2023-07-04 北京集度科技有限公司 Monitoring computer, software debugging method, software debugging system, medium and product
CN116383014B (en) * 2023-06-02 2023-08-01 北京集度科技有限公司 Monitoring computer, software debugging method, software debugging system, medium and product
CN117076141A (en) * 2023-10-17 2023-11-17 深圳迅策科技有限公司 High-applicability off-line data processing task issuing method and system
CN117076141B (en) * 2023-10-17 2024-01-26 深圳迅策科技有限公司 High-applicability off-line data processing task issuing method and system
CN117155864A (en) * 2023-11-01 2023-12-01 南京市微驰数字科技有限公司 Flow management system and method based on Internet
CN117155864B (en) * 2023-11-01 2024-01-30 南京市微驰数字科技有限公司 Flow management system and method based on Internet

Also Published As

Publication number Publication date
CN110806954B (en) 2023-06-16
CN110806954A (en) 2020-02-18

Similar Documents

Publication Publication Date Title
WO2021051529A1 (en) Method, apparatus and device for estimating cloud host resources, and storage medium
US20180253657A1 (en) Real-time credit risk management system
JP5218390B2 (en) Autonomous control server, virtual server control method and program
WO2020220758A1 (en) Method for detecting abnormal transaction node, and device
WO2021189831A1 (en) Log optimization method, apparatus and device, and readable storage medium
CN111343161B (en) Abnormal information processing node analysis method, abnormal information processing node analysis device, abnormal information processing node analysis medium and electronic equipment
CN110442712B (en) Risk determination method, risk determination device, server and text examination system
WO2020168756A1 (en) Cluster log feature extraction method, and apparatus, device and storage medium
WO2021098384A1 (en) Data abnormality detection method and apparatus
WO2020082588A1 (en) Method and apparatus for identifying abnormal service request, electronic device, and medium
US20150095719A1 (en) Data preprocessing device and method thereof
CN111435463A (en) Data processing method and related equipment and system
EP3876201A1 (en) Object detection and candidate filtering system
EP2916254A1 (en) Biometric authentication apparatus and method
WO2020232902A1 (en) Abnormal object identification method and apparatus, computing device, and storage medium
CN111274256A (en) Resource control method, device, equipment and storage medium based on time sequence database
US20180052882A1 (en) Isolation anomaly quantification through heuristical pattern detection
WO2021189845A1 (en) Detection method and apparatus for time series anomaly point, and device and readable storage medium
WO2012088760A1 (en) Data analysis device and method therefor based on analysis factors calculated on demand
WO2020220438A1 (en) Method for predicting concurrent volume of services of different types for virtual machine
CN112035159B (en) Configuration method, device, equipment and storage medium of audit model
US20220239673A1 (en) System and method for differentiating between human and non-human access to computing resources
CN114785616A (en) Data risk detection method and device, computer equipment and storage medium
US20190138931A1 (en) Apparatus and method of introducing probability and uncertainty via order statistics to unsupervised data classification via clustering
CN112734352A (en) Document auditing method and device based on data dimensionality

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19945843

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19945843

Country of ref document: EP

Kind code of ref document: A1