CN117931366A - Application programming method, device, equipment and storage medium based on performance perception - Google Patents
Application programming method, device, equipment and storage medium based on performance perception Download PDFInfo
- Publication number
- CN117931366A CN117931366A CN202410015266.7A CN202410015266A CN117931366A CN 117931366 A CN117931366 A CN 117931366A CN 202410015266 A CN202410015266 A CN 202410015266A CN 117931366 A CN117931366 A CN 117931366A
- Authority
- CN
- China
- Prior art keywords
- application
- performance
- model
- storage
- inputting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000008447 perception Effects 0.000 title abstract description 12
- 238000012544 monitoring process Methods 0.000 claims abstract description 180
- 238000012549 training Methods 0.000 claims abstract description 55
- 238000013507 mapping Methods 0.000 claims abstract description 50
- 238000010801 machine learning Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 2
- 206010063385 Intellectualisation Diseases 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000003068 static effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Abstract
The invention relates to the field of application programming, and discloses a performance awareness-based application programming method, device, equipment and storage medium. The method comprises the following steps: acquiring a history log of an application, extracting application characteristic parameters and storage configuration parameters, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters; inputting parameters into a performance monitoring model for training, and inputting a mapping relation into a performance prediction model; acquiring an application programming plan, and respectively inputting the application programming plan into a trained performance monitoring model and a trained performance prediction model to obtain monitoring data and prediction data; and creating an application storage performance model, training, and optimizing an application programming plan by using the trained application storage performance model. The invention provides an application programming method based on performance perception, which is characterized in that a storage performance model of an application program is established by monitoring a stored access mode and a storage system load characteristic, and the application program and storage resources are cooperatively distributed, so that a scheduling technology of win-win application performance and storage performance is realized.
Description
Technical Field
The present invention relates to the field of application programming, and in particular, to a performance awareness-based application programming method, apparatus, device, and storage medium.
Background
Application programming, commonly referred to as container programming, is a very popular technique for managing extremely large numbers of containers, involving a large number of tasks; the good application arrangement scheme can remarkably improve the application management efficiency for development teams, and is beneficial to improving the performance, reliability and safety of the application.
However, the current application programming and scheduling system focuses more on the allocation of computing resources, but is insensitive to the perception and optimization of storage resources, while existing new applications, such as big data and AI, have increasingly higher requirements on storage performance, and the monitoring and perception mechanisms of the storage access mode in the prior art are weaker.
Accordingly, the prior art is still in need of improvement and development.
Disclosure of Invention
The invention mainly aims to solve the problems of insufficient storage capacity, access hot spot and performance reduction caused by neglecting the importance of storage resources according to excessive attention to computing resource allocation in the application arrangement in the prior art.
The first aspect of the invention provides an application programming method based on performance perception, comprising the following steps: acquiring a history log of an application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters; creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model; acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data; and creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan.
Optionally, in a first implementation manner of the first aspect of the present invention, the step of obtaining a history log of the application, extracting an application feature parameter and a storage configuration parameter according to the history log, and establishing a mapping relationship between the application feature parameter and the storage configuration parameter includes: acquiring a history log of an application, extracting an application source code according to the history log, analyzing the application source code, and extracting application characteristic parameters of the application; obtaining the read-write characteristics of the application according to the storage information in the history log, and generating storage configuration parameters of the application according to the read-write characteristics; and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters.
Optionally, in a second implementation manner of the first aspect of the present invention, the creating a performance monitoring model and a performance prediction model, inputting the application feature parameter and the storage configuration parameter into the performance monitoring model to perform training to obtain a target performance monitoring model, inputting the mapping relationship into the performance prediction model, and obtaining the target performance prediction model includes: creating a performance monitoring model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model, and training by using a machine learning algorithm to obtain a target performance monitoring model; and creating a performance prediction model, inputting the mapping relation into the performance prediction model, and training by using a machine learning algorithm to obtain a target performance prediction model.
Optionally, in a third implementation manner of the first aspect of the present invention, the step of obtaining an application programming plan input by a user, and inputting the application programming plan into the target performance monitoring model and the target performance prediction model respectively, to obtain monitoring data and prediction data respectively includes: acquiring an application programming plan input by a user, extracting corresponding current application characteristic parameters according to the application programming plan, and inputting the current application characteristic parameters into the target performance monitoring model to obtain monitoring data of the application programming plan; and establishing a current mapping relation between the current application characteristic parameters and the monitoring data, and inputting the current mapping relation into the target performance prediction model to obtain the prediction data of the application programming plan.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the monitoring data includes: load monitoring data, performance monitoring data, and capacity monitoring data; the step of inputting the current application characteristic parameters into the target performance monitoring model to obtain the monitoring data of the application programming plan comprises the following steps: determining a current access mode of the current application for storage according to the characteristic parameters of the current application, and acquiring current access mode information of the current access mode; and inputting the current access mode information into the target performance monitoring model to obtain the load monitoring data, the performance monitoring data and the capacity monitoring data of the application programming plan.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the creating an application storage performance model, inputting the history log into the application storage performance model to train to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan includes: creating an application storage performance model, extracting historical application storage performance data from the historical logs, and inputting the historical application storage performance data into the application storage performance model for training to obtain a current application storage performance model; and inputting the monitoring data and the prediction data into the current application storage performance model to output an application scheduling decision, and optimizing the application scheduling plan according to the application scheduling decision.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the step of inputting the monitoring data and the prediction data into the current application storage performance model, outputting an application scheduling decision, and optimizing the application scheduling plan according to the application scheduling decision includes: inputting the monitoring data and the prediction data into the current application storage performance model, outputting an ideal application programming plan, and generating an application scheduling decision according to the difference between the ideal application programming plan and the application programming plan; and according to the application scheduling decision, adjusting the starting quantity of the applications, migrating the applications or limiting the storage of the low-priority applications, and optimizing the application scheduling plan.
A second aspect of the present invention provides an application orchestration device based on performance awareness, comprising: the parameter acquisition module is used for acquiring a history log of the application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters; the model construction module is used for creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model; the data acquisition module is used for acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data; and the programming optimization model is used for creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming.
Optionally, in a first implementation manner of the second aspect of the present invention, the parameter obtaining module includes: the application characteristic parameter extraction unit is used for acquiring a history log of the application, extracting an application source code according to the history log, analyzing the application source code and extracting application characteristic parameters of the application; the storage configuration parameter generation unit is used for obtaining the read-write characteristics of the application according to the storage information in the history log and generating the storage configuration parameters of the application according to the read-write characteristics; and the mapping relation establishing unit is used for establishing the mapping relation between the application characteristic parameters and the storage configuration parameters.
Optionally, in a second implementation manner of the second aspect of the present invention, the model building module includes: the performance monitoring model training unit is used for creating a performance monitoring model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model, and training by using a machine learning algorithm to obtain a target performance monitoring model; and the performance prediction model training unit is used for creating a performance prediction model, inputting the mapping relation into the performance prediction model, and training by using a machine learning algorithm to obtain a target performance prediction model.
Optionally, in a third implementation manner of the second aspect of the present invention, the data acquisition module includes: the monitoring data acquisition unit is used for acquiring an application programming plan input by a user, extracting corresponding current application characteristic parameters according to the application programming plan, and inputting the current application characteristic parameters into the target performance monitoring model to obtain monitoring data of the application programming plan; and the predicted data acquisition unit is used for establishing a current mapping relation between the current application characteristic parameters and the monitoring data, and inputting the current mapping relation into the target performance prediction model to obtain predicted data of the application programming plan.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the monitoring data acquisition unit further includes: an access mode determining subunit, configured to determine a current access mode of the current application for storage according to the current application feature parameter, and obtain current access mode information of the current access mode; and the information acquisition subunit is used for inputting the current access mode information into the target performance monitoring model to obtain the load monitoring data, the performance monitoring data and the capacity monitoring data of the application programming plan.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the orchestration plan optimization model includes: the storage performance model training unit is used for creating an application storage performance model, extracting historical application storage performance data from the historical logs, inputting the historical application storage performance data into the application storage performance model for training, and obtaining a current application storage performance model; and the plan optimizing unit is used for inputting the monitoring data and the prediction data into the current application storage performance model to output an application scheduling decision, and optimizing the application scheduling plan according to the application scheduling decision.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the plan optimization unit further includes: a decision generation subunit, configured to input the monitoring data and the prediction data into the current application storage performance model, output an ideal application scheduling plan, and generate an application scheduling decision according to a difference between the ideal application scheduling plan and the application scheduling plan; and the application programming plan optimizing subunit is used for adjusting the starting quantity of the applications, migrating the applications or limiting the storage of the low-priority applications according to the application scheduling decision and optimizing the application programming plan.
A third aspect of the present invention provides an application orchestration device based on performance awareness, comprising: a memory and at least one processor, the memory having computer readable instructions stored therein, the memory and the at least one processor being interconnected by a line; the at least one processor invokes the computer-readable instructions in the memory to cause the performance-aware based application orchestration device to perform the steps of the performance-aware based application orchestration method described above.
A fourth aspect of the present invention provides a computer readable storage medium having stored therein computer readable instructions which, when run on a computer, cause the computer to perform the steps of the performance awareness based application orchestration method described above.
The beneficial effects are that: according to the technical scheme, a history log of an application is obtained, application characteristic parameters and storage configuration parameters are extracted according to the history log, and a mapping relation between the application characteristic parameters and the storage configuration parameters is established; creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model; acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data; and creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan. The invention provides an application programming method based on performance perception, which comprehensively acquires application characteristic parameters and storage configuration parameters of an application through static and dynamic program analysis, simultaneously monitors the load and performance states of a storage system in real time, constructs two application storage performance modeling methods based on rules and data training, predicts the performance of the application deployed under different storage configurations with high precision, combines a performance model and real-time monitoring, dynamically determines an optimal application scheduling strategy, ensures the storage performance and maximizes the application performance, so as to obviously improve the storage performance experience of different application types, improve the continuity and quality of the whole service, and obviously improve the intellectualization and accuracy of scheduling decisions.
Drawings
FIG. 1 is a first flowchart of an application orchestration method based on performance awareness according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a performance awareness based application orchestration method according to embodiments of the present invention;
FIG. 3 is a third flowchart of a performance awareness based application orchestration method according to an embodiment of the present invention;
FIG. 4 is a fourth flowchart of a performance awareness based application orchestration method according to embodiments of the present invention;
FIG. 5 is a fifth flowchart of a performance awareness based application orchestration method according to embodiments of the present invention;
FIG. 6 is a sixth flowchart of a performance aware-based application orchestration method according to an embodiment of the present invention;
FIG. 7 is a seventh flowchart of a performance aware based application orchestration method according to embodiments of the present invention;
FIG. 8 is a schematic structural diagram of an application programming device based on performance awareness according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of another configuration of an application programming device based on performance awareness according to an embodiment of the present invention;
Fig. 10 is a schematic structural diagram of an application programming device based on performance awareness according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an application programming method, device, equipment and storage medium based on performance perception, which are used for acquiring a history log of an application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters; creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model; acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data; and creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan. The invention provides an application programming method based on performance perception, which comprehensively acquires application characteristic parameters and storage configuration parameters of an application through static and dynamic program analysis, simultaneously monitors the load and performance states of a storage system in real time, constructs two application storage performance modeling methods based on rules and data training, predicts the performance of the application deployed under different storage configurations with high precision, combines a performance model and real-time monitoring, dynamically determines an optimal application scheduling strategy, ensures the storage performance and maximizes the application performance, so as to obviously improve the storage performance experience of different application types, improve the continuity and quality of the whole service, and obviously improve the intellectualization and accuracy of scheduling decisions.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, a specific flow of an embodiment of the present invention is described below, referring to fig. 1, where a first embodiment of an application orchestration method based on performance awareness according to an embodiment of the present invention includes:
s101, acquiring a history log of an application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters;
in the embodiment, a history log in an application arranging process is obtained, an application source static code is analyzed according to the history log, information such as a storage access mode and a data structure of an application is extracted, and application characteristic parameters are extracted; and meanwhile, by implanting a tracking probe, information such as the number of times, the size, the delay and the like of storage access is recorded in the running process, and then by periodically sampling a storage access log of an application, the read-write characteristics of the application are obtained, and the storage configuration parameters are extracted.
Further, according to an experience rule, a mapping relation between application characteristic parameters and storage configuration parameters is predefined to form a qualitative model, and load and performance states of a storage system are monitored in real time, for example, random reading speed of SSD equipment can reach xx MB/s, and a mechanical hard disk can only have xx MB/s, so that it can be determined what storage needs to be deployed by the application to reach a performance target.
S102, creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model;
In this embodiment, on the basis of acquiring the application feature parameters and the storage configuration parameters, a performance monitoring model and a performance prediction model are created, the application feature parameters and the storage configuration parameters are input, and a neural network, an SVM and other machine learning algorithms are used for training, so as to predict subsequent application programming, and monitor the application programming results.
S103, acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data;
In this embodiment, after an application programming plan input by a user is obtained, an application programming result of the application programming plan is predicted by using a target performance prediction model, and application performance of the application programming result is judged by using a target performance monitoring model, so as to obtain predicted data and monitored data respectively, and load and performance states of a storage system are monitored in real time based on two application storage performance modeling methods of rules and data training.
S104, creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan.
In this embodiment, an application storage performance model is created, and a history log of application programming is used to input training, and based on a target performance monitoring model and a target performance prediction model which are established in the foregoing, and acquired monitoring data and prediction data, the application storage performance model can intelligently make an application scheduling decision, for example, when monitoring finds that the load of a storage system is too high, a new application can be suspended to start, or a non-critical application can be migrated to other storage, so as to ensure the storage performance of key services such as a rhyme express query system.
Referring to fig. 2, a second embodiment of the performance-aware-based application programming method according to an embodiment of the present invention includes:
S201, acquiring a history log of an application, extracting an application source code according to the history log, analyzing the application source code, and extracting application characteristic parameters of the application;
S202, obtaining the read-write characteristics of the application according to the storage information in the history log, and generating storage configuration parameters of the application according to the read-write characteristics;
s203, establishing a mapping relation between the application characteristic parameters and the storage configuration parameters.
In this embodiment, by analyzing an application source code in a history log of application arrangement, extracting information such as a storage access mode and a data structure of an application, extracting an application feature parameter, by implanting a tracking probe, recording information such as the number of times, the size and the delay of storage access during running, judging the read-write feature of the application, and extracting a storage configuration parameter.
Further, by periodically sampling the storage access log of the application, configuration information such as the application access stored foltprint, throughput, I/O SIZE and the like is extracted, for example, for an order query system of the rhyme express delivery, the code and log analysis shows that the storage access is mainly read operation, a large number of small file queries are carried out, the access mode is relatively stable, and the characteristic information can be used as an important reference for arranging the type and the scale of storage resources required by the application, so that the load and the performance state of the storage system can be monitored in real time.
Referring to fig. 3, a third embodiment of an application programming method based on performance awareness according to an embodiment of the present invention includes:
S301, creating a performance monitoring model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model, and training by using a machine learning algorithm to obtain a target performance monitoring model;
S302, a performance prediction model is created, the mapping relation is input into the performance prediction model, and training is carried out by using a machine learning algorithm to obtain a target performance prediction model.
In this embodiment, a performance monitoring model and a performance prediction model are created, the application feature parameters and the storage configuration parameters acquired in the previous step are input into the performance monitoring model, and training is performed by using a machine learning algorithm to optimize the performance monitoring model, wherein the application feature parameters may include a read-write ratio (r) of an applied read operation to all IO accesses, an average I/O size(s) of an applied average IO request size, an IOPS (I) of an average I/O number/second of an application, and an access mode (m) of an applied access mode (sequential access/random access), and the storage configuration parameters include a device type (t) of a type of a storage device (a mechanical hard disk/a solid state hard disk SSD) and a number of devices (n) of the storage device.
Further, according to experience rules extracted from the history log, deploying actual applications by using different storage configurations (equipment types, quantity and the like), obtaining performance indexes such as response time, throughput and the like of the applications, forming an input characteristic and performance annotation data set, predefining a mapping relation between application characteristic parameters and storage configuration parameters, training a performance prediction model by using machine learning algorithms such as a neural network and SVM according to the mapping relation under the different application storage configurations, and using storage access characteristic modeling and performance prediction means to remarkably improve the intellectualization and accuracy of scheduling decisions.
For example, if the application is read intensive (r > 80%), the I/O size is not large (s <16 k), the IOPS is large (I > 1000), the access pattern is random (m=random), SSD storage devices (t=ssd) are required, and the number of devices is not less than 2 (n > =2).
Referring to fig. 4, a fourth embodiment of the application programming method based on performance awareness of a express item in an embodiment of the present invention includes:
s401, acquiring an application programming plan input by a user, extracting corresponding current application characteristic parameters according to the application programming plan, and inputting the current application characteristic parameters into the target performance monitoring model to obtain monitoring data of the application programming plan;
S402, establishing a current mapping relation between the current application characteristic parameters and the monitoring data, and inputting the current mapping relation into the target performance prediction model to obtain the prediction data of the application programming plan.
In this embodiment, for a new application scheduling plan, inputting its characteristic parameters into a target performance monitoring model, monitoring performance indexes such as response time and throughput of an application to obtain monitoring data of application performance, inputting a current mapping relationship between a current application characteristic parameter and the monitoring data into a target performance prediction model, predicting an influence of the current application scheduling plan on subsequent application performance, where the indexes can measure a service capacity allowance of a storage system, predict a load change trend of the system in a future period, and provide a basis for subsequent application scheduling decisions; the two application storage performance modeling methods based on rules and data training can predict the performance of the application deployed under different storage configurations with high precision.
Referring to fig. 5, a fifth embodiment of the application programming method based on performance awareness of a express item according to the embodiment of the present invention includes:
S501, determining a current access mode of the current application for storage according to the characteristic parameters of the current application, and acquiring current access mode information of the current access mode;
s502, inputting the current access mode information into the target performance monitoring model to obtain the load monitoring data, the performance monitoring data and the capacity monitoring data of the application programming plan.
In this embodiment, the operation state of the monitoring storage system is simulated when the storage condition during the running of the application is predicted, and specific monitoring indexes include load monitoring data (monitoring the IO utilization rate of the storage device, queue delay, and the like, judging the load condition), performance monitoring data (tracking information such as read-write speed, IOPS, and the like, judging the performance condition), and capacity monitoring data (monitoring the capacity use condition of the storage device).
Referring to fig. 6, a sixth embodiment of the application programming method based on performance awareness of a express item according to the embodiment of the present invention includes:
S601, creating an application storage performance model, extracting historical application storage performance data from the historical logs, inputting the historical application storage performance data into the application storage performance model for training, and obtaining a current application storage performance model;
S602, inputting the monitoring data and the prediction data into the current application storage performance model to output an application scheduling decision, and optimizing the application scheduling plan according to the application scheduling decision.
In the embodiment, an application storage performance model is created, the application storage performance model is trained by utilizing historical application storage performance data extracted by a historical log, and then an optimal application scheduling strategy is dynamically determined by combining the performance model with real-time monitoring and prediction data, so that the storage performance is ensured, the application performance is maximized, and compared with a mode of statically distributing storage resources, the dynamic coordination optimization of the application program performance and the storage system performance is realized.
For example, when the monitoring finds that the load of the storage system is too high, starting a new application can be suspended, or non-critical applications can be migrated to other storage, so that the storage performance of key services such as a rhyme express inquiry system is ensured. Referring to fig. 7, a seventh embodiment of a method for arranging applications based on performance awareness of a widget according to an embodiment of the present invention includes:
S701, inputting the monitoring data and the prediction data into the current application storage performance model, outputting an ideal application programming plan, and generating an application scheduling decision according to the difference between the ideal application programming plan and the application programming plan;
S702, according to the application scheduling decision, the starting number of the applications is adjusted, the applications are migrated or the storage of the low-priority applications is limited, and the application scheduling plan is optimized.
In this embodiment, based on the performance monitoring model and the performance prediction model established above and the real-time prediction and monitoring data, the storage performance aware scheduler may intelligently make application scheduling decisions, such as application start-up quantity adjustment (reasonably setting the concurrency of application instances, controlling access pressure to storage), application migration (migrating applications to better-performing storage servers, improving performance), and storage load shaping (limiting storage access of low-priority applications appropriately according to load monitoring, and relieving storage pressure).
The method comprises the steps of establishing a storage access characteristic model, performing storage access characteristic prediction, and performing storage access characteristic model selection, wherein the storage access characteristic model and performance prediction are utilized, and the optimal application scheduling scheme can be selected based on prediction and perception of the state of a storage system through application scheduling, so that the intellectualization and accuracy of scheduling decisions are remarkably improved, the storage performance is ensured, the application performance is maximized, the storage performance experience of different application types is remarkably improved, and the continuity and quality of the whole service are improved.
The invention provides an application programming method based on performance perception, which comprehensively acquires application characteristic parameters and storage configuration parameters of an application through static and dynamic program analysis, simultaneously monitors the load and performance states of a storage system in real time, constructs two application storage performance modeling methods based on rules and data training, predicts the performance of the application deployed under different storage configurations with high precision, combines a performance model and real-time monitoring, dynamically determines an optimal application scheduling strategy, ensures the storage performance and maximizes the application performance, so as to obviously improve the storage performance experience of different application types, improve the continuity and quality of the whole service, and obviously improve the intellectualization and accuracy of scheduling decisions.
The above describes an application programming method based on performance sensing in the embodiment of the present invention, and the following describes an application programming device based on performance sensing in the embodiment of the present invention, referring to fig. 8, an embodiment of the application programming device based on performance sensing in the embodiment of the present invention includes:
the parameter acquisition module 50 is configured to acquire a history log of an application, extract application feature parameters and storage configuration parameters according to the history log, and establish a mapping relationship between the application feature parameters and the storage configuration parameters;
The model building module 60 is configured to create a performance monitoring model and a performance prediction model, input the application feature parameter and the storage configuration parameter to the performance monitoring model for training to obtain a target performance monitoring model, and input the mapping relationship to the performance prediction model to obtain a target performance prediction model;
A data acquisition module 70, configured to acquire an application programming plan input by a user, and input the application programming plan into the target performance monitoring model and the target performance prediction model respectively, so as to obtain monitoring data and prediction data respectively;
And the scheduling optimization model 80 is configured to create an application storage performance model, input the history log into the application storage performance model for training to obtain a current application storage performance model, input the monitoring data and the prediction data into the current application storage performance model, and optimize the application scheduling.
In this embodiment, application feature parameters and storage configuration parameters of an application are comprehensively obtained through static and dynamic program analysis, meanwhile, load and performance states of a storage system are monitored in real time, two application storage performance modeling methods based on rules and data training are constructed, performance performances of the application deployed under different storage configurations are predicted with high precision, an optimal application scheduling strategy is dynamically determined by combining a performance model and real-time monitoring, storage performance is guaranteed, meanwhile, application performance is maximized, storage performance experience of different application types is obviously improved, overall service continuity and quality are improved, and intelligence and accuracy of scheduling decisions are remarkably improved.
Referring to fig. 9, another embodiment of the performance aware application programming device according to an embodiment of the present invention includes:
the parameter acquisition module 50 is configured to acquire a history log of an application, extract application feature parameters and storage configuration parameters according to the history log, and establish a mapping relationship between the application feature parameters and the storage configuration parameters;
The model building module 60 is configured to create a performance monitoring model and a performance prediction model, input the application feature parameter and the storage configuration parameter to the performance monitoring model for training to obtain a target performance monitoring model, and input the mapping relationship to the performance prediction model to obtain a target performance prediction model;
A data acquisition module 70, configured to acquire an application programming plan input by a user, and input the application programming plan into the target performance monitoring model and the target performance prediction model respectively, so as to obtain monitoring data and prediction data respectively;
And the scheduling optimization model 80 is configured to create an application storage performance model, input the history log into the application storage performance model for training to obtain a current application storage performance model, input the monitoring data and the prediction data into the current application storage performance model, and optimize the application scheduling.
In this embodiment, the parameter obtaining module 50 includes:
An application feature parameter extraction unit 501, configured to obtain a history log of an application, extract an application source code according to the history log, analyze the application source code, and extract an application feature parameter of the application;
a storage configuration parameter generating unit 502, configured to obtain a read-write characteristic of the application according to the storage information in the history log, and generate a storage configuration parameter of the application according to the read-write characteristic;
a mapping relation establishing unit 503, configured to establish a mapping relation between the application feature parameter and the storage configuration parameter.
In this embodiment, the model building module 60 includes:
The performance monitoring model training unit 601 is configured to create a performance monitoring model, input the application feature parameter and the storage configuration parameter into the performance monitoring model, and perform training by using a machine learning algorithm to obtain a target performance monitoring model;
The performance prediction model training unit 602 is configured to create a performance prediction model, input the mapping relation to the performance prediction model, and perform training by using a machine learning algorithm to obtain a target performance prediction model.
In this embodiment, the joint probability calculation module 70 includes:
an input unit 701, configured to input the prior probability into the trained bayesian network model to obtain a posterior probability;
A second calculating unit 702, configured to calculate a joint probability of the quality level under the influence of a plurality of risk factors according to the posterior probability and the probability distribution relation.
In this embodiment, the data acquisition module 70 includes:
a monitoring data obtaining unit 701, configured to obtain an application programming plan input by a user, extract a corresponding current application feature parameter according to the application programming plan, and input the current application feature parameter to the target performance monitoring model to obtain monitoring data of the application programming plan;
The predicted data obtaining unit 702 is configured to establish a current mapping relationship between the current application feature parameter and the monitoring data, and input the current mapping relationship to the target performance prediction model to obtain predicted data of the application programming plan.
In this embodiment, the monitoring data acquisition unit 701 further includes:
an access mode determining subunit 7011, configured to determine, according to the current application feature parameter, a current access mode in which the current application stores, and obtain current access mode information of the current access mode;
An information obtaining subunit 7012, configured to input the current access mode information to the target performance monitoring model, and obtain the load monitoring data, the performance monitoring data, and the capacity monitoring data of the application arrangement plan.
In this embodiment, the orchestration plan optimization model 80 includes:
a storage performance model training unit 801, configured to create an application storage performance model, extract historical application storage performance data from the history log, and input the historical application storage performance data to the application storage performance model for training, so as to obtain a current application storage performance model;
A plan optimizing unit 802, configured to input the monitoring data and the prediction data into the current application storage performance model to output an application scheduling decision, and optimize the application scheduling plan according to the application scheduling decision.
In this embodiment, the plan optimizing unit 802 further includes:
a decision generation subunit 8021, configured to input the monitoring data and the prediction data into the current application storage performance model, output an ideal application programming plan, and generate an application scheduling decision according to a difference between the ideal application programming plan and the application programming plan;
an application scheduling optimization subunit 8022, configured to adjust the number of applications started, migrate the applications, or limit the storage of low-priority applications according to the application scheduling decision, and optimize the application scheduling.
The invention provides an application programming method based on performance perception, which comprehensively acquires application characteristic parameters and storage configuration parameters of an application through static and dynamic program analysis, simultaneously monitors the load and performance states of a storage system in real time, constructs two application storage performance modeling methods based on rules and data training, predicts the performance of the application deployed under different storage configurations with high precision, combines a performance model and real-time monitoring, dynamically determines an optimal application scheduling strategy, ensures the storage performance and maximizes the application performance, so as to obviously improve the storage performance experience of different application types, improve the continuity and quality of the whole service, and obviously improve the intellectualization and accuracy of scheduling decisions.
The above-mentioned fig. 8 and fig. 9 describe the performance-aware-based application orchestration device according to the embodiment of the present invention in detail from the point of view of the modularized functional entity, and the performance-aware-based application orchestration device according to the embodiment of the present invention is described in detail from the point of view of hardware processing below.
Fig. 10 is a schematic structural diagram of a performance-aware-based application programming device according to an embodiment of the present invention, where the performance-aware-based application programming device 10 may have a relatively large difference according to a configuration or performance, and may include one or more processors (central processing units, CPU) 11 (e.g., one or more processors) and a memory 12, and one or more storage mediums 13 (e.g., one or more mass storage devices) storing application programs 133 or data 132. Wherein the memory 12 and the storage medium 13 may be transitory or persistent storage. The program stored on storage medium 13 may include one or more modules (not shown), each of which may include a series of instruction operations in performance-aware-based application orchestration device 10. Still further, processor 11 may be arranged to communicate with storage medium 13 to execute a series of instruction operations in storage medium 13 on performance-aware-based application orchestration device 10.
Performance-aware based application orchestration device 10 may also include one or more power supplies 14, one or more wired or wireless network interfaces 15, one or more input/output interfaces 16, and/or one or more operating systems 131, such as Windows Serve, mac OS X, unix, linux, freeBSD, etc. It will be appreciated by those skilled in the art that the device structure shown in FIG. 10 is not limiting of performance-aware based application orchestration device 10, and may include more or fewer components than shown, or may combine certain components, or may be arranged in a different arrangement of components.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, or a volatile computer readable storage medium, having stored therein instructions that, when executed on a computer, cause the computer to perform the steps of a performance-aware-based application orchestration method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system or apparatus and unit described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A performance awareness based application orchestration method, wherein the performance awareness based application orchestration method comprises:
Acquiring a history log of an application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters;
Creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model;
Acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data;
And creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming plan.
2. The performance awareness based application orchestration method according to claim 1, wherein the step of obtaining a history log of an application, extracting application feature parameters and storage configuration parameters according to the history log, and establishing a mapping relationship between the application feature parameters and the storage configuration parameters comprises:
Acquiring a history log of an application, extracting an application source code according to the history log, analyzing the application source code, and extracting application characteristic parameters of the application;
Obtaining the read-write characteristics of the application according to the storage information in the history log, and generating storage configuration parameters of the application according to the read-write characteristics;
and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters.
3. The performance-aware-based application programming method of claim 1, wherein the creating a performance monitoring model and a performance prediction model, inputting the application feature parameters and the storage configuration parameters to the performance monitoring model for training to obtain a target performance monitoring model, inputting the mapping relationship to the performance prediction model, and obtaining the target performance prediction model comprises:
Creating a performance monitoring model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model, and training by using a machine learning algorithm to obtain a target performance monitoring model;
And creating a performance prediction model, inputting the mapping relation into the performance prediction model, and training by using a machine learning algorithm to obtain a target performance prediction model.
4. The performance-aware-based application programming method of claim 1, wherein the step of obtaining the user-entered application programming plan, inputting the application programming plan to the target performance monitoring model and the target performance prediction model, respectively, and obtaining the monitoring data and the prediction data, respectively, comprises:
Acquiring an application programming plan input by a user, extracting corresponding current application characteristic parameters according to the application programming plan, and inputting the current application characteristic parameters into the target performance monitoring model to obtain monitoring data of the application programming plan;
And establishing a current mapping relation between the current application characteristic parameters and the monitoring data, and inputting the current mapping relation into the target performance prediction model to obtain the prediction data of the application programming plan.
5. The performance-aware-based application orchestration method according to claim 4, wherein the monitoring data comprises: load monitoring data, performance monitoring data, and capacity monitoring data;
the step of inputting the current application characteristic parameters into the target performance monitoring model to obtain the monitoring data of the application programming plan comprises the following steps:
Determining a current access mode of the current application for storage according to the characteristic parameters of the current application, and acquiring current access mode information of the current access mode;
And inputting the current access mode information into the target performance monitoring model to obtain the load monitoring data, the performance monitoring data and the capacity monitoring data of the application programming plan.
6. The performance-aware-based application orchestration method according to claim 1, wherein the creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application orchestration plan comprises:
Creating an application storage performance model, extracting historical application storage performance data from the historical logs, and inputting the historical application storage performance data into the application storage performance model for training to obtain a current application storage performance model;
and inputting the monitoring data and the prediction data into the current application storage performance model to output an application scheduling decision, and optimizing the application scheduling plan according to the application scheduling decision.
7. The performance-aware-based application orchestration method according to claim 6, wherein the step of inputting the monitoring data and the prediction data into the current application storage performance model, outputting an application scheduling decision, and optimizing the application orchestration plan according to the application scheduling decision comprises:
Inputting the monitoring data and the prediction data into the current application storage performance model, outputting an ideal application programming plan, and generating an application scheduling decision according to the difference between the ideal application programming plan and the application programming plan;
and according to the application scheduling decision, adjusting the starting quantity of the applications, migrating the applications or limiting the storage of the low-priority applications, and optimizing the application scheduling plan.
8. An application orchestration device based on performance awareness, comprising:
the parameter acquisition module is used for acquiring a history log of the application, extracting application characteristic parameters and storage configuration parameters according to the history log, and establishing a mapping relation between the application characteristic parameters and the storage configuration parameters;
The model construction module is used for creating a performance monitoring model and a performance prediction model, inputting the application characteristic parameters and the storage configuration parameters into the performance monitoring model for training to obtain a target performance monitoring model, and inputting the mapping relation into the performance prediction model to obtain a target performance prediction model;
The data acquisition module is used for acquiring an application programming plan input by a user, and respectively inputting the application programming plan into the target performance monitoring model and the target performance prediction model to respectively obtain monitoring data and prediction data;
And the programming optimization model is used for creating an application storage performance model, inputting the history log into the application storage performance model for training to obtain a current application storage performance model, inputting the monitoring data and the prediction data into the current application storage performance model, and optimizing the application programming.
9. An application orchestration device based on performance awareness, comprising a memory and at least one processor, the memory having computer-readable instructions stored therein;
the at least one processor invoking the computer readable instructions in the memory to perform the steps of the performance awareness based application orchestration method according to any one of claims 1-7.
10. A computer readable storage medium having stored thereon computer readable instructions, which when executed by a processor, implement the steps of the performance awareness based application orchestration method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410015266.7A CN117931366A (en) | 2024-01-04 | 2024-01-04 | Application programming method, device, equipment and storage medium based on performance perception |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410015266.7A CN117931366A (en) | 2024-01-04 | 2024-01-04 | Application programming method, device, equipment and storage medium based on performance perception |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117931366A true CN117931366A (en) | 2024-04-26 |
Family
ID=90765627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410015266.7A Pending CN117931366A (en) | 2024-01-04 | 2024-01-04 | Application programming method, device, equipment and storage medium based on performance perception |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117931366A (en) |
-
2024
- 2024-01-04 CN CN202410015266.7A patent/CN117931366A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10983895B2 (en) | System and method for data application performance management | |
| Hernández et al. | Using machine learning to optimize parallelism in big data applications | |
| Nghiem et al. | Towards efficient resource provisioning in MapReduce | |
| Kim et al. | A trust evaluation model for QoS guarantee in cloud systems | |
| CN102449603B (en) | Server control program, control server, virtual server distribution method | |
| US8924328B1 (en) | Predictive models for configuration management of data storage systems | |
| Hussein et al. | A light-weight data replication for cloud data centers environment | |
| US10642652B2 (en) | Best trade-off point on an elbow curve for optimal resource provisioning and performance efficiency | |
| CN101593133A (en) | Load balancing of resources of virtual machine method and device | |
| EP2755133B1 (en) | Application execution controller and application execution method | |
| Bhatotia et al. | Slider: Incremental sliding-window computations for large-scale data analysis | |
| CN112685170A (en) | Dynamic optimization of backup strategies | |
| US20220342578A1 (en) | Data movement between storage tiers of a clustered storage system based on predicted data access frequency trend patterns | |
| US20220100548A1 (en) | Network performance assurance system and network performance assurance method | |
| CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
| Mehmood et al. | Prediction of cloud computing resource utilization | |
| Madireddy et al. | Machine learning based parallel I/O predictive modeling: A case study on Lustre file systems | |
| Smith | Prediction services for distributed computing | |
| Qazi et al. | Workload prediction of virtual machines for harnessing data center resources | |
| Nguyen et al. | A black-box fork-join latency prediction model for data-intensive applications | |
| CN117931366A (en) | Application programming method, device, equipment and storage medium based on performance perception | |
| CN115913967A (en) | Micro-service elastic scaling method based on resource demand prediction in cloud environment | |
| Handaoui et al. | Salamander: a holistic scheduling of mapreduce jobs on ephemeral cloud resources | |
| CN113946438A (en) | Online migration scheduling method, device, equipment and system based on container integration | |
| Glatard et al. | Probabilistic and dynamic optimization of job partitioning on a grid infrastructure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |