CN112445687A - Blocking detection method of computing equipment and related device - Google Patents
Blocking detection method of computing equipment and related device Download PDFInfo
- Publication number
- CN112445687A CN112445687A CN201910816774.4A CN201910816774A CN112445687A CN 112445687 A CN112445687 A CN 112445687A CN 201910816774 A CN201910816774 A CN 201910816774A CN 112445687 A CN112445687 A CN 112445687A
- Authority
- CN
- China
- Prior art keywords
- data
- detection
- stuck
- training
- marked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a stuck detection method of computing equipment, which comprises the following steps: performing stuck characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked; the performance index data is obtained by preprocessing the received performance index data to be processed; performing pause labeling on data to be labeled according to a statistical algorithm to obtain training data; training a classification model to be trained by adopting training data to obtain a classification detection model; and performing stuck detection on the received performance data to be detected by adopting a classification detection model to obtain a detection result. The data to be marked are firstly screened out and then corresponding marking operation is carried out to obtain training data, and the stuck detection model obtained after training is adopted for detection, so that automatic detection of stuck is realized, and the detection efficiency and accuracy are improved. The application also discloses a stuck detection device of the computing equipment, the computing equipment and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a stuck detection method for a computing device, a stuck detection apparatus, a computing device, and a computer-readable storage medium.
Background
In the field of Virtual machine technology, in order to provide better desktop system experience for users, a Virtual Desktop Infrastructure (VDI) technology is created, which stores all operating system software, application software and user data required by all desktop PCs in a backend server, and gives a specific user through a special management system, the user is connected to desktop resources allocated by the backend server through a dedicated network transport protocol, after connection, the user can directly use a desktop system running in a background on a local terminal, and the use experience is basically consistent with that of a physical computer.
When this technique is applied to provide services to a user, a stuck condition may occur. In order to reduce the stuck condition and improve the user experience, every occurrence of stuck is detected for subsequent improvement. In the prior art, when a virtual machine user experiences a jam, field information is recorded in the form of a snapshot or a log, and the cause of the jam is searched in a mode of manual analysis afterwards to form an experience library for subsequent detection.
However, when the number of times of the detection is more, the prior art cannot realize real-time detection, needs to rely on a large amount of manual experience, and is long in time consumption and large in workload. Meanwhile, due to the limitation of the experience library, the seizure condition of unknown reasons cannot be judged. In addition, the cause of the stuck is many, and the manually analyzed stuck experience base is not necessarily accurate. The existing detection technology is low in efficiency and cannot provide effective detection results.
Therefore, how to improve the efficiency of the detection process and maintain the accuracy of the detection result is a key issue of attention by those skilled in the art.
Disclosure of Invention
The purpose of the application is to provide a stuck detection method of computing equipment, a stuck detection device, the computing equipment and a computer readable storage medium, training data are obtained by screening data to be marked firstly and then carrying out corresponding marking operation, and a stuck detection model obtained after training is adopted for detection, so that automatic detection of stuck is realized, and the detection efficiency and accuracy are improved.
In order to solve the above technical problem, the present application provides a method for detecting a stuck state of a computing device, including:
performing stuck characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked; the performance index data is obtained by preprocessing the received performance index data to be processed;
performing pause labeling on the data to be labeled according to a statistical algorithm to obtain training data;
training a classification model to be trained by adopting the training data to obtain a classification detection model;
and performing stuck detection on the received performance data to be detected by adopting the classification detection model to obtain a detection result.
Optionally, preprocessing the received performance index data to be processed to obtain the performance index data, including:
performing performance index acquisition on computing equipment, and performing format unification processing on the obtained original data to obtain the performance index data to be processed;
preprocessing the performance index data to be processed to obtain the performance index data; the preprocessing comprises de-duplication processing, missing value completion processing and data normalization processing.
Optionally, performing hiton feature screening processing on the performance index data according to the labeled sample data to obtain data to be labeled, including:
performing statistical analysis on the labeled sample data to obtain a statistical analysis result;
determining the stuck characteristic according to the statistical analysis result;
and eliminating data irrelevant to the stuck characteristic in the performance index data to obtain the data to be marked.
Optionally, performing hiton labeling on the data to be labeled according to a statistical algorithm to obtain training data, including:
calculating the data to be marked according to a statistical algorithm to obtain a statistical result;
taking data with the variation larger than the statistical variation in the statistical result in the data to be labeled as abnormal data, and eliminating the abnormal data in the data to be labeled to obtain target data to be labeled;
and marking all target data to be marked in the data to be marked according to the statistical result to obtain the training data.
Optionally, performing hiton labeling on the data to be labeled according to a statistical algorithm to obtain training data, including:
detecting the data to be marked by adopting an anomaly detection algorithm to obtain marked data to be determined;
screening the marking data to be determined according to the statistical result to obtain Kanton performance index data; the statistical result is obtained by calculating the data to be marked;
and marking all Kanton performance index data in the data to be marked to obtain the training data.
Optionally, the training data is used to train a classification model to be trained, so as to obtain a classification detection model, including:
and training the classification model to be trained according to the grid search algorithm and the training data.
Optionally, the method further includes:
and adding the detection result into the training data so as to take the classification detection model as a classification model to be trained, and training the classification model to be trained by adopting the training data to obtain a new classification detection model.
The present application further provides a device for detecting stuck of a computing device, comprising:
the data screening module is used for carrying out blockage characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked;
the data marking module is used for performing pause marking on the data to be marked according to a statistical algorithm to obtain training data;
the model training module is used for training a classification model to be trained by adopting the training data to obtain a classification detection model;
and the stuck detection module is used for performing stuck detection on the received performance data to be detected by adopting the classification detection model to obtain a detection result.
The present application further provides a computing device comprising:
a memory for storing a computer program;
a processor for implementing the steps of the stuck detection method as described above when executing said computer program.
The present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the stuck detection method as described above.
The application provides a stuck detection method of a computing device, which comprises the following steps: performing stuck characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked; the performance index data is obtained by preprocessing the received performance index data to be processed; performing pause labeling on the data to be labeled according to a statistical algorithm to obtain training data; training a classification model to be trained by adopting the training data to obtain a classification detection model; and performing stuck detection on the received performance data to be detected by adopting the classification detection model to obtain a detection result.
The performance index data is subjected to stuck characteristic screening processing according to the marked sample data to obtain data to be marked, irrelevant characteristic data in the performance index data are removed, influence of the irrelevant characteristic is reduced, accuracy of the characteristic data is improved, then stuck conditions of the data to be marked are marked to obtain training data, a classification model to be trained is trained by the training data to obtain a classification detection model, finally the performance data to be detected is subjected to stuck detection by the model to obtain a detection result, stuck detection of computing equipment is realized, data is not analyzed manually, stuck detection depending on subjective manual experience is avoided, objectivity of the stuck detection is improved, accuracy of the detection is maintained, and efficiency of the detection process is improved due to the fact that the computer is adopted for detection, the time cost is reduced.
The application further provides a stuck detection device of a computing device, the computing device and a computer readable storage medium, which have the above beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a stuck detection method of a first computing device according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a second method for detecting stuck at a computing device according to an embodiment of the present disclosure;
FIG. 3 is a flowchart of a third method for detecting stuck at a computing device according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a stuck detection apparatus of a computing device according to an embodiment of the present disclosure.
Detailed Description
The core of the application is to provide a stuck detection method of computing equipment, a stuck detection device, the computing equipment and a computer readable storage medium.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
When this technique is applied to provide services to a user, a stuck condition may occur. In order to reduce the jamming condition and improve the user experience in the prior art, the jamming occurring each time can be detected for subsequent improvement. In the prior art, when a virtual machine user experiences a jam, field information is recorded in the form of a snapshot or a log, and the cause of the jam is searched in a mode of manual analysis afterwards to form an experience library for subsequent detection. However, when the number of times of the detection is more, the prior art cannot realize real-time detection, needs to rely on a large amount of manual experience, and is long in time consumption and large in workload. Meanwhile, due to the limitation of the experience library, the seizure condition of unknown reasons cannot be judged. In addition, the cause of the stuck is many, and the manually analyzed stuck experience base is not necessarily accurate. The existing detection technology is low in efficiency and cannot provide effective detection results.
Therefore, the application provides a stuck detection method of computing equipment, which comprises the steps of firstly carrying out stuck characteristic screening processing on performance index data according to marked sample data to obtain data to be marked, eliminating data of irrelevant characteristics in the performance index data to reduce the influence of the irrelevant characteristics and improve the accuracy of the characteristic data, then carrying out stuck condition marking on the data to be marked to obtain training data, then training a classification model to be trained by adopting the training data to obtain a classification detection model, and finally carrying out stuck detection on the received performance data to be detected through the model to obtain a detection result, thereby realizing the stuck detection of the computing equipment instead of analyzing the data in a manual mode, avoiding the dependence on subjective manual experience to realize the stuck detection, improving the objectivity of the stuck detection and keeping the accuracy of the detection, because the computer is adopted for detection, the efficiency of the detection process is improved, and the time cost is reduced.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for detecting a stuck-at of a computing device according to an embodiment of the present disclosure.
In this embodiment, the method may include:
s101, performing hiton feature screening processing on the performance index data according to the marked sample data to obtain data to be marked; the performance index data is obtained by preprocessing the received performance index data to be processed;
the method mainly comprises the steps of selecting important features from labeled sample data, and then screening performance index data according to the important features, namely removing data which are irrelevant to the important features from the performance index data to obtain data to be labeled. Specifically, the generally acquired performance index data includes a plurality of index data, the selected important feature is also one or more of the index features, and at this time, the index data related to other index features are removed.
For example, the performance index data includes CPU data size, CPU frequency, CPU voltage, memory IO, memory remaining capacity, disk read-write speed, network bandwidth, and the like. Although there may be a number of factors that contribute to whether a computing device is stuck, excessive performance data is not only redundant, but also tends to affect the final determination. Therefore, the correlation between the three performance characteristics of the CPU frequency, the internal memory IO and the disk read-write speed determined according to the marked sample data and the stuck data is high, so that the performance data corresponding to other characteristics in the performance index data are removed to obtain the data to be marked.
Furthermore, the method for determining the important features according to the labeled sample data can select the features related to the katton from the labeled sample data through a statistical method. The important features can be selected from the labeled sample data according to a random forest algorithm, or selected from the labeled sample data by statistical methods such as frequency analysis, threshold analysis, mean analysis, variance analysis and the like, or selected by the two methods. It can be seen that the manner of determining the important features according to the labeled sample data in this step is not unique, and a suitable method may be selected according to the actual application situation, which is not specifically limited herein.
The performance index data is obtained by preprocessing the received performance index data to be processed. That is, in this embodiment, the acquired raw data, that is, the performance data to be processed, may be preprocessed first to obtain the usable performance index data. The preprocessing process may include deduplication processing, missing value incomplete processing, and data normalization processing. And the to-be-processed performance index data is obtained by carrying out format unification processing on the acquired original data.
Therefore, preferably, the process of obtaining the performance indicator data may include:
performing performance index acquisition on computing equipment, and performing format unification processing on the obtained original data to obtain performance index data to be processed; preprocessing the performance index data to be processed to obtain performance index data; the preprocessing comprises de-duplication processing, missing value completion processing and data normalization processing.
The deduplication processing, the missing value completion processing, and the data normalization processing in this step may all adopt any corresponding processing manner provided in the prior art, and are not specifically limited herein.
Optionally, on the basis of acquiring the performance index data, optionally, the process of acquiring the data to be labeled in this step may include:
step one, carrying out statistical analysis on the labeled sample data to obtain a statistical analysis result;
determining the stuck characteristic according to the statistical analysis result;
and step three, eliminating data irrelevant to the stuck characteristic in the performance index data to obtain data to be marked.
Therefore, in the alternative, firstly, the marked data samples are subjected to statistical analysis to obtain a statistical analysis result, then the stuck characteristic is determined according to the statistical analysis result, and finally, the data irrelevant to the stuck characteristic in the performance index data are removed. And the influence of irrelevant data in the performance index data on the detection process is reduced. The marked sample data is subjected to statistical analysis to obtain a statistical analysis result, wherein the statistical analysis result can be obtained by carrying out average value analysis, threshold value analysis and variance analysis on the data, and can also be obtained by carrying out correlation analysis on the data to obtain a final statistical analysis result. Obviously, the corresponding stuck feature, that is, the important feature related to the stuck phenomenon, can be obtained through the statistical analysis result. And finally, removing the characteristic data which is irrelevant to the important characteristic in the performance index data to obtain the data to be marked.
S102, performing pause labeling on data to be labeled according to a statistical algorithm to obtain training data;
on the basis of S101, the method aims to realize the Canton labeling of the data to be labeled to obtain the training data. Namely, data when pause appears in data to be marked is marked, and finally all data are used as training data.
Specifically, in the step, the data can be labeled manually, or the data to be labeled can be screened from the data to be labeled by adopting a statistical method and labeled, or the data to be labeled can be labeled by adopting a classification identification model. It can be seen that the labeling manner in this embodiment is not unique, and the specific labeling manner is not limited. In addition, different labeling methods have different advantages, and the above labeling methods can be mixed for use, for example, manual labeling and statistical labeling can be adopted, or all three methods can be used. Correspondingly, different methods can be adopted to repeatedly label the data so as to improve the labeling accuracy. Specifically, the three methods are adopted to perform unsmooth labeling on the data to be labeled, and the labeling result with the same result of the two or the three methods is selected as the final labeling result to obtain the training data. For example, a data is labeled as stuck by three methods, or by two methods. Otherwise, the case is marked as not stuck.
Therefore, the method for labeling the data to be labeled in the step is not unique, and a suitable labeling mode can be selected in an actual application environment, which is not specifically limited herein.
S103, training the classification model to be trained by adopting training data to obtain a classification detection model;
on the basis of S102, the step mainly adopts training data to train the classification model to be trained, and obtains a classification detection model. The classification model to be trained may be an untrained initial classification model, or may be an introduced trained classification model. That is, the classification model that has been used may be trained continuously according to the obtained training data, and the classification detection model may be updated.
The classification model to be trained used in this embodiment may be any classification model provided in the prior art, and is not specifically limited herein.
In this embodiment, the classification model to be trained may be trained by any model training method provided in the prior art, which is not specifically limited herein.
Optionally, in order to improve the generalization ability of the model and keep the accuracy unchanged, the process of performing model training may include: and training the classification model to be trained according to the grid search algorithm and the training data. That is, the superparameter is continuously adjusted through a grid search algorithm, and the generalization capability of the model is improved.
It is conceivable that if the expected standard of the model is not reached during the training process, the execution of S101 is continued until the model reaches the expected standard.
And S104, performing stuck detection on the received performance data to be detected by adopting a classification detection model to obtain a detection result.
On the basis of S103, this step aims to perform stuck detection using the trained classification detection model to obtain a detection result. The detection result can judge how many times the situation is stuck, or directly judge whether the current situation is stuck. Furthermore, the step can judge whether the frequency of the occurrence of the jamming is greater than the preset frequency, and if so, the occurrence of the jamming is judged. The number of misjudgments is reduced.
In addition, in order to update the classification detection model, the present embodiment maintains the detection accuracy of the model. The method can also comprise the following steps: and adding the detection result into the training data so as to take the classification detection model as a to-be-trained classification model, and training the to-be-trained classification model by adopting the training data to obtain a new classification detection model.
In summary, in the embodiment, data to be labeled is obtained by performing stuck feature screening processing on performance index data according to labeled sample data, data of irrelevant features in the performance index data is removed, influence of the irrelevant features is reduced, accuracy of the feature data is improved, then, the stuck condition of the data to be labeled is labeled to obtain training data, then, training is performed on a classification model to be trained by using the training data to obtain a classification detection model, and finally, stuck detection is performed on the received performance data to be detected by using the model to obtain a detection result, so that stuck detection of computing equipment is realized, instead of analyzing the data manually, stuck detection depending on subjective manual experience is avoided, objectivity of stuck detection is improved, accuracy of detection is maintained, and efficiency of a detection process is improved due to the adoption of computer for detection, the time cost is reduced.
The following further describes a method for detecting stuck-at of a computing device according to an embodiment.
Referring to fig. 2, fig. 2 is a flowchart illustrating a second method for detecting a stuck-at of a computing device according to an embodiment of the present disclosure.
In this embodiment, the method may include:
s201, performing Kanton characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked;
s202, calculating data to be annotated according to a statistical algorithm to obtain a statistical result;
s203, taking the data with the variation larger than the statistical variation in the statistical result in the data to be labeled as abnormal data, and eliminating the abnormal data in the data to be labeled to obtain target data to be labeled;
s204, labeling all target data to be labeled in the data to be labeled according to the statistical result to obtain training data;
s205, training the classification model to be trained by adopting training data to obtain a classification detection model;
and S206, performing stuck detection on the received performance data to be detected by adopting a classification detection model to obtain a detection result.
Specifically, in this embodiment, how to acquire the training data in the application is further described through S202 to S204. In order to further improve the accuracy of labeling the data to be labeled in the embodiment, the situations of label missing and label excess are reduced. In this embodiment, the abnormal data in the data to be annotated is first removed, that is, further filtered, so as to obtain the target data to be annotated. The main reason is that abnormal data similar to the performance index of the stuck performance index exists in the performance index data, and the abnormal data is usually caused by data errors, operation errors and uncontrollable errors. But at this point the computing device has not stuck, so this type of data is handled as anomalous data.
Specifically, in this embodiment, the data to be labeled is calculated to obtain a statistical result, and then the abnormal data is found out through the statistical result and is removed. The method mainly comprises the steps of judging whether the variation of data in the data to be marked is larger than that of the statistical result, and if so, taking the data as abnormal data. Since abnormal data can reach abnormal index data without a data change process, whether the data is abnormal data can be determined by determining whether the data has a data change process. Thus, the amount of change in the data to be annotated may be the speed at which the data changes. Then S203 determines whether the change speed of the data in the data to be labeled is greater than the change speed in the statistical result, and if so, determines that the data is abnormal.
After the abnormal data are removed, all the data with abnormal performance indexes in the data to be marked can be marked as stuck data. The performance index abnormality can be determined by determining whether the performance index is greater than a threshold value.
For the specific implementation of the steps S201, S205, and S206, reference may be made to the content of the foregoing embodiments, and details are not repeated herein.
The following further describes a method for detecting stuck-at of a computing device according to an embodiment.
Referring to fig. 3, fig. 3 is a flowchart illustrating a method for detecting stuck-at in a third computing device according to an embodiment of the present disclosure.
In this embodiment, the method may include:
s301, performing hiton feature screening processing on the performance index data according to the marked sample data to obtain data to be marked;
s302, detecting the data to be marked by adopting an anomaly detection algorithm to obtain marked data to be determined;
s303, screening the to-be-determined labeling data according to the statistical result to obtain Kanton performance index data; wherein, the statistical result is obtained by calculating the data to be annotated;
s304, labeling all Kanton performance index data in the data to be labeled to obtain training data;
s305, training a classification model to be trained by adopting training data to obtain a classification detection model;
s306, performing stuck detection on the received performance data to be detected by adopting a classification detection model to obtain a detection result.
Specifically, in this embodiment, how to acquire the training data in the application is further described through S302 to S304. In order to further improve the accuracy of labeling the data to be labeled in the embodiment, the situations of label missing and label excess are reduced. In this embodiment, the abnormal data in the data to be annotated is first removed, that is, further filtered, so as to obtain the target data to be annotated. The main reason is that abnormal data similar to the performance index of the stuck performance index exists in the performance index data, and the abnormal data is usually caused by data errors, operation errors and uncontrollable errors. But at this point the computing device has not stuck, so this type of data is handled as anomalous data.
Specifically, in this embodiment, data with abnormal performance indexes in the data to be labeled is detected through an abnormality detection algorithm, where the data includes abnormal data and data when a stuck occurs. Therefore, in the subsequent steps, the abnormal data in the to-be-determined labeling data are removed, and the remaining data can be subjected to pause labeling to realize pause labeling of the data. Specifically, the data to be labeled can be calculated to obtain a statistical result, and then the data to be determined labeled is screened according to the statistical result to obtain the katton performance index data. The process of screening according to the statistical result may refer to the content of the foregoing embodiment, and is not described herein again.
For the specific implementation of the steps S301, S305, and S306, reference may be made to the content of the foregoing embodiments, and details are not repeated herein.
The following further illustrates a method for detecting stuck in a computing device provided by the present application by a more specific embodiment.
In this embodiment, the method may include:
s401, performing data acquisition operation to obtain performance index data;
data acquisition refers to that an acquisition Agent collects common performance indexes of a virtual machine under the scene according to actual conditions, wherein the common performance indexes include but are not limited to a CPU, a memory, a disk, a network and the like. The collected data needs to be formatted and processed uniformly, meanwhile, an analyst needs to observe the collected data, and partial marking is carried out according to manual experience.
S402, performing data storage operation on the performance index data;
the data storage means that the collected data after basic preprocessing is stored in batch at different time intervals through a message queue. The purpose of warehousing is to uniformly manage and check original data; and constructing model training data and training a Katon model. Wherein, the Kanton model is a classification detection model.
S403, performing characteristic analysis and pretreatment on the performance index data after being put in storage to obtain data to be marked;
wherein, the preprocessing comprises (1) deduplication processing and deduplication; (2) filling missing values if the missing values are lost or cannot be acquired in the process of acquiring partial values; (3) and (4) selecting and processing important features, namely selecting the important features in the performance index data by adopting a machine learning model and a statistical method. The important features can be selected through a random forest algorithm, acquired through statistical methods such as frequency, threshold comparison and average value analysis, and associated through a histogram. And finally, removing other data. (4) The data normalization processing has different unit dimensions of different types of data, and can cause different final results. The invention adopts min-max normalization to normalize all indexes to be between 0 and 1.
When new data is added, the max and min in the normalization formula can be changed, and the max and min need to be defined again to avoid errors.
S404, performing labeling processing and anomaly detection on data to be labeled to obtain training data;
the purpose of the anomaly detection is to find out anomalous data from unlabeled samples, and then find out stuck samples by statistical methods such as mean value and variance, and label stuck samples. In the step, the data can be manually marked again, so that the accuracy of algorithm detection is further enhanced. In the invention, the adopted anomaly detection algorithm is a one-class-SVM.
S405, performing model training by using training data to obtain a Kanton model;
in this step, there is supervised model training, and the katton information labeled in step S404 is directly adopted. Because seizure and non-seizure are mutually exclusive problems, the invention adopts a classical binary algorithm SVC. The super-parameters are continuously adjusted through a grid search technology, so that the model achieves better generalization capability, and meanwhile, the accuracy is guaranteed. If not, go back to step S403 and repeat the process.
S406, loading a Kanton model;
and forming the cured Katon model into model prediction service in a service loading mode, and providing a Katon prediction API. When the detection is carried out, the API can be directly called to realize the stuck detection. Furthermore, manual experience judgment can be added into the model prediction service, a double-insurance mode of manual experience and a model is realized, and the accuracy of the stuck detection is improved.
And S407, performing stuck detection by adopting a stuck model.
The new data requests the stuck model, and the method also adds a statistical judgment mode. And under the condition that the data acquisition frequency is fixed, if the new data is stuck continuously for preset times, judging that the current machine is stuck. Otherwise, judging that the card is not stuck. This is beneficial to increasing the accuracy of the stuck detection. For example, if it is determined that performance data is stuck for 3 times continuously, it is determined that the performance data is stuck. The final detection result can be used as training data to optimize the Kanton model, and the detection effect of the Kanton model is improved.
It can be seen that, in the embodiment, data to be labeled is obtained by performing stuck feature screening processing on performance index data according to labeled sample data, data of irrelevant features in the performance index data is removed, influence of the irrelevant features is reduced, accuracy of the feature data is improved, then, stuck condition labeling is performed on the data to be labeled to obtain training data, then, training is performed on a classification model to be trained by using the training data to obtain a classification detection model, finally, stuck detection is performed on the received performance data to be detected by using the model to obtain a detection result, stuck detection of computing equipment is realized, data is not analyzed manually, stuck detection is avoided depending on subjective manual experience, objectivity of stuck detection is improved, accuracy of detection is maintained, and efficiency of a detection process is improved due to the adoption of computer for detection, the time cost is reduced.
In the following, a description is given of a stuck detection device of a computing device provided in an embodiment of the present application, and a stuck detection device of a computing device described below and a stuck detection method of a computing device described above may be referred to correspondingly.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a stuck detection apparatus of a computing device according to an embodiment of the present disclosure.
In this embodiment, the method may include:
the data screening module 100 is configured to perform hiton feature screening processing on the performance index data according to the labeled sample data to obtain data to be labeled;
the data labeling module 200 is used for performing pause labeling on data to be labeled according to a statistical algorithm to obtain training data;
the model training module 300 is configured to train a classification model to be trained by using training data to obtain a classification detection model;
and the stuck detection module 400 is configured to perform stuck detection on the received performance data to be detected by using a classification detection model to obtain a detection result.
An embodiment of the present application further provides a computing device, including:
a memory for storing a computer program;
a processor for implementing the steps of the stuck detection method as described in the above embodiments when executing the computer program.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the stuck detection method according to the above embodiment are implemented.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The present application provides a method, an apparatus, a computing device, and a computer-readable storage medium for detecting a stuck state of a computing device. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A method of stuck detection for a computing device, comprising:
performing stuck characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked; the performance index data is obtained by preprocessing the received performance index data to be processed;
performing pause labeling on the data to be labeled according to a statistical algorithm to obtain training data;
training a classification model to be trained by adopting the training data to obtain a classification detection model;
and performing stuck detection on the received performance data to be detected by adopting the classification detection model to obtain a detection result.
2. The katon detection method of claim 1, wherein preprocessing the received performance indicator data to be processed to obtain the performance indicator data comprises:
performing performance index acquisition on computing equipment, and performing format unification processing on the obtained original data to obtain the performance index data to be processed;
preprocessing the performance index data to be processed to obtain the performance index data; the preprocessing comprises de-duplication processing, missing value completion processing and data normalization processing.
3. The stuck detection method according to claim 2, wherein the stuck feature screening processing is performed on the performance index data according to the labeled sample data to obtain data to be labeled, and the method includes:
performing statistical analysis on the labeled sample data to obtain a statistical analysis result;
determining the stuck characteristic according to the statistical analysis result;
and eliminating data irrelevant to the stuck characteristic in the performance index data to obtain the data to be marked.
4. The katton detection method according to any one of claims 1 to 3, wherein performing katton labeling on the data to be labeled according to a statistical algorithm to obtain training data comprises:
calculating the data to be marked according to a statistical algorithm to obtain a statistical result;
taking data with the variation larger than the statistical variation in the statistical result in the data to be labeled as abnormal data, and eliminating the abnormal data in the data to be labeled to obtain target data to be labeled;
and marking all target data to be marked in the data to be marked according to the statistical result to obtain the training data.
5. The katton detection method according to any one of claims 1 to 3, wherein performing katton labeling on the data to be labeled according to a statistical algorithm to obtain training data comprises:
detecting the data to be marked by adopting an anomaly detection algorithm to obtain marked data to be determined;
screening the marking data to be determined according to the statistical result to obtain Kanton performance index data; the statistical result is obtained by calculating the data to be marked;
and marking all Kanton performance index data in the data to be marked to obtain the training data.
6. The katon detection method of claim 5, wherein training a classification model to be trained using the training data to obtain a classification detection model comprises:
and training the classification model to be trained according to a grid search algorithm and the training data.
7. The stuck detection method of claim 6, further comprising:
and adding the detection result into the training data so as to take the classification detection model as a classification model to be trained, and training the classification model to be trained by adopting the training data to obtain a new classification detection model.
8. A jamming detection apparatus of a computing device, comprising:
the data screening module is used for carrying out blockage characteristic screening processing on the performance index data according to the marked sample data to obtain data to be marked;
the data marking module is used for performing pause marking on the data to be marked according to a statistical algorithm to obtain training data;
the model training module is used for training a classification model to be trained by adopting the training data to obtain a classification detection model;
and the stuck detection module is used for performing stuck detection on the received performance data to be detected by adopting the classification detection model to obtain a detection result.
9. A computing device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the katon detection method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, having a computer program stored thereon, which, when being executed by a processor, carries out the steps of the stuck detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910816774.4A CN112445687A (en) | 2019-08-30 | 2019-08-30 | Blocking detection method of computing equipment and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910816774.4A CN112445687A (en) | 2019-08-30 | 2019-08-30 | Blocking detection method of computing equipment and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112445687A true CN112445687A (en) | 2021-03-05 |
Family
ID=74734125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910816774.4A Pending CN112445687A (en) | 2019-08-30 | 2019-08-30 | Blocking detection method of computing equipment and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112445687A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407344A (en) * | 2021-06-25 | 2021-09-17 | 北京字节跳动网络技术有限公司 | Method and device for processing stuck |
| CN113961354A (en) * | 2021-10-29 | 2022-01-21 | 重庆长安汽车股份有限公司 | Machine-based stuck identification method and system based on weak supervision learning vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130198119A1 (en) * | 2012-01-09 | 2013-08-01 | DecisionQ Corporation | Application of machine learned bayesian networks to detection of anomalies in complex systems |
| CN104834597A (en) * | 2015-03-06 | 2015-08-12 | 深信服网络科技（深圳）有限公司 | Method and system for measuring application response duration |
| CN108600790A (en) * | 2018-05-17 | 2018-09-28 | 北京奇艺世纪科技有限公司 | A kind of detection method and device of interim card failure |
| CN108647593A (en) * | 2018-04-26 | 2018-10-12 | 东华大学 | Unmanned plane road surface breakage classification and Detection method based on image procossing and SVM |
| CN108737193A (en) * | 2018-06-05 | 2018-11-02 | 亚信科技（中国）有限公司 | A kind of failure prediction method and device |
| CN109240875A (en) * | 2018-07-12 | 2019-01-18 | 北京百度网讯科技有限公司 | A kind of Caton analysis method and system |
| US20190166024A1 (en) * | 2017-11-24 | 2019-05-30 | Institute For Information Industry | Network anomaly analysis apparatus, method, and non-transitory computer readable storage medium thereof |
-
2019
- 2019-08-30 CN CN201910816774.4A patent/CN112445687A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130198119A1 (en) * | 2012-01-09 | 2013-08-01 | DecisionQ Corporation | Application of machine learned bayesian networks to detection of anomalies in complex systems |
| CN104834597A (en) * | 2015-03-06 | 2015-08-12 | 深信服网络科技（深圳）有限公司 | Method and system for measuring application response duration |
| US20190166024A1 (en) * | 2017-11-24 | 2019-05-30 | Institute For Information Industry | Network anomaly analysis apparatus, method, and non-transitory computer readable storage medium thereof |
| CN108647593A (en) * | 2018-04-26 | 2018-10-12 | 东华大学 | Unmanned plane road surface breakage classification and Detection method based on image procossing and SVM |
| CN108600790A (en) * | 2018-05-17 | 2018-09-28 | 北京奇艺世纪科技有限公司 | A kind of detection method and device of interim card failure |
| CN108737193A (en) * | 2018-06-05 | 2018-11-02 | 亚信科技（中国）有限公司 | A kind of failure prediction method and device |
| CN109240875A (en) * | 2018-07-12 | 2019-01-18 | 北京百度网讯科技有限公司 | A kind of Caton analysis method and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407344A (en) * | 2021-06-25 | 2021-09-17 | 北京字节跳动网络技术有限公司 | Method and device for processing stuck |
| CN113961354A (en) * | 2021-10-29 | 2022-01-21 | 重庆长安汽车股份有限公司 | Machine-based stuck identification method and system based on weak supervision learning vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10031829B2 (en) | Method and system for it resources performance analysis | |
| CN111475370A (en) | Operation and maintenance monitoring method, device and equipment based on data center and storage medium | |
| US8886660B2 (en) | Method and apparatus for tracking a change in a collection of web documents | |
| CN113254255B (en) | Cloud platform log analysis method, system, device and medium | |
| CN111078513B (en) | Log processing method, device, equipment, storage medium and log alarm system | |
| CN110826494A (en) | Method and device for evaluating quality of labeled data, computer equipment and storage medium | |
| US20190213198A1 (en) | Similarity analyses in analytics workflows | |
| CN110275878B (en) | Service data detection method and device, computer equipment and storage medium | |
| CN110995273B (en) | Data compression method, device, equipment and medium for power database | |
| CN112445687A (en) | Blocking detection method of computing equipment and related device | |
| CN113723555A (en) | Abnormal data detection method and device, storage medium and terminal | |
| US9201752B2 (en) | System and method for correlating empirical data with user experience | |
| CN114785616A (en) | Data risk detection method and device, computer equipment and storage medium | |
| CN115184674A (en) | Insulation test method and device, electronic terminal and storage medium | |
| CN113448955B (en) | Data set quality evaluation method and device, computer equipment and storage medium | |
| CN113205130B (en) | Data auditing method and device, electronic equipment and storage medium | |
| CN113033639A (en) | Training method of abnormal data detection model, electronic device and storage medium | |
| CN112365269A (en) | Risk detection method, apparatus, device and storage medium | |
| CN115658441B (en) | Method, equipment and medium for monitoring abnormality of household service system based on log | |
| CN115470034A (en) | Log analysis method, device and storage medium | |
| CN112882854B (en) | Method and device for processing request exception | |
| CN114327266A (en) | Card slow identification method, device and medium of storage device | |
| CN110955710B (en) | Dirty data processing method and device in data exchange operation | |
| CN110674839B (en) | Abnormal user identification method and device, storage medium and electronic equipment | |
| CN114553473A (en) | Abnormal login behavior detection system and method based on login IP and login time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210305 |