CN109614299B - System anomaly detection method and device and electronic equipment - Google Patents
System anomaly detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN109614299B CN109614299B CN201811119691.1A CN201811119691A CN109614299B CN 109614299 B CN109614299 B CN 109614299B CN 201811119691 A CN201811119691 A CN 201811119691A CN 109614299 B CN109614299 B CN 109614299B
- Authority
- CN
- China
- Prior art keywords
- time period
- abnormal
- service request
- current time
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The specification discloses a system anomaly detection method and device and electronic equipment. The method comprises the following steps: obtaining all types of service requests appearing in the current time period of the system and magnitude and historical characteristic data of each type of service request in all types of service requests, and predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model according to the type and the historical characteristic data of each type of service request; calculating the proportion of the abnormal service requests in the current time period according to the abnormal probability and magnitude of each type of service request in all types of service requests; and if the proportion of the abnormal service requests in the current time period is increased steeply relative to the proportion of the abnormal service requests in the previous time period, determining that the system is abnormal and giving an alarm. Based on the technical scheme, the system abnormity detection does not depend on a specific service scene and does not need manual intervention, and the technical problems of high time cost and high labor cost of system abnormity detection in the prior art are solved.
Description
Technical Field
The present disclosure relates to the field of software technologies, and in particular, to a method and an apparatus for detecting system anomaly, and an electronic device.
Background
With the rapid development of the internet technology, the load of a system platform is larger and larger, and negligence and errors in any link can cause system risks, thereby bringing huge losses to companies. In order to avoid system failure, it is usually necessary to perform anomaly detection, and to perform timely and accurate alarm according to an anomaly detection result, so as to help emergency personnel find problems in the shortest time and restore the system to a normal state. Anomaly detection typically detects anomalous traffic requests. The existing abnormal service request is defined as a request which is far from the overall distribution as an abnormal request, the abnormal request needs to be manually defined according to a specific service scene, the time cost and the labor cost are high, and a new abnormal detection method is urgently needed to reduce the time cost and the labor cost.
Disclosure of Invention
The embodiment of the specification provides a system abnormity detection method, a system abnormity detection device and electronic equipment, which are used for reducing time cost and labor cost of system abnormity detection.
In a first aspect, an embodiment of the present specification provides a system anomaly detection method, including:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and the quantity information of each type of service requests in all types of service requests;
acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
In a second aspect, an embodiment of the present specification provides a system anomaly detection apparatus, including:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring all types of service requests appearing in the current time period of the system and the quantity information of each type of service request in all types of service requests; acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
the prediction unit is used for predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model according to the type and the historical characteristic data of each type of service request;
the calculating unit is used for calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and the quantity information of each type of service requests in all types of service requests; acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period;
and the alarm unit is used for determining that the system is abnormal and giving an alarm under the condition that the ratio increase parameter is greater than a set threshold value.
In a third aspect, the embodiments of the present specification provide a computer-readable storage medium, on which a computer program is stored, and the program, when executed by a processor, implements the following steps:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each type of service requests in all types of service requests;
acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period before the current time period;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
In a third aspect, embodiments of the present specification provide an electronic device, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory, and configured to be executed by the one or more processors includes instructions for:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each type of service requests in all types of service requests;
acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
One or more technical solutions in the embodiments of the present specification have at least the following technical effects:
the embodiment of the specification provides a system anomaly detection method, which is used for acquiring all types of service requests appearing at the current moment of a system and quantity information of each type of service request in all types of service requests; acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request; according to the category and historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model; calculating and obtaining the proportion of the abnormal service requests at the current moment according to the abnormal probability and the quantity information of each type of service requests in all types of service requests; obtaining an occupation ratio increase parameter of the abnormal service request at the current moment according to the occupation ratio of the abnormal service request at the current moment and the occupation ratio of the abnormal service request at each moment in a preset time period before the current moment; if the percentage increase parameter is larger than the set threshold, determining that the system is abnormal and giving an alarm, namely, the technical scheme detects the abnormal probability of the service request at the current moment, obtains the abnormal service percentage according to the abnormal probability, and determines whether the system is abnormal according to the increase condition of the abnormal service percentage, so that the system abnormal detection does not need to define the abnormal request according to a specific service scene, does not depend on the specific service scene, does not need manual intervention, solves the technical problems of high time cost and labor cost of system abnormal detection in the prior art, and reduces the time cost and labor cost of system abnormal detection.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present specification, a brief description will be given below of the embodiments or the drawings required in the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present specification, and it is obvious for a person skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a flowchart of a system anomaly detection method provided in an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a system abnormality detection apparatus according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a training apparatus for an abnormal request detection model according to an embodiment of the present disclosure;
fig. 4 is a schematic view of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step are within the scope of the present specification.
The embodiment of the specification provides a system abnormality detection method, a system abnormality detection device and an electronic device, which are used for reducing the time cost and the labor cost of system abnormality detection.
The main implementation principle, the specific implementation mode and the corresponding beneficial effects of the technical solutions of the embodiments of the present specification are described in detail below with reference to the accompanying drawings.
Description of the professional terms
Access parameters: to complete a service request, the system needs to perform multiple function calls, each function call is given several entries, the function returns an entry, and the entry are also called entry parameters;
coding access parameters: n is required to complete a service request1An access parameter from n1Reselecting m according to the access parameters1The information summarization algorithm md5 coding is carried out on the main access parameters, and the coding is named shortstructured 5, shortstructured 5 represents a type of service request in the access parameter dimension;
and (3) entering parameter coding: from n2Selecting m from the ginseng2The main parameters are coded by an information summarization algorithm md5, and the codes are named as inputparamd 5, and inputparamd 5 represents a class of requests in the entry dimension;
and (3) abnormal service request: the service request which can cause system failure or the service request which represents that the system has failed is an abnormal service request, the frequency of the abnormal service request is low, and the abnormal service request has no following regularity.
The embodiment of the present specification provides a new definition of an abnormal service request: the service request with lower frequency is an abnormal service request, or the service request with frequency lower than the set threshold is an abnormal service request. Normally, a normal service request will occur frequently, and the frequency of an abnormal service request should be low. For example, a request for a successful transfer by bank normally occurs daily, while a request for a failed transfer by bank because the bank transfer channel is not available, is often infrequent in either history or future when the system is operating properly. If the low frequency traffic request is increasing rapidly at a certain moment, the reason for this is likely to be a system failure. Therefore, the embodiments of the present disclosure provide a system anomaly detection method, which determines whether a system fails and needs to alarm based on a low-frequency service request ratio increase condition.
Before system anomaly detection is performed, an anomaly request detection model is trained in advance in the embodiments of the present specification, and is used for predicting the probability that a certain type of service request is an abnormal service request. The probability that a certain class of service requests is an abnormal service request is referred to as an abnormal probability hereinafter.
The training method of the abnormal request detection model specifically comprises the following steps:
1. obtaining training samples
| Observation period | Time of sample | Period of performance | |
| Training sample | T-n | T+0 | T+m |
| Test specimen | T-n+1 | T+1 | T+m+1 |
Watch 1
Referring to table one, when a training sample is obtained, an observation period n and a presentation period m need to be constructed and determined, if service request data in a time period T is taken for training, a sample time corresponding to the service request data is T, the service request data in the observation period T-n is used for obtaining historical characteristic data of the sample, and the service request data in the presentation period T + m is used for obtaining an abnormal probability of the sample. Different service scenes, the observation period n and the presentation period m may have different lengths, for example: for a certain business scene, 1 week is taken as a presentation period, and the observation period is 4 weeks (28 days) in the past, namely m is 7 days, and n is 28 days. And acquiring the category, quantity information, historical characteristic data and abnormal probability of each type of service request in all the service requests in the time period T.
The abnormal probability of each type of service data is calibrated according to whether the similar service request occurs in a subsequent preset time period, i.e. a presentation period, for example: if the service request does not appear in the T +7 days, the calibration result label is 1, and the service request is an abnormal service request and the abnormal probability is 1.
The category of each type of service request can be characterized by acquiring the access code shortstructure 5 and/or the access code inputparamd 5 of the service request.
The quantity information of each type of service request may be the quantity of each type of service request, or may be an order of magnitude of the quantity of each type of service request, for example: if the number of the service requests is 0-101) The corresponding service requests are of the order of 0 and the number is 101～102) The corresponding service requests are of the order of 1 and the number is 102～103) The corresponding service request is of the order of 2 and so on.
The historical characteristic data of each type of service request can acquire the number of days and magnitude of each type of service request in the kth preset time period before the time period T according to the historical service request record, wherein k is more than or equal to 1, and the length of the preset time period is set according to a specific service scene; and/or obtaining the magnitude ratio of each type of service request on the kth day in the kth preset time period before the time period T according to the historical service request records, wherein the T belongs to [1, k ]. The magnitude ratio is the ratio between the magnitude of each type of service request on the t-th day and the maximum magnitude of the service request in the corresponding time period. For example: taking a preset time period of 7 days as an example, the historical characteristic data of each type of service request comprises:
number of days in history 7 days before time period T where such service request occurs
Magnitude of occurrence of such service request in historical 7 days before time period T
The magnitude ratio of the occurrence of such service requests on the first day, e.g., monday, of historical 7 days prior to time period T
The magnitude ratio of the occurrence of such service requests on the second day, e.g., tuesday, of the historical 7 days prior to time period T
……
Number of days in history 14 days before time period T where such service request occurs
Magnitude of historical 14 days before time period T of occurrence of such service requests
……
The sample data contained in each sample is shown in table two below:
watch two
After a single sample is constructed, all samples can form a sample set, and the training data combined by the samples can be described by the following matrix:
and when the sample set is constructed, the type and the historical characteristic data of each sample in the sample set are used as input data of a classification model, the abnormal probability is used as a calibration result of the classification model for model training, and an abnormal request detection model is obtained through training. The specific model training can be performed by using two classification methods such as GBDT and XGboost.
After the anomaly request detection model is trained, the anomaly request detection model can be further tested through a test sample. The test specimen can be constructed in the same manner according to the time set of Table one above. And using the abnormal request detection model after the test is passed for predicting whether the service request is abnormal or not. When an abnormal request detection model is used for prediction, the same characteristics need to be constructed for samples to be predicted, the prediction result is a floating point number between 0 and 1, namely abnormal probability, and the larger the abnormal probability is, the lower the probability of the request appearing in the next preset time period is, namely the higher the abnormal degree is.
Based on a pre-trained anomaly request detection model, an embodiment of the present specification provides a system anomaly detection method, please refer to fig. 1, which includes the following steps S110 to S160:
s110: and acquiring all types of service requests occurring in the current time period of the system and the magnitude of each type of service request in all types of service requests.
The embodiment does not limit the specific duration of the current time period, and may be 1 minute, 1 hour, 1 day, and the like. The category of the service request is represented by an access code and/or an access code of the service request.
S120: and acquiring historical characteristic data of each type of service request according to the historical service request record, wherein the historical characteristic data represents the periodic information of each type of service request. The historical characteristic data is obtained in the same way as the historical characteristic data in the sample.
S130: and predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model according to the type and the historical characteristic data of each type of service request.
Specifically, the category of each type of service request, such as an access code and/or an access code, and historical characteristic data, are input into an abnormal request detection model for prediction, and a predicted value output by the abnormal request detection model is obtained as the abnormal probability of the service request.
S140: and calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and the magnitude of each type of service request in all types of service requests.
Specifically, the proportion of the abnormal service request in the current time period may be obtained by any one of the following methods:
in a first mode
Wherein r isjRepresenting the proportion of abnormal service requests in the current time period, N representing the number of types of service requests in the current time period, wiIndicates the abnormal probability of the ith class service request in all classes of service requests, hiIndicating the magnitude of the class i service request, i ∈ [1, N]。
Mode two
Obtaining the abnormal probability greater than the preset probability threshold value in the current time periodService request, namely abnormal service request, and obtaining the number E of the types of the abnormal service requests and the magnitude h of the E-th abnormal service requestse，e∈[1,E]Calculating and obtaining the proportion of abnormal service requests in the current time period according to the following formula:
wherein r isjRepresenting the proportion of abnormal service requests in the current time period, N representing the number of the types of the service requests in the current time period, wiIndicates the abnormal probability of the ith class service request in all classes of service requests, hiIndicating the magnitude of the class i service request, i ∈ [1, N]，weThe abnormal probability of the e-th abnormal service request in all the abnormal service requests is expressed, heRepresenting the magnitude of the class i service request, E ∈ [1, E]。
S150: and obtaining the percentage increase parameter of the abnormal service request in the current time period according to the percentage of the abnormal service request in the current time period and the percentage of the abnormal service request in each time period in a preset time period before the current time period.
The proportion increase parameter is a value representing the increase of the proportion of the abnormal service request relative to the proportion of the abnormal service request in the past period. Specifically, the mean value and the standard deviation of the proportion of the abnormal service requests in the preset time period before the current time period can be obtained according to the proportion of the abnormal service requests in each time period before the current time period; and acquiring the difference value between the ratio of the abnormal service request in the current time period and the mean value and the ratio between the difference value and the standard deviation, and taking the ratio as a ratio increase parameter.
For example: assuming that 1 minute is taken as 1 time period, the average of the proportion of 30 abnormal service requests 30 minutes before the current time period is muiStandard deviation of σiIt should be noted that the preset time period may be long or short, and may be n times the length of the current time period. Calculating a percentage increase parameter
S160: and judging whether the ratio increase parameter is larger than a set threshold value or not. And if the percentage increase parameter of the abnormal service request in the current time period is greater than a set threshold value, determining that the system is abnormal and giving an alarm. And otherwise, if the percentage increase parameter of the abnormal service request in the current time period is not greater than the set threshold, determining that the system is normal and not alarming.
In the embodiment, the system anomaly detection changes the idea, a new definition is made for the abnormal service request, the request which is far from the overall distribution is no longer used as the abnormal service request, and the request with lower occurrence frequency is defined as the abnormal service request. The definition of the abnormal service request is generally used in each service scene, and the interpretability is strong. Moreover, based on the definition of the abnormal service request, the unsupervised abnormality detection method is successfully converted into the supervised abnormality detection method, so that the detection effect of the system is easier to adjust and more stable.
Based on the above system anomaly detection method provided in the embodiment, the embodiment further provides a system anomaly detection apparatus, referring to fig. 2, the apparatus includes:
an obtaining unit 21, configured to obtain all categories of service requests occurring in a current time period of the system and a magnitude of each category of service requests in all categories of service requests; acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
the prediction unit 22 is configured to predict and obtain an abnormal probability of each type of service request through a pre-trained abnormal request detection model according to the type and historical feature data of each type of service request;
the calculating unit 23 is configured to calculate, according to the abnormal probability and magnitude of each type of service request in all types of service requests, a ratio of the abnormal service requests in the current time period; acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period;
and the alarm unit 24 is used for determining the system abnormity and giving an alarm under the condition that the ratio increase parameter is larger than a set threshold value.
As an optional implementation manner, the calculating unit 23 is specifically configured to:
acquiring the mean value and the standard deviation of the proportion of the abnormal service requests in a preset time period before the current time period according to the proportion of the abnormal service requests in each time period before the current time period; and acquiring a difference value between the ratio of the abnormal service request in the current time period and the mean value and a ratio value between the difference value and the standard deviation, and taking the ratio value as the ratio increase parameter. When calculating the percentage increase parameter, the calculating unit 23 may be specifically configured to calculate according to the following formula:
wherein r isjRepresenting the proportion of the abnormal service requests in the current time period, N representing the number of the types of the service requests in the current time period, wiIndicates the abnormal probability of the ith class service request in all classes of service requests, hiRepresenting the magnitude of said class i service request, i ∈ [1, N]。
As an alternative embodiment, the obtaining unit 21 is configured to obtain the following data as the historical feature data:
acquiring the number of days and magnitude of each type of service request in the kth preset time period before the current time period according to the historical service request records, wherein k is more than or equal to 1; and/or obtaining the magnitude ratio of each type of service request in the kth day in a kth preset time period before the current time period according to the historical service request records, wherein the t belongs to [1, k ].
As an alternative embodiment, the apparatus further comprises a training unit 25 for:
obtaining a sample set according to historical service request records, wherein each sample in the sample set comprises the category, historical characteristic data and abnormal probability of a class of service requests; and performing model training by using the category data and the historical characteristic data of each sample in the sample set as input data of a classification model and using the abnormal probability as a calibration result of the classification model to obtain the abnormal request detection model.
Similarly, based on the above training method for the abnormal request detection model, this embodiment further provides a training apparatus for the abnormal request detection model, referring to fig. 3, where the apparatus includes:
the system comprises a sample acquisition module 31, a service request processing module and a service request processing module, wherein the sample acquisition module 31 is used for acquiring a sample set according to a historical service request record, and each sample in the sample set comprises category data, historical characteristic data and abnormal probability of a class of service request, the historical characteristic data represents periodic information of each class of service request, and the abnormal probability is calibrated according to whether the similar service request occurs in a subsequent preset time period;
the training module 32 is configured to perform model training by using the category and the historical feature data of each sample in the sample set as input data of a classification model, and using the abnormal probability as a calibration result of the classification model, so as to obtain the abnormal request detection model.
As an optional implementation, the sample acquiring module is further configured to acquire, as the historical feature data, the following data:
acquiring the number of days and magnitude of each type of service request in the kth preset time period before the time period T of the sample according to the historical service request record, wherein k is more than or equal to 1; and/or acquiring the magnitude ratio of each type of service request occurring at the kth day in the kth preset time period before the time period T of the sample according to the historical service request records, wherein T belongs to [1, k ].
With regard to the apparatus in the above embodiments, the specific manner in which each unit and module performs operations has been described in detail in the embodiments of the related method, and is not set forth in detail herein.
Referring to fig. 4, a block diagram of an electronic device 700 for implementing a data query method is shown, according to an example embodiment. For example, the electronic device 700 may be a computer, database console, tablet device, personal digital assistant, and the like.
Referring to fig. 4, electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an input/output (I/O) interface 710, and a communication component 712.
The processing component 702 generally controls overall operation of the electronic device 700, such as operations associated with display, data communication, and recording operations. The processing element 702 may include one or more processors 720 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 702 may include one or more modules that facilitate interaction between the processing component 702 and other components.
The memory 704 is configured to store various types of data to support operation at the device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 704 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 706 provides power to the various components of the electronic device 700. The power components 706 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 700.
The I/O interface 710 provides an interface between the processing component 702 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The communication component 712 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication part 712 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 712 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as the memory 704 comprising instructions, executable by the processor 720 of the electronic device 700 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
A non-transitory computer-readable storage medium having instructions therein, which when executed by a processor of a mobile terminal, enable an electronic device to perform a data query method, the method comprising:
acquiring all types of service requests appearing in the current time period of the system and the magnitude of each type of service request in all types of service requests; acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request; according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model; calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and magnitude of each type of service request in all types of service requests; acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period; and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is only limited by the appended claims
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (14)
1. A method of system anomaly detection, the method comprising:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each type of service requests in all types of service requests;
obtaining an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period, wherein the occupation ratio increase parameter comprises: acquiring the mean value and the standard deviation of the proportion of the abnormal service requests in a preset time period before the current time period according to the proportion of the abnormal service requests in each time period before the current time period; acquiring a difference value between the ratio of the abnormal service request in the current time period and the mean value and a ratio value between the difference value and the standard deviation, and taking the ratio value as the ratio increase parameter;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
2. The method of claim 1, wherein the calculating the fraction of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each service request in all the classes of service requests comprises:
wherein r isjRepresenting the proportion of the abnormal service requests in the current time period, N representing the number of the types of the service requests in the current time period, wiRepresents the abnormal probability of the ith service request in all the service requests of the category hiRepresenting the magnitude of said class i service request, i ∈ [1, N]。
3. The method of claim 1, wherein obtaining historical feature data of each type of service request according to a historical service request record comprises:
acquiring the number of days and magnitude of each type of service request in a kth preset time period before the current time period according to historical service request records, wherein k is more than or equal to 1; and/or
And according to the historical service request record, obtaining the magnitude ratio of each type of service request in the kth preset time period before the current time period, wherein the t belongs to [1, k ].
4. The method of any of claims 1-3, wherein the method of training the anomaly request detection model comprises:
obtaining a sample set according to historical service request records, wherein each sample in the sample set comprises the category, historical characteristic data and abnormal probability of a class of service requests;
and performing model training by using the category data and the historical characteristic data of each sample in the sample set as input data of a classification model and using the abnormal probability as a calibration result of the classification model to obtain the abnormal request detection model.
5. A training method of an abnormal request detection model, which is applied before the system abnormal detection method according to any one of claims 1-4, comprising:
obtaining a sample set according to a historical service request record, wherein each sample in the sample set comprises category data, historical characteristic data and abnormal probability of a class of service requests, the historical characteristic data represents periodic information of each class of service requests, and the abnormal probability is calibrated according to whether the class of service requests appear in a subsequent preset time period;
and performing model training by taking the category and the historical characteristic data of each sample in the sample set as input data of a classification model and taking the abnormal probability as a calibration result of the classification model to obtain the abnormal request detection model.
6. The method of claim 5, wherein the historical feature data is obtained by a method comprising:
acquiring the number of days and magnitude of each type of service request in the kth preset time period before the time period T of the sample according to the historical service request record, wherein k is more than or equal to 1; and/or the presence of a gas in the gas,
and according to the historical service request record, obtaining the magnitude ratio of each type of service request occurring at the kth day in the kth preset time period before the time period T of the sample, wherein T belongs to [1, k ].
7. A system anomaly detection apparatus, said apparatus comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring all types of service requests appearing in the current time period of the system and the quantity information of each type of service request in all types of service requests; acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
the prediction unit is used for predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model according to the type and the historical characteristic data of each type of service request;
the calculating unit is used for calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and the quantity information of each type of service requests in all types of service requests; acquiring an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period; the computing unit is specifically configured to: acquiring the mean value and the standard deviation of the proportion of the abnormal service requests in a preset time period before the current time period according to the proportion of the abnormal service requests in each time period before the current time period; acquiring a difference value between the ratio of the abnormal service request in the current time period and the mean value and a ratio value between the difference value and the standard deviation, and taking the ratio value as the ratio increase parameter;
and the alarm unit is used for determining that the system is abnormal and giving an alarm under the condition that the ratio increase parameter is greater than a set threshold value.
8. The apparatus of claim 7, the computing unit further to:
wherein r isjRepresenting the proportion of the abnormal service requests in the current time period, N representing the number of the types of the service requests in the current time period, wiRepresents the abnormal probability of the ith service request in all the service requests of the category hiRepresenting the magnitude of said class i service request, i ∈ [1, N]。
9. The apparatus according to claim 7, said acquisition unit being configured to acquire, as the history feature data:
acquiring the number of days and magnitude of each type of service request in the kth preset time period before the current time period according to the historical service request records, wherein k is more than or equal to 1; and/or
And according to the historical service request record, obtaining the magnitude ratio of each type of service request in the kth preset time period before the current time period, wherein the t belongs to [1, k ].
10. The apparatus according to any of claims 7-9, the apparatus further comprising a training unit for:
obtaining a sample set according to historical service request records, wherein each sample in the sample set comprises the category, historical characteristic data and abnormal probability of a class of service requests;
and performing model training by using the category data and the historical characteristic data of each sample in the sample set as input data of a classification model and using the abnormal probability as a calibration result of the classification model to obtain the abnormal request detection model.
11. A training apparatus for an abnormal request detection model, applied before the system abnormal detection apparatus according to any one of claims 7-10, comprising:
the system comprises a sample acquisition module, a data processing module and a data processing module, wherein the sample acquisition module is used for acquiring a sample set according to historical service request records, and each sample in the sample set comprises category data, historical characteristic data and abnormal probability of a class of service requests, wherein the historical characteristic data represents periodic information of each class of service requests, and the abnormal probability is calibrated according to whether the similar service requests appear in a subsequent preset time period;
and the training module is used for performing model training by taking the category and the historical characteristic data of each sample in the sample set as input data of a classification model and taking the abnormal probability as a calibration result of the classification model to obtain the abnormal request detection model.
12. The apparatus of claim 11, the sample acquisition module further configured to acquire, as historical feature data:
acquiring the number of days and magnitude of each type of service request in the kth preset time period before the time period T of the sample according to the historical service request record, wherein k is more than or equal to 1; and/or the presence of a gas in the gas,
and according to the historical service request record, obtaining the magnitude ratio of each type of service request occurring at the kth day in the kth preset time period before the time period T of the sample, wherein T belongs to [1, k ].
13. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, performing the steps of:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each type of service requests in all types of service requests;
obtaining an occupation ratio increase parameter of the abnormal service request in the current time period according to the occupation ratio of the abnormal service request in the current time period and the occupation ratio of the abnormal service request in each time period in a preset time period before the current time period, wherein the occupation ratio increase parameter comprises: acquiring the mean value and the standard deviation of the proportion of the abnormal service requests in a preset time period before the current time period according to the proportion of the abnormal service requests in each time period before the current time period; acquiring a difference value between the ratio of the abnormal service request in the current time period and the mean value and a ratio value between the difference value and the standard deviation, and taking the ratio value as the ratio increase parameter;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
14. An electronic device comprising memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors the one or more programs comprising instructions for:
acquiring all types of service requests appearing in the current time period of the system and quantity information of each type of service request in all types of service requests;
acquiring historical characteristic data of each type of service request according to a historical service request record, wherein the historical characteristic data represents periodic information of each type of service request;
according to the category and the historical characteristic data of each type of service request, predicting and obtaining the abnormal probability of each type of service request through a pre-trained abnormal request detection model;
calculating and obtaining the proportion of the abnormal service requests in the current time period according to the abnormal probability and quantity information of each type of service requests in all types of service requests;
obtaining an increase parameter of the proportion of the abnormal service request in the current time period according to the proportion of the abnormal service request in the current time period and the proportion of the abnormal service request in each time period before the current time period, wherein the increase parameter comprises: acquiring the mean value and the standard deviation of the proportion of the abnormal service requests in a preset time period before the current time period according to the proportion of the abnormal service requests in each time period before the current time period; acquiring a difference value between the ratio of the abnormal service request in the current time period and the mean value and a ratio value between the difference value and the standard deviation, and taking the ratio value as the ratio increase parameter;
and if the percentage increase parameter is larger than a set threshold value, determining that the system is abnormal and giving an alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811119691.1A CN109614299B (en) | 2018-09-25 | 2018-09-25 | System anomaly detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811119691.1A CN109614299B (en) | 2018-09-25 | 2018-09-25 | System anomaly detection method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614299A CN109614299A (en) | 2019-04-12 |
| CN109614299B true CN109614299B (en) | 2022-05-31 |
Family
ID=66002472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811119691.1A Active CN109614299B (en) | 2018-09-25 | 2018-09-25 | System anomaly detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614299B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111835696B (en) * | 2019-04-23 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Method and device for detecting abnormal request individuals |
| CN110458581B (en) * | 2019-07-11 | 2024-01-16 | 创新先进技术有限公司 | Method and device for identifying business turnover abnormality of commercial tenant |
| CN112994960B (en) * | 2019-12-02 | 2022-09-16 | 中国移动通信集团浙江有限公司 | Method and device for detecting business data abnormity and computing equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017051575A1 (en) * | 2015-09-25 | 2017-03-30 | 三菱重工業株式会社 | Abnormality diagnosis system and abnormality diagnosis method |
| CN106874280A (en) * | 2015-12-10 | 2017-06-20 | 博雅网络游戏开发（深圳）有限公司 | The alarm method and device of abnormal data |
| CN107526666A (en) * | 2017-07-17 | 2017-12-29 | 阿里巴巴集团控股有限公司 | Alarm method, system, device and electronic equipment based on deep learning |
| CN108563548A (en) * | 2018-03-19 | 2018-09-21 | 阿里巴巴集团控股有限公司 | Method for detecting abnormality and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9349103B2 (en) * | 2012-01-09 | 2016-05-24 | DecisionQ Corporation | Application of machine learned Bayesian networks to detection of anomalies in complex systems |
| CN107154880B (en) * | 2016-03-03 | 2020-12-15 | 创新先进技术有限公司 | System monitoring method and device |
-
2018
- 2018-09-25 CN CN201811119691.1A patent/CN109614299B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017051575A1 (en) * | 2015-09-25 | 2017-03-30 | 三菱重工業株式会社 | Abnormality diagnosis system and abnormality diagnosis method |
| CN106874280A (en) * | 2015-12-10 | 2017-06-20 | 博雅网络游戏开发（深圳）有限公司 | The alarm method and device of abnormal data |
| CN107526666A (en) * | 2017-07-17 | 2017-12-29 | 阿里巴巴集团控股有限公司 | Alarm method, system, device and electronic equipment based on deep learning |
| CN108563548A (en) * | 2018-03-19 | 2018-09-21 | 阿里巴巴集团控股有限公司 | Method for detecting abnormality and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614299A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108055281B (en) | Account abnormity detection method, device, server and storage medium | |
| CN109614299B (en) | System anomaly detection method and device and electronic equipment | |
| US20170371757A1 (en) | System monitoring method and apparatus | |
| US20170278382A1 (en) | Risk early warning method and apparatus | |
| US20120284211A1 (en) | Identifying abnormalities in resource usage | |
| CN109543891B (en) | Method and apparatus for establishing capacity prediction model, and computer-readable storage medium | |
| CN110674009A (en) | Application server performance monitoring method and device, storage medium and electronic equipment | |
| US8229692B2 (en) | Methods, systems, and computer-readable media for facility integrity testing | |
| CN112131381A (en) | Method and device for identifying high-alarm-level place, electronic equipment and storage medium | |
| CN113641994A (en) | Data processing method and system based on graph data | |
| CN108900339B (en) | Method and device for measuring service quality and electronic equipment | |
| CN114662772A (en) | Traffic noise early warning method, model training method, device, equipment and medium | |
| JP6147060B2 (en) | Outlier cause determination apparatus and outlier cause determination method | |
| CN111427878B (en) | Data monitoring alarm method, device, server and storage medium | |
| CN109783313B (en) | System exception handling method and system | |
| CN109508356B (en) | Data abnormality early warning method, device, computer equipment and storage medium | |
| CN114968696A (en) | Index monitoring method, electronic equipment and chip system | |
| CN113568773B (en) | Abnormal service classification method, device, equipment and storage medium | |
| CN108762960B (en) | Mobile application monitoring system | |
| US20220245567A1 (en) | Asset tracking management system for categorizing a dormant status of an asset | |
| CN116566034B (en) | Distribution network distribution monitoring system and method | |
| CN111369346B (en) | User credit evaluation method, device, server and storage medium | |
| US20220108196A1 (en) | Improved computer-implemented event forecasting and information provision | |
| CN116886498A (en) | Fault alarm processing method, device, equipment and storage medium | |
| CN114579825A (en) | Data abnormity identification method, system, electronic device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |