CN116450471A - Alarm method and device for abnormal log, storage medium and computer equipment - Google Patents
Alarm method and device for abnormal log, storage medium and computer equipment Download PDFInfo
- Publication number
- CN116450471A CN116450471A CN202310423282.5A CN202310423282A CN116450471A CN 116450471 A CN116450471 A CN 116450471A CN 202310423282 A CN202310423282 A CN 202310423282A CN 116450471 A CN116450471 A CN 116450471A
- Authority
- CN
- China
- Prior art keywords
- alarm
- log
- log data
- level
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 75
- 238000013145 classification model Methods 0.000 claims abstract description 46
- 238000012545 processing Methods 0.000 claims abstract description 31
- 230000001960 triggered effect Effects 0.000 claims abstract description 11
- 238000013527 convolutional neural network Methods 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 18
- 238000012790 confirmation Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 10
- 230000006399 behavior Effects 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses an alarm method and device for an abnormal log, a storage medium and computer equipment, relates to the technical field of data monitoring and digital medical treatment, and mainly aims to solve the problem of poor alarm accuracy of the existing abnormal log. Comprising the following steps: acquiring at least one group of log data generated by an application program; carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels; and when the alarm matching waiting time is triggered and the log data is matched with a log alarm matching strategy, generating abnormal alarm information of the log data.
Description
Technical Field
The present invention relates to the field of data monitoring and digital medical technology, and in particular, to an abnormality log alarming method and apparatus, a storage medium, and a computer device.
Background
With the advent of the big data age, diversified application program layers are endless, and log recording becomes an essential function of an application program, namely, an application system monitors the application program by recording a data log, so that the application system supports functions of disease auxiliary diagnosis, health management, remote consultation and the like.
At present, after the existing medical application program generates a log, the content of the log is judged based on alarm rules, and the alarm triggering action is determined to repair, so that the normal operation of the medical application program is ensured. However, since the log alarm platform classifies all logs, so that massive logs are operated simultaneously, the processing speed of the medical system is greatly influenced, the identification accuracy of important alarm demands is reduced, and the log cannot be rapidly and accurately alarmed, so that the abnormal log alarm accuracy is poor.
Disclosure of Invention
In view of the above, the present invention provides an alarm method and apparatus for an abnormal log, a storage medium, and a computer device, and aims to solve the problem of poor alarm accuracy of an abnormal log in the existing medical application.
According to one aspect of the present invention, there is provided an alarm method of an anomaly log, including:
Acquiring at least one group of log data generated by an application program;
carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels;
and when the alarm matching waiting time is triggered and the log data is matched with a log alarm matching strategy, generating abnormal alarm information of the log data.
Further, before the determining the alarm matching waiting duration corresponding to the alarm level label, the method further includes:
determining at least one of an application type, a client type, a service type and a device type of the log data;
verifying the alarm level label according to at least one of the application type, the client type, the service type and the equipment type;
dividing preset time lengths according to the number of the alarm level labels and the alarm levels to obtain a plurality of alarm matching waiting time lengths, and establishing a matching relation between the alarm level labels and the alarm matching waiting time lengths so as to determine the alarm matching waiting time lengths corresponding to the alarm level labels based on the matching relation.
Further, before the generating of the abnormal alarm information of the log data, the method further includes:
starting waiting time;
and when the timing duration of waiting to be timed matches the alarm matching waiting duration, invoking a log alarm matching policy according to at least one of the application type, the client type, the service type and the equipment type, wherein the log alarm matching policy comprises at least one of an application alarm matching sub-policy, a client alarm matching sub-policy, a service alarm matching sub-policy and an equipment alarm matching sub-policy.
Further, before the log data is subjected to classification marking processing by the alarm classification model based on the completed model training to obtain the alarm level label, the method further comprises:
obtaining a log training sample, and constructing a three-layer convolutional neural network, wherein the log training sample comprises log data marked with at least three alarm level labels;
and carrying out model training on the three-layer convolutional neural network according to the log training sample to obtain an alarm classification model, wherein the hierarchical weight of the three-layer convolutional neural network is configured based on the at least three alarm level labels.
Further, the method further comprises:
if the number of the alarm level labels is three, respectively configuring a first level weight, a second level weight and a third level weight into first weight values corresponding to the alarm level labels, wherein the sum of the first weight values is smaller than 1;
if the number of the alarm level labels is four, respectively configuring a first level weight and a second level weight as a second weight value corresponding to the alarm level label, and configuring a third level weight as a third weight value corresponding to the alarm level label, wherein the second weight value is larger than the third weight value, and the sum of the second weight value and the third weight value is equal to 1.
Further, after the generating the abnormal alarm information of the log data, the method further includes:
receiving a confirmation message of the abnormal alarm information, wherein the confirmation message carries instruction content for updating or confirming the alarm level label of the log data;
and updating the log training sample according to the updated or confirmed alarm level label, and carrying out model training on the alarm classification model again according to a preset time interval based on the updated log training sample.
Further, the acquiring at least one set of log data generated by the application program includes;
when receiving a log data stream transmitted by the application program, identifying an alarm key identifier in the log data stream, wherein the alarm key identifier comprises at least one of an alarm key word, an alarm key character string and an alarm key symbol;
and when the alarm key identification is identified, grouping the log data streams according to a preset data processing unit to obtain log data in a group unit, and performing classification marking processing of an alarm classification model based on each group of log data.
According to another aspect of the present invention, there is provided an alarm device for an anomaly log, including:
the acquisition module is used for acquiring at least one group of log data generated by the application program;
the processing module is used for carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, and training the alarm classification model based on log training samples marked with at least two alarm level labels;
And the generation module is used for generating abnormal alarm information of the log data when the alarm matching waiting time is triggered and the log data is matched with the log alarm matching strategy.
Further, the apparatus further comprises:
the determining module is used for determining at least one of the application type, the client type, the service type and the equipment type of the log data;
the distribution module is used for verifying the alarm level label according to at least one of the application type, the client type, the service type and the equipment type;
the building module is used for dividing preset time lengths according to the number of the alarm level labels and the alarm levels to obtain a plurality of alarm matching waiting time lengths, and building a matching relation between the alarm level labels and the alarm matching waiting time lengths so as to determine the alarm matching waiting time lengths corresponding to the alarm level labels based on the matching relation.
Further, the apparatus further comprises:
the starting module is used for starting waiting timing;
and the calling module is used for calling a log alarm matching strategy according to at least one of the application type, the client type, the service type and the equipment type when the timing duration of the waiting timing matches the alarm matching waiting duration, wherein the log alarm matching strategy comprises at least one of an application alarm matching sub-strategy, a client alarm matching sub-strategy, a service alarm matching sub-strategy and an equipment alarm matching sub-strategy.
Further, the apparatus further comprises:
the acquisition module is used for acquiring a log training sample and constructing a three-layer convolutional neural network, wherein the log training sample comprises log data marked with at least three alarm level labels;
and the training module is used for carrying out model training on the three-layer convolutional neural network according to the log training sample to obtain an alarm classification model, wherein the hierarchical weight of the three-layer convolutional neural network is configured based on the at least three alarm level labels.
Further, the apparatus further comprises:
the first configuration module is used for respectively configuring the first level weight, the second level weight and the third level weight into first weight values corresponding to the alarm level labels if the number of the alarm level labels is three, and the sum of the first weight values is smaller than 1;
and the second configuration module is used for respectively configuring the first level weight and the second level weight as second weight values corresponding to the alarm level labels and configuring the third level weight as third weight values corresponding to the alarm level labels if the number of the alarm level labels is four, wherein the second weight values are larger than the third weight values, and the sum of the second weight values and the third weight values is equal to 1.
Further, the apparatus further comprises:
the receiving module is used for receiving a confirmation message of the abnormal alarm information, wherein the confirmation message carries instruction content for updating or confirming the alarm level label of the log data;
and the updating module is used for updating the log training sample according to the updated or confirmed alarm level label and carrying out model training on the alarm classification model again according to a preset time interval based on the updated log training sample.
Further, the acquisition module is specifically configured to identify, when receiving a log data stream transmitted to the application program, an alarm key identifier in the log data stream, where the alarm key identifier includes at least one of an alarm keyword, an alarm key character string, and an alarm key symbol; and when the alarm key identification is identified, grouping the log data streams according to a preset data processing unit to obtain log data in a group unit, and performing classification marking processing of an alarm classification model based on each group of log data.
According to still another aspect of the present invention, there is provided a storage medium having stored therein at least one executable instruction for causing a processor to perform operations corresponding to the alerting method of the above-described exception log.
According to still another aspect of the present invention, there is provided a computer apparatus including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the alarm method of the abnormal log.
By means of the technical scheme, the technical scheme provided by the embodiment of the invention has at least the following advantages:
compared with the prior art, the embodiment of the invention obtains at least one group of log data generated by an application program; carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels; when the alarm matching waiting time is triggered and the log data is matched with the log alarm matching policy, abnormal alarm information of the log data is generated, so that abnormal alarms in massive logs of medical application systems are rapidly carried out, the speed of alarming the abnormal logs of the medical systems is greatly increased, the accuracy of log alarms is improved, and therefore accurate judgment of massive logs in medical application programs is met, and rapid abnormal alarms are achieved.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 shows a flowchart of an alarm method for an exception log according to an embodiment of the present invention;
FIG. 2 is a block diagram showing an alarm device for exception logs according to an embodiment of the present invention;
fig. 3 shows a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the invention can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Based on this, in an embodiment, as shown in fig. 1, an alarm method of an anomaly log is provided in the embodiment of the present invention, and the method is described by using a computer device such as a server as an example, where the server may be an independent server, or may be a cloud server that provides cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network (Content Delivery Network, CDN), and basic cloud computing services such as a big data and an artificial intelligent platform, such as an intelligent medical system, a digital medical platform, and the like. The method comprises the following steps:
101. At least one set of log data generated by the application is obtained.
In the embodiment of the invention, the current execution end is a service end for executing different services by the application program so as to receive log data of the application program when executing the services, and the application program comprises but is not limited to various types of medical application program apps and the like, such as a medical registration system, a medical project operation application system and the like, so as to acquire the generated log data. Wherein the log data is data generated by the application program when executing different medical services, including but not limited to user operation behavior, service execution results, system operation data, etc., to execute the alarm method of the anomaly log of steps 102-103 in the embodiment of the present invention.
In order to improve the judging efficiency of the abnormal log, the current execution end acquires log data in a group mode, and at this time, the log data in a group unit can be grouped according to time, service type, data type and the like so as to accelerate the alarm speed of the log data.
102. And carrying out classification marking processing on the log data based on the alarm classification model trained by the completed model to obtain an alarm level label, and determining alarm matching waiting time corresponding to the alarm level label.
In the embodiment of the invention, in order to accelerate the judgment of the abnormal log and improve the alarm efficiency, the current execution end carries out classification marking on the log data based on the alarm classification model which is trained by the completed model so as to obtain the alarm level label corresponding to the log data, and at the moment, the alarm classification model is obtained by training based on the log training samples of at least two marked alarm level labels. The alarm classification model is constructed for a machine learning model, and comprises but is not limited to a different-level convolutional neural network model, and is trained through log training samples marked with different alarm level labels. For example, the log training sample includes user behavior in the user behavior log 1, which includes user behavior in a blacklist, and is marked as a first-level alarm level label, so that model training is performed according to the user behavior log 1 as a training sample, which is not particularly limited in the embodiment of the present invention.
In order to reduce a large amount of log data and simultaneously alarm, reduce the processing pressure of the system, the current execution end configures alarm waiting time for different alarm level labels in advance, the alarm matching waiting time is preferably 1s, 3s and 5s, the alarm level labels comprise a first stage, a second stage and a third stage, the alarm level labels can be configured into the first stage to be 1s, the second stage to be 3s and the third stage to be 5s in advance, and the embodiment of the invention is not particularly limited.
103. And when the alarm matching waiting time is triggered and the log data is matched with a log alarm matching strategy, generating abnormal alarm information of the log data.
In the embodiment of the invention, after the current execution end determines the alarm matching waiting time length, timing is performed according to the alarm matching waiting time length, and after the timing time matches the alarm matching waiting time length, log data is matched with a log alarm matching strategy, so that whether the log data is abnormal is determined, and log alarm information is generated. The method for configuring log data of different medical application programs in advance by using the log alarm matching policy includes, but is not limited to, if the log data includes user behavior, the corresponding log alarm matching policy is a WeChat alarm, if the log data includes an ip address, the corresponding log alarm matching policy is a system error dialog box alarm, and if the log data includes a service result, the corresponding log alarm matching policy is a mail alarm. In addition, the abnormal alarm information is used for containing specific alarm content, for example, generating a WeChat message as log data is a first-level alarm, etc., and the embodiment of the invention is not limited in detail.
In another embodiment of the present invention, for further explanation and limitation, before determining the alert matching waiting duration corresponding to the alert level tag, the method further includes:
determining at least one of an application type, a client type, a service type and a device type of the log data;
verifying the alarm level label according to at least one of the application type, the client type, the service type and the equipment type;
dividing preset time lengths according to the number of the alarm level labels and the alarm levels to obtain a plurality of alarm matching waiting time lengths, and establishing a matching relation between the alarm level labels and the alarm matching waiting time lengths so as to determine the alarm matching waiting time lengths corresponding to the alarm level labels based on the matching relation.
In order to avoid the problem that the abnormal logs to be alerted centralize the alerts, which causes the system to process the blocking, before determining the alert matching waiting time length, the alert matching waiting time length needs to be configured for different alert level tags in advance. Specifically, an application type, a client type, a service type and a device type of log data are determined, the application type is an application program type for generating log data, such as a terminal app, a terminal webpage and the like, the client type is a user classification for generating behavior data in the log data, such as a new client, an old client, a male client, a female client and the like, the service type is a service executed by generating log data, such as a reservation service, an operation service, a diagnosis service, a medicine opening service and the like in a medical application program, the device type is a type of terminal device for generating log data, such as a mobile phone type of different brands and different versions, wherein the application type, the client type, the service type and the device type of the log data can be identified through specific identification, character strings and the like in the log data, and the embodiment of the invention is not limited specifically.
In addition, the current execution end may verify the alarm level tag according to at least one of an application type, a client type, a service type and a device type according to a pre-configured verification requirement, where the verification requirement includes a pre-configured alarm level tag that has to correspond to a specific type, for example, the log data is classified and marked by an alarm classification model trained by a model to obtain an alarm level tag as a secondary alarm tag, and when the pre-configured verification requirement in the current execution end includes a service type as an insurance type, the pre-configured verification requirement is a secondary alarm tag, so that the embodiment of the invention is not specifically limited by verification.
When the alarm matching waiting time is configured, firstly, dividing preset time according to the number of alarm level labels and alarm level grades, wherein the preset time can be 1 minute, 30 seconds and the like, for example, if the alarm level labels are 3, the alarm level grades are respectively one-level, two-level and three-level decrementing, the alarm matching waiting time is divided into 0 second for 30 seconds, the alarm matching waiting time is corresponding to the one-level alarm level label, the alarm matching waiting time is 13 seconds, the alarm matching waiting time is corresponding to the two-level alarm level label, the alarm matching waiting time is 30 seconds, and the alarm level label is corresponding to the three-level alarm. After a plurality of alarm matching waiting time periods are obtained through dividing, a matching relation between the alarm level label and the alarm matching waiting time periods is established, so that the alarm matching waiting time periods corresponding to the alarm level label are determined based on the matching relation, and the embodiment of the invention is not particularly limited.
In another embodiment of the present invention, for further explanation and limitation, before the step of generating the abnormal alarm information of the log data, the method further includes:
starting waiting time;
and when the timing duration of waiting to be timed is matched with the alarm matching waiting duration, invoking a log alarm matching strategy according to at least one of the application type, the client type, the service type and the equipment type.
In order to accurately carry out effective and accurate alarm on each log data according to the alarm matching waiting time length, after the alarm matching waiting time length is determined, a current execution end starts waiting timing so as to determine when to call the log alarm matching strategy. The method comprises the steps that waiting time can be started through a timer, and when the time duration recorded by the timer matches each alarm matching waiting time duration, a log alarm matching strategy is called according to at least one of an application type, a client type, a service type and a device type in log data. Specifically, the log alarm matching policy includes at least one of an application alarm matching sub-policy, a client alarm matching sub-policy, a service alarm matching sub-policy, and a device alarm matching sub-policy, where the application alarm matching sub-policy is an alarm policy preconfigured by application type log data, the client type is an alarm policy preconfigured by client type log data, the service type is an alarm policy preconfigured by service type log data, and the device type is an alarm policy preconfigured by device type log data.
In another embodiment of the present invention, for further explanation and limitation, the step of classifying and marking the log data based on the alarm classification model that has completed model training, and before obtaining the alarm level label, the method further includes:
acquiring a log training sample, and constructing a three-layer convolutional neural network;
according to the log training sample pair the three-layer convolutional neural network performs model training, and obtaining an alarm classification model.
In order to effectively judge abnormal conditions of log data based on a machine learning model, a log training sample is firstly obtained and a three-layer convolutional neural network is constructed to perform model training before the log data is classified and marked based on an alarm classification model. The log training sample includes log data marked with at least three alarm level labels, different alarm level labels are used for representing different emergency states of the log data, for example, four alarm level labels, namely, a first alarm level label, a second alarm level label, a third alarm level label and a fourth alarm level label, the first alarm level label corresponds to an alarm level of the most emergency state, the second alarm level label corresponds to an alarm requirement of the next emergency state, the third alarm level label corresponds to an alarm requirement of a non-emergency state, and the fourth alarm level label corresponds to a requirement of no alarm. In addition, when the three-layer convolutional neural network is constructed, the hierarchical weight of the three-layer convolutional neural network is configured based on at least three alarm level labels, so that when the three-layer convolutional neural network is subjected to model training according to the log training samples, the judgment accuracy of log data abnormality is improved, and an alarm classification model is obtained.
In another embodiment of the present invention, for further explanation and limitation, the steps further include:
if the number of the alarm level labels is three, respectively configuring a first level weight, a second level weight and a third level weight into first weight values corresponding to the alarm level labels, wherein the sum of the first weight values is smaller than 1;
if the number of the alarm level labels is four, respectively configuring a first level weight and a second level weight as a second weight value corresponding to the alarm level label, and configuring a third level weight as a third weight value corresponding to the alarm level label, wherein the second weight value is larger than the third weight value, and the sum of the second weight value and the third weight value is equal to 1.
In order to improve accuracy of model training and determine abnormality of log data, specifically, in an application scenario of the embodiment of the present invention, if the number of alert level labels of a medical application program is three, a first level weight, a second level weight, and a third level weight are respectively configured to be first weight values corresponding to the alert level labels, where a sum of the first weight values is less than 1, for example, the alert level labels include a first level alert label, a second level alert label, and a third level alert label, and at this time, the first weight value is preferably one third. In another application scenario of the embodiment of the present invention, if the number of the alarm level labels is four, the first level weight and the second level weight are respectively configured as a second weight value corresponding to the alarm level label, and the third level weight is configured as a third weight value corresponding to the alarm level label, where the alarm level label includes a first level alarm label, a second level alarm label, a third level alarm label and a fourth level alarm label, the first level weight is a weight of an input connection level of the three-layer convolutional neural network, and the third level weight is an output connection level weight of the three-layer convolutional neural network. Therefore, in configuring the weight values, specifically, the first level weight and the second level weight are configured as the second weight value corresponding to the first level alarm tag and the second level alarm tag, preferably 0.4, and the third level weight is configured as the third weight value corresponding to the third alarm level tag and the fourth alarm level tag, preferably 0.3.
It should be noted that, the embodiments of the present invention in which the first weight value, the second weight value, and the third weight value are configured or set in advance in the current execution end are not limited in particular.
In another embodiment of the present invention, for further explanation and limitation, after the step of generating the abnormal alarm information of the log data, the method further includes:
receiving a confirmation message of the abnormal alarm information;
and updating the log training sample according to the updated or confirmed alarm level label, and carrying out model training on the alarm classification model again according to a preset time interval based on the updated log training sample.
In order to improve the accuracy and effectiveness of the machine learning model in judging the abnormal log data, after generating the abnormal alarm information of the log data, the current execution end can receive the confirmation message of the abnormal alarm information from the user, so as to update the log training sample. When the user receives the abnormal alarm information, the user can manually confirm the abnormal alarm information and trigger the corresponding confirmation message so that the current execution end receives the confirmation message, for example, the abnormal alarm information is that the log data a of the WeChat message is a first-level alarm, the user can confirm the abnormal alarm information based on the log data a in a manual mode to generate a confirmation message, and at this time, the confirmation message carries instruction content for updating or confirming the alarm level label of the log data, for example, the first-level alarm of the log data a is correct, so that the alarm level label is updated to a log training sample, and model training is performed on the alarm classification model again based on a new log training sample after a preset time interval is reached.
In another embodiment of the present invention, for further explanation and limitation, the step of obtaining at least one set of log data generated by the application program includes;
when receiving a log data stream transmitted by the application program, identifying an alarm key identifier in the log data stream;
and when the alarm key identification is identified, grouping the log data streams according to a preset data processing unit to obtain log data in a group unit, and performing classification marking processing of an alarm classification model based on each group of log data.
In order to reduce the pressure of the system for data processing and improve the judgment effectiveness of abnormal log data, when the log data needing to be subjected to abnormal alarm identification is obtained, specifically, the current execution end receives the log data stream transmitted from the application program and identifies the alarm key identification in the log data stream in advance, so that one-time grouping is carried out. The alarm key identifier includes at least one of an alarm key word, an alarm key character string and an alarm key symbol, that is, the japanese data is grouped according to at least one of the alarm key word, the alarm key character string and the alarm key symbol identified in the log data stream, the alarm key word, the alarm key character string and the alarm key symbol are preconfigured identifiers that need to be grouped, for example, a plurality of log data including the alarm key word with "appointment register" is used as a group, or a plurality of log data including the alarm key character string with "β" is used as a group, or a plurality of log data including "β" is used as a group: "as one packet, thereby obtaining log data in units of a group, for the classification marking process, embodiments of the present invention are not particularly limited.
Compared with the prior art, the embodiment of the invention obtains at least one group of log data generated by an application program; carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels; when the alarm matching waiting time is triggered and the log data is matched with the log alarm matching policy, abnormal alarm information of the log data is generated, so that abnormal alarms in massive logs of medical application systems are rapidly carried out, the speed of alarming the abnormal logs of the medical systems is greatly increased, the accuracy of log alarms is improved, and therefore accurate judgment of massive logs in medical application programs is met, and rapid abnormal alarms are achieved.
Further, as an implementation of the method shown in fig. 1, an embodiment of the present invention provides an alarm device for an anomaly log, as shown in fig. 2, where the device includes:
An acquisition module 21 for acquiring at least one set of log data generated by the application program;
the processing module 22 is configured to perform classification marking processing on the log data based on an alarm classification model that has been trained on the model, obtain an alarm level tag, and determine an alarm matching waiting duration corresponding to the alarm level tag, where the alarm classification model is obtained by training based on log training samples that have been marked with at least two alarm level tags;
and the generating module 23 is configured to generate abnormal alarm information of the log data when the alarm matching waiting duration is triggered and the log data matches the log alarm matching policy.
Further, the apparatus further comprises:
the determining module is used for determining at least one of the application type, the client type, the service type and the equipment type of the log data;
the distribution module is used for verifying the alarm level label according to at least one of the application type, the client type, the service type and the equipment type;
the building module is used for dividing preset time lengths according to the number of the alarm level labels and the alarm levels to obtain a plurality of alarm matching waiting time lengths, and building a matching relation between the alarm level labels and the alarm matching waiting time lengths so as to determine the alarm matching waiting time lengths corresponding to the alarm level labels based on the matching relation.
Further, the apparatus further comprises:
the starting module is used for starting waiting timing;
and the calling module is used for calling a log alarm matching strategy according to at least one of the application type, the client type, the service type and the equipment type when the timing duration of the waiting timing matches the alarm matching waiting duration, wherein the log alarm matching strategy comprises at least one of an application alarm matching sub-strategy, a client alarm matching sub-strategy, a service alarm matching sub-strategy and an equipment alarm matching sub-strategy.
Further, the apparatus further comprises:
the acquisition module is used for acquiring a log training sample and constructing a three-layer convolutional neural network, wherein the log training sample comprises log data marked with at least three alarm level labels;
and the training module is used for carrying out model training on the three-layer convolutional neural network according to the log training sample to obtain an alarm classification model, wherein the hierarchical weight of the three-layer convolutional neural network is configured based on the at least three alarm level labels.
Further, the apparatus further comprises:
the first configuration module is used for respectively configuring the first level weight, the second level weight and the third level weight into first weight values corresponding to the alarm level labels if the number of the alarm level labels is three, and the sum of the first weight values is smaller than 1;
And the second configuration module is used for respectively configuring the first level weight and the second level weight as second weight values corresponding to the alarm level labels and configuring the third level weight as third weight values corresponding to the alarm level labels if the number of the alarm level labels is four, wherein the second weight values are larger than the third weight values, and the sum of the second weight values and the third weight values is equal to 1.
Further, the apparatus further comprises:
the receiving module is used for receiving a confirmation message of the abnormal alarm information, wherein the confirmation message carries instruction content for updating or confirming the alarm level label of the log data;
and the updating module is used for updating the log training sample according to the updated or confirmed alarm level label and carrying out model training on the alarm classification model again according to a preset time interval based on the updated log training sample.
Further, the acquisition module is specifically configured to identify, when receiving a log data stream transmitted to the application program, an alarm key identifier in the log data stream, where the alarm key identifier includes at least one of an alarm keyword, an alarm key character string, and an alarm key symbol; and when the alarm key identification is identified, grouping the log data streams according to a preset data processing unit to obtain log data in a group unit, and performing classification marking processing of an alarm classification model based on each group of log data.
Compared with the prior art, the embodiment of the invention obtains at least one group of log data generated by an application program; carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels; when the alarm matching waiting time is triggered and the log data is matched with the log alarm matching policy, abnormal alarm information of the log data is generated, so that abnormal alarms in massive logs of medical application systems are rapidly carried out, the speed of alarming the abnormal logs of the medical systems is greatly increased, the accuracy of log alarms is improved, and therefore accurate judgment of massive logs in medical application programs is met, and rapid abnormal alarms are achieved.
According to an embodiment of the present invention, there is provided a storage medium storing at least one executable instruction that can perform the method for alerting an abnormality log in any of the above-described method embodiments.
Fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention, and the specific embodiment of the present invention is not limited to the specific implementation of the computer device.
As shown in fig. 3, the computer device may include: a processor (processor) 302, a communication interface (Communications Interface) 304, a memory (memory) 306, and a communication bus 308.
Wherein: processor 302, communication interface 304, and memory 306 perform communication with each other via communication bus 308.
A communication interface 304 for communicating with network elements of other devices, such as clients or other servers.
The processor 302 is configured to execute the program 310, and may specifically perform relevant steps in the foregoing alert method embodiment of the exception log.
In particular, program 310 may include program code including computer-operating instructions.
The processor 302 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the computer device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
Memory 306 for storing programs 310. Memory 306 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Program 310 may be specifically operable to cause processor 302 to:
acquiring at least one group of log data generated by an application program;
carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels;
and when the alarm matching waiting time is triggered and the log data is matched with a log alarm matching strategy, generating abnormal alarm information of the log data.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An alarm method of an anomaly log, comprising:
acquiring at least one group of log data generated by an application program;
carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, wherein the alarm classification model is obtained by training based on log training samples marked with at least two alarm level labels;
and when the alarm matching waiting time is triggered and the log data is matched with a log alarm matching strategy, generating abnormal alarm information of the log data.
2. The method of claim 1, wherein prior to determining the alert matching wait period corresponding to the alert level tag, the method further comprises:
Determining at least one of an application type, a client type, a service type and a device type of the log data;
verifying the alarm level label according to at least one of the application type, the client type, the service type and the equipment type;
dividing preset time lengths according to the number of the alarm level labels and the alarm levels to obtain a plurality of alarm matching waiting time lengths, and establishing a matching relation between the alarm level labels and the alarm matching waiting time lengths so as to determine the alarm matching waiting time lengths corresponding to the alarm level labels based on the matching relation.
3. The method of claim 2, wherein prior to generating the anomaly alert information for the log data, the method further comprises:
starting waiting time;
and when the timing duration of waiting to be timed matches the alarm matching waiting duration, invoking a log alarm matching policy according to at least one of the application type, the client type, the service type and the equipment type, wherein the log alarm matching policy comprises at least one of an application alarm matching sub-policy, a client alarm matching sub-policy, a service alarm matching sub-policy and an equipment alarm matching sub-policy.
4. The method of claim 1, wherein before performing the classification marking process on the log data based on the alarm classification model with the completed model training to obtain the alarm level label, the method further comprises:
obtaining a log training sample, and constructing a three-layer convolutional neural network, wherein the log training sample comprises log data marked with at least three alarm level labels;
and carrying out model training on the three-layer convolutional neural network according to the log training sample to obtain an alarm classification model, wherein the hierarchical weight of the three-layer convolutional neural network is configured based on the at least three alarm level labels.
5. The method according to claim 4, wherein the method further comprises:
if the number of the alarm level labels is three, respectively configuring a first level weight, a second level weight and a third level weight into first weight values corresponding to the alarm level labels, wherein the sum of the first weight values is smaller than 1;
if the number of the alarm level labels is four, respectively configuring a first level weight and a second level weight as a second weight value corresponding to the alarm level label, and configuring a third level weight as a third weight value corresponding to the alarm level label, wherein the second weight value is larger than the third weight value, and the sum of the second weight value and the third weight value is equal to 1.
6. The method of claim 4, wherein after generating the anomaly alert information for the log data, the method further comprises:
receiving a confirmation message of the abnormal alarm information, wherein the confirmation message carries instruction content for updating or confirming the alarm level label of the log data;
and updating the log training sample according to the updated or confirmed alarm level label, and carrying out model training on the alarm classification model again according to a preset time interval based on the updated log training sample.
7. The method of any of claims 1-6, wherein the acquiring at least one set of log data generated by an application comprises;
when receiving a log data stream transmitted by the application program, identifying an alarm key identifier in the log data stream, wherein the alarm key identifier comprises at least one of an alarm key word, an alarm key character string and an alarm key symbol;
and when the alarm key identification is identified, grouping the log data streams according to a preset data processing unit to obtain log data in a group unit, and performing classification marking processing of an alarm classification model based on each group of log data.
8. An alarm device for an anomaly log, comprising:
the acquisition module is used for acquiring at least one group of log data generated by the application program;
the processing module is used for carrying out classification marking processing on the log data based on an alarm classification model which is trained by the model, obtaining alarm level labels, determining alarm matching waiting time length corresponding to the alarm level labels, and training the alarm classification model based on log training samples marked with at least two alarm level labels;
and the generation module is used for generating abnormal alarm information of the log data when the alarm matching waiting time is triggered and the log data is matched with the log alarm matching strategy.
9. A storage medium having stored therein at least one executable instruction for causing a processor to perform operations corresponding to the method of alerting an exception log according to any one of claims 1-7.
10. A computer device, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
The memory is configured to store at least one executable instruction, where the executable instruction causes the processor to perform an operation corresponding to the method for alerting an exception log according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310423282.5A CN116450471A (en) | 2023-04-18 | 2023-04-18 | Alarm method and device for abnormal log, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310423282.5A CN116450471A (en) | 2023-04-18 | 2023-04-18 | Alarm method and device for abnormal log, storage medium and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116450471A true CN116450471A (en) | 2023-07-18 |
Family
ID=87133311
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310423282.5A Pending CN116450471A (en) | 2023-04-18 | 2023-04-18 | Alarm method and device for abnormal log, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116450471A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312098A (en) * | 2023-11-22 | 2023-12-29 | 中国电子信息产业集团有限公司第六研究所 | Log abnormity alarm method and device |
-
2023
- 2023-04-18 CN CN202310423282.5A patent/CN116450471A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312098A (en) * | 2023-11-22 | 2023-12-29 | 中国电子信息产业集团有限公司第六研究所 | Log abnormity alarm method and device |
| CN117312098B (en) * | 2023-11-22 | 2024-03-01 | 中国电子信息产业集团有限公司第六研究所 | Log abnormity alarm method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113626241B (en) | Abnormality processing method, device, equipment and storage medium for application program | |
| CN111639101B (en) | Method, device and system for correlating rule engine system of internet of things and storage medium | |
| CN112491643A (en) | Deep packet inspection method, device, equipment and storage medium | |
| CN108388802A (en) | A kind of alarm method and warning system of script injection attacks | |
| CN116450471A (en) | Alarm method and device for abnormal log, storage medium and computer equipment | |
| US20210081879A1 (en) | Systems and methods for escalation policy activation | |
| US20060248147A1 (en) | System and method for automatically sending messages to service personnel | |
| CN116484867A (en) | Named entity recognition method and device, storage medium and computer equipment | |
| CN111198804A (en) | Gateway-based industrial internet platform third-party micro-service monitoring and early warning method | |
| CN114185743A (en) | Data processing method and device, computer equipment and storage medium | |
| CN113343248A (en) | Vulnerability identification method, device, equipment and storage medium | |
| CN111917848A (en) | Data processing method based on edge computing and cloud computing cooperation and cloud server | |
| CN115147031B (en) | Clearing workflow execution method, device, equipment and medium | |
| CN113946464B (en) | Alarm noise reduction method combining model and experience pre-training and parallel deduction | |
| CN114756850A (en) | Data acquisition method, device, equipment and storage medium | |
| CN114220550A (en) | Single disease reporting general method and device, electronic equipment and storage medium | |
| US20200134528A1 (en) | Systems and methods for coordinating escalation policy activation | |
| CN115544202A (en) | Alarm processing method, device and storage medium | |
| CN112948341A (en) | Method and apparatus for identifying abnormal network device logs | |
| CN113505145A (en) | Customized SQL engine linkage analysis method based on Internet of things and related device | |
| CN112367326B (en) | Method and device for identifying traffic of Internet of vehicles | |
| CN114760188B (en) | Abnormal node determination method and device | |
| CN117171800B (en) | Sensitive data identification method and device based on zero trust protection system | |
| CN114598588B (en) | Server fault determination method and device and terminal equipment | |
| CN114743553A (en) | Man-machine conversation method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |