CN110691070B - Network abnormity early warning method based on log analysis - Google Patents
Network abnormity early warning method based on log analysis Download PDFInfo
- Publication number
- CN110691070B CN110691070B CN201910844932.7A CN201910844932A CN110691070B CN 110691070 B CN110691070 B CN 110691070B CN 201910844932 A CN201910844932 A CN 201910844932A CN 110691070 B CN110691070 B CN 110691070B
- Authority
- CN
- China
- Prior art keywords
- log data
- log
- network
- preset
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/1458—Denial of Service
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Debugging And Monitoring (AREA)
Abstract
A network abnormity early warning method based on log analysis belongs to the technical field of computer networks and solves the problem that the existing network abnormity early warning method based on log data source IP statistical analysis is small in early warning range. The method comprises the following steps: collecting log data of a target network in a preset time period; filtering the collected log data according to a log data filtering rule to obtain log data to be analyzed; converting the format of the log data to be analyzed into a preset log data format according to a log data analysis rule; dividing the log data with unified format into a plurality of groups of training data according to the log data correlation evaluation rule; training a corresponding preset neural network model by adopting each group of training data to obtain a network anomaly prediction model corresponding to the group of training data; and performing abnormity prediction on the target network based on the obtained multiple network abnormity prediction models, and performing early warning on the target alarm object when the level of the predicted log reaches the alarm level.
Description
Technical Field
The invention relates to a network abnormity early warning method, and belongs to the technical field of computer networks.
Background
With the rapid development of information-based construction and the rapid integration of Information and Communication Technology (ICT) of telecommunication operators, the traffic demand is rapidly increased, and a large amount of software and hardware resources are accumulated in the operator network, including: switches, routers, firewalls, servers, and the like. The load borne by the network equipment is getting larger and larger, and the executed service is getting more and more complicated. These devices continuously log a large number of logs. The log file is used as a recording tool of hardware equipment, a system and user behaviors, and plays a significant role in monitoring network operation conditions, investigating equipment faults, protecting system safety and the like. By analyzing the log file, information about equipment faults, user abnormal behaviors, network operation conditions and the like can be acquired, so that network security events and software and hardware faults can be treated in time, and the stability and the safety of the network are ensured.
In the existing computer network, log sources are numerous, formats are different, the volume is huge, and the data volume stored for a long time can reach TB level or PB level. The traditional log analysis system processes mass data by using a single machine technology and meets bottlenecks in both storage and calculation.
Based on the above background, the chinese patent application with application publication No. CN 110098957a discloses a big data analysis system based on weblog. The big data analysis system comprises an acquisition layer, a storage layer, a service layer and a display layer. The acquisition layer is used for acquiring the log record file from the target network and sending the acquired log record file to the storage layer; the storage layer is used for receiving the log record file sent by the acquisition layer and performing distributed storage; the service layer is used for acquiring the log record file from the storage layer, performing statistical analysis on the log record file and sending the statistical analysis result to the display layer; and the display layer is used for receiving and displaying the statistical analysis result sent by the service layer. The big data analysis system performs distributed storage on the obtained log record files, and performs statistical analysis on the stored log record files based on a big data analysis technology.
However, although the big data analysis system performs statistical analysis on the stored log record file based on the big data analysis technology, and further implements the anomaly prediction on the target network, the anomaly prediction means of the big data analysis system on the target network only performs statistical analysis on the source IP in the log data under the big data view angle, and the statistical analysis on the IP request times can only predict Dos attacks and DDos attacks, and cannot predict other types of network anomalies, such as hardware device problems, software bugs, system bugs, and the like. Therefore, the early warning range of the network abnormity early warning method based on the log data source IP statistical analysis is smaller.
Disclosure of Invention
The invention provides a network abnormity early warning method based on log analysis, aiming at solving the problem that the early warning range of the existing network abnormity early warning method based on log data source IP statistical analysis is small.
The invention relates to a network abnormity early warning method based on log analysis, which comprises the following steps:
collecting log data in a preset time period of a target network;
according to a preset log data filtering rule, filtering the collected log data to obtain the log data to be analyzed;
uniformly converting the format of the log data to be analyzed into a preset log data format according to a preset log data analysis rule;
dividing the log data with unified format into a plurality of groups of training data according to a preset log data correlation evaluation rule;
training a corresponding preset neural network model by adopting each group of training data to obtain a network anomaly prediction model corresponding to the group of training data;
and performing abnormity prediction on the target network based on the obtained multiple network abnormity prediction models, and performing early warning on the target alarm object when the level of the predicted log reaches a preset alarm level.
Preferably, in the network anomaly early warning method, after collecting log data in a predetermined period of time of a target network, the method further includes:
and storing the collected log data in the target network within a preset time period.
Preferably, in the network anomaly early warning method, dividing the log data with unified format into a plurality of groups of training data includes:
dividing the log data with unified format into a plurality of log data sets according to the source of the log data to obtain a log data set sequence;
taking a log data set sequence as an object, and extracting a log data sequence in a time window with a preset length;
dividing the extracted log data sequence to obtain a log data group of each log data source;
calculating a weight matrix of a log data group of each log data source;
selecting a preset number of previous key participles from the weight matrix of the log data group of each log data source, and summarizing the previous key participles into a participle dictionary;
determining the characteristic vector of the log data group of each log data source according to the word frequency of the log data group of each log data source to the participles in the participle dictionary;
calculating the cosine similarity of the log data groups of any two log data sources according to the characteristic vectors of the log data groups of the two log data sources,
when the cosine similarity is larger than a preset log data correlation threshold value, the log data groups of the two log data sources are classified into the same training data group,
and when the cosine similarity is less than or equal to a preset log data correlation threshold, grouping the log data groups of the two log data sources into two different groups of training data.
Preferably, in the network anomaly early warning method, after the log data sequence within a time window of a predetermined length is extracted, the method further includes:
encoding a level of each log data within the extracted sequence of log data.
Preferably, the network anomaly early warning method adopts a Tri-Gram language model to segment the extracted log data sequence.
Preferably, in the network anomaly early warning method, after the extracted log data sequence is segmented, the method further includes:
and marking the head and the tail of each log data obtained by segmentation.
Preferably, in the network anomaly early warning method, the weight matrix is a TF-IDF matrix.
Preferably, the network anomaly early warning method selects the top 10 key participles from the TF-IDF matrix of the log data group of each log data source, and summarizes the key participles into a participle dictionary.
Preferably, in the network anomaly early warning method, the neural network model is a Seq2Seq model introducing a self-attention mechanism.
Preferably, in the network anomaly early warning method, the predicting anomalies of the target network based on the obtained multiple network anomaly prediction models includes:
collecting newly generated log data of a target network in real time;
according to a preset log data filtering rule, filtering the collected log data to obtain the log data to be analyzed;
converting the format of the log data to be analyzed into a preset log data format according to a preset log data analysis rule;
determining a prior training data group which is strongly related to the analyzed log data according to a predetermined log data correlation evaluation rule;
and converting the analyzed log data into a sequence form, and sequentially inputting the log data into the network anomaly prediction model corresponding to the prior training data set according to the time sequence to obtain predicted log data.
The invention relates to a network anomaly early warning method based on log analysis, which comprises the steps of collecting log data in a preset time period of a target network, filtering the collected log data according to a preset log data filtering rule to obtain the log data to be analyzed, uniformly converting the format of the log data to be analyzed into a preset log data format according to a preset log data analyzing rule, dividing the log data with uniform format into a plurality of groups of training data according to a preset log data correlation judging rule, training a corresponding preset neural network model by adopting each group of training data to obtain a network anomaly prediction model corresponding to the group of training data, performing anomaly prediction on the target network based on a plurality of obtained network anomaly prediction models, and when the level of the predicted log reaches a preset warning level, and early warning is carried out on the target warning object. Compared with the conventional network anomaly early warning method based on log data source IP statistical analysis, the network anomaly early warning method based on log analysis can predict not only Dos attacks and DDos attacks, but also other types of network anomalies, such as hardware equipment self problems, software bugs, system bugs and the like. Therefore, the network abnormity early warning method based on log analysis has more comprehensive early warning types and larger early warning range.
Drawings
The network anomaly early warning method based on log analysis according to the present invention will be described in more detail below based on embodiments and with reference to the accompanying drawings, in which:
FIG. 1 is a flowchart illustrating an implementation of a network anomaly early warning method based on log analysis according to an embodiment;
FIG. 2 is a flowchart illustrating an implementation of dividing the unified format log data into a plurality of sets of training data according to an embodiment;
FIG. 3 is a flowchart illustrating an implementation of an anomaly prediction for a target network based on a plurality of obtained network anomaly prediction models according to an embodiment;
fig. 4 is a structural block diagram of a Seq2Seq model incorporating a self-attention mechanism according to the embodiment.
Detailed Description
The network anomaly early warning method based on log analysis according to the present invention will be further described with reference to the accompanying drawings.
Example (b): the present embodiment is described in detail below with reference to fig. 1 to 4.
The network abnormity early warning method based on log analysis comprises the following steps:
step S1, collecting log data in a preset time period of the target network;
in this embodiment, the target network is a college network, and the collected log data is from a server, a firewall, a router, an IDS/IPS, an operating system, an application program, and the like in the college network.
In this embodiment, log data is acquired by using an existing log acquisition tool, such as logstack, filebed, fluent, Logagent, logmail, and the like.
In this embodiment, log data of a network of a college is collected within one month.
Step S2, according to the preset log data filtering rule, filtering the collected log data to obtain the log data to be analyzed;
although the college weblog data are sourced and abundant, not all log data have analytical value. Since the computing power of the related hardware is limited and in order to avoid too much unrelated log data from affecting the predictive analysis, step S2 filters the collected log data according to a predetermined log data filtering rule, and filters out a part of the log data without analysis value. The log data filtering rules can be adjusted according to actual conditions.
Step S3, uniformly converting the format of the log data to be analyzed into a predetermined log data format according to a predetermined log data analysis rule;
the diversity of the log data sources determines that the obtained log data are multi-source heterogeneous data. The transmission protocol of the log message comprises syslog, snmp, http, ftp and the like, in order to avoid disordered log data from interfering with further analysis processing, a corresponding log data analysis rule is formulated by combining the known forms of all the obtained log data and the meaning of each field, and all the obtained log data are converted into a fixed format according to the log data analysis rule.
The predetermined log data format of this embodiment is [ priority ] [ IP/source ] [ YYYYYY-MM-DD ] [ hh: MM: ss ] [ URI ] [ message ]. The log data analysis rule can be adjusted according to actual conditions.
Step S4, dividing the log data with unified format into a plurality of groups of training data according to a preset log data correlation evaluation rule;
in this embodiment, the correlation between log data from different sources is calculated according to a predetermined log data correlation evaluation rule. The strong correlation among the log data of different sources shows that the mutual influence among corresponding devices or systems is large, and the log data are trained together to obtain a network anomaly prediction model. The weak correlation among the log data of different sources indicates that the corresponding equipment or system operates relatively independently, and the log data of each source is independently used as training data to obtain a network anomaly prediction model.
Step S5, training a corresponding preset neural network model by adopting each group of training data to obtain a network anomaly prediction model corresponding to the group of training data;
and step S6, performing abnormity prediction on the target network based on the obtained multiple network abnormity prediction models, and performing early warning on the target alarm object when the level of the predicted log reaches a preset alarm level.
In the present embodiment, step S4 includes:
step S401, dividing the log data with unified format into a plurality of log data sets according to the source of the log data to obtain a log data set sequence;
step S402, taking the log data set sequence as an object, and extracting the log data sequence in a time window with a preset length;
step S403, the extracted log data sequence is segmented to obtain a log data group of each log data source;
step S404, calculating a weight matrix of a log data group of each log data source;
s405, selecting a preset number of previous key participles from the weight matrix of the log data group of each log data source, and summarizing the previous key participles into a participle dictionary;
step S406, determining a feature vector of a log data group of each log data source according to the word frequency of the log data group of each log data source to the participles in the participle dictionary;
step S407, calculating cosine similarity of log data groups of any two log data sources according to the feature vectors of the log data groups of the two log data sources;
step S408, judging whether the cosine similarity is larger than a preset log data correlation threshold, if so, executing step S409, otherwise, executing step S410;
step S409, grouping the log data groups of the two log data sources into the same group of training data;
step S410, grouping the log data sets of the two log data sources into two different sets of training data.
The process of step S4 is described below with a specific example:
the n sources of log data sets form a log data set sequence L1,L2,…,Ln}, each log data set LiContains m subsets of log data li,1,li,2,…,li,m}。
The length of the time window is set to be T, and only the correlation among the log data in the time T is considered.
The level of the log data in T time, such as INFO, WARNING, ERROR, is expressed by using one-hot code coding.
And (3) adopting a Tri-Gram model to segment log data sequences in a time window, and marking a label of < begin > and a label of < end > at the head and the tail of each piece of log data obtained by segmentation.
A TF-IDF matrix is calculated and generated for each source log data set within the time window, wherein,
TF-IDF＝TF×IDF。
selecting the first 10 key participles in each source log data group according to the TF-IDF matrix to form a dictionary, counting the word frequency of each log data group for the participles in the dictionary to obtain a feature vector
And calculating the correlation among log data groups from different sources by using cosine similarity:
in the formula, A and B represent log data groups from two different sources, i is a word segmentation serial number, and j is the total number of words in the dictionary.
And if the cosine similarity is greater than the threshold value gamma, judging that the log data groups of the two sources have correlation and classifying the log data groups as the same training data group.
The neural network model of the embodiment is a Seq2Seq model introducing an attention mechanism, and comprises a training data sequence generator, an encoder, an attention mechanism introduction link, a decoder and a regression classifier;
the training data sequence generator is used for inputting log data for training the network anomaly prediction model into the encoder in sequence mode and according to the time sequence.
The encoder and the decoder both adopt an LSTM model, and a self-attention mechanism introduction link is used for generating an attention weight matrix according to input training data so as to prevent the currently input training data from covering the previously input training data.
The hidden state of the encoder at the current moment is ht＝LSTM(ht-1,xt) Generating semantic vector c by weight conversion of hidden layer information of encodert＝f(h1,h2,…,hT) Decoder hidden state s at the current timet＝q(st-1,yt-1,ct) Decoded output yt＝g(yt-1,st,ct). Per output prediction Log sequence is put on<begin><end>And (4) a label.
In this embodiment, performing an anomaly prediction on a target network based on the obtained multiple network anomaly prediction models includes:
s61, collecting newly generated log data of the target network in real time;
s62, filtering the collected log data according to a preset log data filtering rule to obtain the log data to be analyzed;
s63, converting the format of the log data to be analyzed into a preset log data format according to a preset log data analysis rule;
s64, determining a prior training data group which is strongly related to the analyzed log data according to a preset log data correlation evaluation rule;
and S65, converting the analyzed log data into a sequence form, and sequentially inputting the log data into the network anomaly prediction model corresponding to the prior training data set according to the time sequence to obtain predicted log data.
The network anomaly early warning method based on log analysis is realized based on a corresponding network anomaly early warning platform, and the network anomaly early warning platform comprises a log data acquisition module, a log data filtering module, a log data analysis module, a log data correlation evaluation module, a network anomaly prediction module and a network anomaly early warning module. The network abnormity early warning module is used for sending a network abnormity early warning signal to a platform manager when the level of the log data predicted by the network abnormity prediction module reaches a preset warning level, and sending the predicted log data and the historical predicted log data in a preset time period to the platform manager so as to allow the platform manager to troubleshoot problems. And when the level of the log data predicted by the network abnormity prediction module does not reach the preset alarm level, the network abnormity early warning module ignores the predicted log data and does not act.
The network abnormity early warning platform also comprises a log data storage module and a log data query module. The log data storage module is used for temporarily storing the log data acquired by the log data acquisition module. And the log data query module is used for searching related log data in the log data range acquired by the log data acquisition module and displaying the searched log data when receiving a log data query command manually input by a platform manager. The log data query module supports keyword search and also supports attribute search according to time, priority, log level and the like. The log data searched by the log data query module is displayed in a fixed format after being analyzed and also displayed in an original format, so that comparison and analysis of platform management personnel are facilitated.
According to the network anomaly early warning method based on log analysis, a TF-IDF algorithm and a Seq2Seq neural network model are combined, a self-attention mechanism is introduced, strongly-related log data are trained together, relatively independent log data are trained independently, and accuracy and adaptability of network anomaly early warning are further improved. After training is finished, real-time log data can be received, a prediction result is output, instant response is made based on the prediction result, loss is reduced, and network anomaly prediction in the true sense is achieved.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.
Claims (8)
1. A network abnormity early warning method based on log analysis is characterized by comprising the following steps:
collecting log data in a preset time period of a target network;
according to a preset log data filtering rule, filtering the collected log data to obtain the log data to be analyzed;
uniformly converting the format of the log data to be analyzed into a preset log data format according to a preset log data analysis rule;
dividing the log data with unified format into a plurality of groups of training data according to a preset log data correlation evaluation rule;
training a corresponding preset neural network model by adopting each group of training data to obtain a network anomaly prediction model corresponding to the group of training data;
performing abnormity prediction on a target network based on the obtained multiple network abnormity prediction models, and performing early warning on a target alarm object when the level of a predicted log reaches a preset alarm level;
the neural network model is a Seq2Seq model introducing a self-attention mechanism;
the predetermined log data format is priority IP/source YYYYY-MM-DD hh MM ss URI message;
dividing the log data with unified format into a plurality of groups of training data comprises:
dividing the log data with unified format into a plurality of log data sets according to the source of the log data to obtain a log data set sequence;
taking a log data set sequence as an object, and extracting a log data sequence in a time window with a preset length;
dividing the extracted log data sequence to obtain a log data group of each log data source;
calculating a weight matrix of a log data group of each log data source;
selecting a preset number of previous key participles from the weight matrix of the log data group of each log data source, and summarizing the previous key participles into a participle dictionary;
determining the characteristic vector of the log data group of each log data source according to the word frequency of the log data group of each log data source to the participles in the participle dictionary;
calculating the cosine similarity of the log data groups of any two log data sources according to the characteristic vectors of the log data groups of the two log data sources,
when the cosine similarity is larger than a preset log data correlation threshold value, the log data groups of the two log data sources are classified into the same training data group,
and when the cosine similarity is less than or equal to a preset log data correlation threshold, grouping the log data groups of the two log data sources into two different groups of training data.
2. The network anomaly early warning method based on log analysis as claimed in claim 1, wherein after collecting log data in a predetermined period of time of a target network, further comprising:
and storing the collected log data in the target network within a preset time period.
3. The log analysis-based network anomaly early warning method as claimed in claim 1, further comprising, after extracting log data sequences within a time window of a predetermined length:
encoding a level of each log data within the extracted sequence of log data.
4. The network anomaly early warning method based on log analysis as claimed in claim 1, wherein the Tri-Gram language model is adopted to segment the extracted log data sequence.
5. The log analysis-based network anomaly early warning method as claimed in claim 1, wherein after the extracted log data sequence is segmented, the method further comprises:
and marking the head and the tail of each log data obtained by segmentation.
6. The log analysis-based network anomaly early warning method as claimed in claim 1, wherein the weight matrix is a TF-IDF matrix.
7. The log analysis-based network anomaly early warning method as claimed in claim 6, wherein the top 10 key participles are selected from the TF-IDF matrix of the log data group of each log data source and are collected into a participle dictionary.
8. The log analysis-based network anomaly early warning method as claimed in claim 1, wherein the performing anomaly prediction on the target network based on the obtained multiple network anomaly prediction models comprises:
collecting newly generated log data of a target network in real time;
according to a preset log data filtering rule, filtering the collected log data to obtain the log data to be analyzed;
converting the format of the log data to be analyzed into a preset log data format according to a preset log data analysis rule;
determining a prior training data group which is strongly related to the analyzed log data according to a predetermined log data correlation evaluation rule;
and converting the analyzed log data into a sequence form, and sequentially inputting the log data into the network anomaly prediction model corresponding to the prior training data set according to the time sequence to obtain predicted log data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910844932.7A CN110691070B (en) | 2019-09-07 | 2019-09-07 | Network abnormity early warning method based on log analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910844932.7A CN110691070B (en) | 2019-09-07 | 2019-09-07 | Network abnormity early warning method based on log analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110691070A CN110691070A (en) | 2020-01-14 |
| CN110691070B true CN110691070B (en) | 2022-02-11 |
Family
ID=69107986
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910844932.7A Active CN110691070B (en) | 2019-09-07 | 2019-09-07 | Network abnormity early warning method based on log analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110691070B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797978A (en) * | 2020-07-08 | 2020-10-20 | 北京天融信网络安全技术有限公司 | Internal threat detection method and device, electronic equipment and storage medium |
| CN111914015A (en) * | 2020-08-25 | 2020-11-10 | 河北时代电子有限公司 | Multisource data gateway data analysis early warning system based on industrial protocol |
| CN112069787A (en) * | 2020-08-27 | 2020-12-11 | 西安交通大学 | Log parameter anomaly detection method based on word embedding |
| CN112087448B (en) * | 2020-09-08 | 2023-04-14 | 南方电网科学研究院有限责任公司 | Security log extraction method and device and computer equipment |
| CN111985192A (en) * | 2020-09-28 | 2020-11-24 | 杭州安恒信息安全技术有限公司 | Web attack report generation method, device, equipment and computer medium |
| CN112434949A (en) * | 2020-11-25 | 2021-03-02 | 平安普惠企业管理有限公司 | Service early warning processing method, device, equipment and medium based on artificial intelligence |
| CN112988440B (en) * | 2021-02-23 | 2023-08-01 | 山东英信计算机技术有限公司 | System fault prediction method and device, electronic equipment and storage medium |
| CN113902318A (en) * | 2021-10-15 | 2022-01-07 | 侯荣芹 | Quality management system and quality management method |
| CN116192612B (en) * | 2023-04-23 | 2023-07-25 | 成都新西旺自动化科技有限公司 | System fault monitoring and early warning system and method based on log analysis |
| CN117033334B (en) * | 2023-10-08 | 2023-12-22 | 吉林省高速公路集团有限公司 | Expressway toll lane log acquisition processing method and system |
| CN118300896B (en) * | 2024-06-05 | 2024-08-13 | 温州数码创业投资有限公司 | Abnormal user behavior management method and system for cloud computing service environment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105653444A (en) * | 2015-12-23 | 2016-06-08 | 北京大学 | Internet log data-based software defect failure recognition method and system |
| CN107301118A (en) * | 2017-06-15 | 2017-10-27 | 中国科学院计算技术研究所 | A kind of fault indices automatic marking method and system based on daily record |
| CN108123840A (en) * | 2017-12-22 | 2018-06-05 | 中国联合网络通信集团有限公司 | Log processing method and system |
| WO2019060327A1 (en) * | 2017-09-20 | 2019-03-28 | University Of Utah Research Foundation | Online detection of anomalies within a log using machine learning |
| JP2019074927A (en) * | 2017-10-16 | 2019-05-16 | 株式会社ブリヂストン | Abnormal data detecting method and apparatus thereof from use history data on tire |
-
2019
- 2019-09-07 CN CN201910844932.7A patent/CN110691070B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105653444A (en) * | 2015-12-23 | 2016-06-08 | 北京大学 | Internet log data-based software defect failure recognition method and system |
| CN107301118A (en) * | 2017-06-15 | 2017-10-27 | 中国科学院计算技术研究所 | A kind of fault indices automatic marking method and system based on daily record |
| WO2019060327A1 (en) * | 2017-09-20 | 2019-03-28 | University Of Utah Research Foundation | Online detection of anomalies within a log using machine learning |
| JP2019074927A (en) * | 2017-10-16 | 2019-05-16 | 株式会社ブリヂストン | Abnormal data detecting method and apparatus thereof from use history data on tire |
| CN108123840A (en) * | 2017-12-22 | 2018-06-05 | 中国联合网络通信集团有限公司 | Log processing method and system |
Non-Patent Citations (2)
| Title |
|---|
| Abnormal Event Detection Using Recurrent Neural Network;Xu-Gang Zhou 等;《2015 International Conference on Computer Science and Applications (CSA)》;20151122;全文 * |
| 基于改进时间卷积网络的日志序列异常检测;杨瑞朋 等;《计算机工程》;20190905;第46卷(第8期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110691070A (en) | 2020-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110691070B (en) | Network abnormity early warning method based on log analysis | |
| CN112398779B (en) | Network traffic data analysis method and system | |
| CN107241352B (en) | Network security event classification and prediction method and system | |
| US10678669B2 (en) | Field content based pattern generation for heterogeneous logs | |
| Estevez-Tapiador et al. | Detection of web-based attacks through Markovian protocol parsing | |
| CN110909811A (en) | OCSVM (online charging management system) -based power grid abnormal behavior detection and analysis method and system | |
| CN111092852A (en) | Network security monitoring method, device, equipment and storage medium based on big data | |
| CN109766695A (en) | A kind of network security situational awareness method and system based on fusion decision | |
| CN112468347B (en) | Security management method and device for cloud platform, electronic equipment and storage medium | |
| CN111930592A (en) | Method and system for detecting log sequence abnormity in real time | |
| CN111600919A (en) | Web detection method and device based on artificial intelligence | |
| CN107104951B (en) | Method and device for detecting network attack source | |
| WO2022053163A1 (en) | Distributed trace anomaly detection with self-attention based deep learning | |
| CN110879771A (en) | Log analysis system for user anomaly detection based on keyword sequence mining | |
| CN109583567A (en) | A kind of Web autoscanner fingerprint recognition model based on CNN | |
| CN109088903A (en) | A kind of exception flow of network detection method based on streaming | |
| CN117220920A (en) | Firewall policy management method based on artificial intelligence | |
| CN108989411B (en) | Web user click target identification method based on network flow | |
| Kozik et al. | Pattern extraction algorithm for NetFlow‐based botnet activities detection | |
| CN116684877A (en) | GYAC-LSTM-based 5G network traffic anomaly detection method and system | |
| Skopik et al. | Online log data analysis with efficient machine learning: A review | |
| CN114780810A (en) | Data processing method, data processing device, storage medium and electronic equipment | |
| CN117914599A (en) | Mobile network malicious traffic identification method based on graph neural network | |
| CN110909380A (en) | Abnormal file access behavior monitoring method and device | |
| CN112073396A (en) | Method and device for detecting transverse movement attack behavior of intranet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |