US10237295B2 - Automated event ID field analysis on heterogeneous logs - Google Patents

Automated event ID field analysis on heterogeneous logs Download PDF

Info

Publication number
US10237295B2
US10237295B2 US15/429,849 US201715429849A US10237295B2 US 10237295 B2 US10237295 B2 US 10237295B2 US 201715429849 A US201715429849 A US 201715429849A US 10237295 B2 US10237295 B2 US 10237295B2
Authority
US
United States
Prior art keywords
event
log
fields
hash table
anomaly
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US15/429,849
Other languages
English (en)
Other versions
US20170279840A1 (en
Inventor
Hui Zhang
Guofei Jiang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to US15/429,849 priority Critical patent/US10237295B2/en
Assigned to NEC LABORATORIES AMERICA, INC. reassignment NEC LABORATORIES AMERICA, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: JIANG, GUOFEI, ZHANG, HUI
Priority to PCT/US2017/017869 priority patent/WO2017165018A1/fr
Publication of US20170279840A1 publication Critical patent/US20170279840A1/en
Assigned to NEC CORPORATION reassignment NEC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NEC LABORATORIES AMERICA, INC.
Application granted granted Critical
Publication of US10237295B2 publication Critical patent/US10237295B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection

Definitions

  • the present invention generally relates to event identifiers and more particularly to automatically detecting event identifier (ID) content from heterogeneous logs and utilizing event ID content for log sequence analysis.
  • ID event identifier
  • event IDs are unique and would not usually be frequent
  • mining frequent patterns with event IDs and showing records they occurred in provides an efficient way to mine frequent patterns in many types of databases including multiple tabled and distributed databases.
  • Some techniques propose a set of algorithms for mining frequent patterns with their event IDs in a single transaction database, in a multiple tabled database, and in a distributed database.
  • the event ID attributes in the data base are specified manually, therefore this technique does not apply to heterogeneous system logs that are unstructured and have no attribute labels.
  • a method performed in a network having network devices, including computers, that generate heterogeneous logs which include a plurality of event sequences includes identifying, by a processor from the heterogeneous logs, pattern fields comprised of a plurality of event identifiers.
  • the method further includes generating, by the processor, an automata model by profiling event behaviors of the plurality of event sequences, the plurality of event sequences grouped in the automata model by combinations of one or more pattern fields and one or more event identifiers from among the plurality of event identifiers, wherein for a given combination, the one or more event identifiers therein must be respectively comprised in a same one of the one or more pattern fields with which it is combined.
  • the method also includes detecting, by the processor, an anomaly in one of the plurality of event sequences using the automata model.
  • the method additionally includes controlling, by the processor, an anomaly-initiating one of the network devices based on the anomaly.
  • a computer program product for automata model formation for a network having a plurality of network devices that generate heterogeneous logs which include a plurality of event sequences
  • the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method.
  • the method includes identifying, by a processor from the heterogeneous logs, pattern fields comprised of a plurality of event identifiers.
  • the method further includes generating, by the processor, an automata model by profiling event behaviors of the plurality of event sequences, the plurality of event sequences grouped in the automata model by combinations of one or more pattern fields and one or more event identifiers from among the plurality of event identifiers, wherein for a given combination, the one or more event identifiers therein must be respectively comprised in a same one of the one or more pattern fields with which it is combined.
  • the method also includes detecting, by the processor, an anomaly in one of the plurality of event sequences using the automata model.
  • the method additionally includes controlling, by the processor, an anomaly-initiating one of the network devices based on the anomaly.
  • a computer processing system for anomaly detection for a network having a plurality of network devices that generate heterogeneous logs which include a plurality of event sequences.
  • the computer processing system includes a processor.
  • the processor is configured to identify, from the heterogeneous logs, pattern fields comprised of a plurality of event identifiers.
  • the processor is further configured to generate an automata model by profiling event behaviors of the plurality of event sequences, the plurality of event sequences grouped in the automata model by combinations of one or more pattern fields and one or more event identifiers from among the plurality of event identifiers, wherein for a given combination, the one or more event identifiers therein must be respectively comprised in a same one of the one or more pattern fields with which it is combined.
  • the processor is additionally configured to detect an anomaly in one of the plurality of event sequences using the automata model.
  • the processor is also configured to control an anomaly-initiating one of the network devices based on the anomaly.
  • FIG. 1 show an exemplary sequence of operational logs including event ID content 100 to which the present invention can be applied, in accordance with an embodiment of the present invention
  • FIG. 2 shows an exemplary system/method 200 for an automated event ID field analysis process, in accordance with an embodiment of the present invention
  • FIG. 3 further shows the ID field mining block 210 of FIG. 2 , in accordance with an embodiment of the present invention
  • FIG. 4 shows log formats 400 for the logs in FIG. 1 , in accordance with an embodiment of the present invention
  • FIG. 5 shows partial content of the inverse index hash table 500 for the logs in FIG. 1 with the log formats in FIG. 4 , in accordance with an embodiment of the present invention
  • FIG. 6 further shows the event automata modeling block 220 of FIG. 2 , in accordance with an embodiment of the present invention
  • FIG. 7 shows an event automata model 700 for the logs in FIG. 1 , in accordance with an embodiment of the present invention
  • FIG. 8 further shows the event sequence anomaly detection block 230 of FIG. 2 , in accordance with an embodiment of the present invention
  • FIG. 9 is a block diagram illustrating a computer system, in accordance with an embodiment of the present invention.
  • FIG. 10 is a flow diagram illustrating a method for heterogeneous log anomaly detection, in accordance with an embodiment of the present invention.
  • An embodiment of the present invention solves the problem of automatically detecting event ID content from heterogeneous logs and utilizing ID content for log sequence analysis in complex systems.
  • Event IDs in streams of system event logs are the type of content which has the following patterns: (i) appear the same in multiple log instances; (ii) appear in many unique values through the history; (iii) appear at stable locations in the same log event type; and (iv) appear in stable structure across multiple log event types.
  • Event IDs allow deterministic association of logs representing system/service behaviors such as database transactions, operational requests, work job scheduling events, and administration actions
  • an exemplary sequence of operational logs including event ID content 100 is illustratively shown.
  • logs 1 to 6 of the sequence of operational logs including event ID content 100 they make up one event and have the same event ID content 6432.12951 110 ; another event is composed of logs 7 to 12 with the same event ID content 6432.12987 120 .
  • Each event in the exemplary sequence has an open entry, a handshake entry, a bind entry, a search entry, an unbind entry, and a close entry.
  • An embodiment of the present invention provides a robust method to automatically discover event ID fields in heterogeneous logs; the event ID discovery algorithm is based on a log pattern set learnt from the same logs.
  • the embodiment of the present invention further provides a fast event automata profiling scheme based on the discovered event ID fields to model the log sequences in the heterogeneous logs in an automata model.
  • the embodiment of the present invention includes a log analysis application of event sequence anomaly detection based on the learnt event ID fields and automata model.
  • an anomaly-initiating one of a plurality of network devices e.g., a computer in a cluster of computers, and so forth
  • a plurality of network devices e.g., a computer in a cluster of computers, and so forth
  • control can involve powering down a root cause computer processing device at the anomaly-initiating one of the plurality of network devices to mitigate an error propagation therefrom.
  • control can involve terminating a root cause process executing on a computer processing device at the anomaly-initiating one of the plurality of network devices to mitigate an error propagation therefrom.
  • the computer causing the anomaly may also be isolated by disabling the port in the switch the computer is using to connect to the network.
  • a system administrator may also be alerted to the anomaly and computers/users causing the anomaly, so the system administrator can investigate the anomaly.
  • An embodiment of the present invention significantly reduces the complexity of finding statistically event sequence patterns in huge amounts of heterogeneous logs, even when prior knowledge about the system might not be available.
  • the embodiment of the present invention constructs an automatic event ID field discovery method for heterogeneous logs in a principled way, and allows faster event sequence analysis.
  • the automated event ID field analysis process 200 may include an ID field mining block 210 .
  • the ID field mining block 210 takes heterogeneous logs from arbitrary/unknown systems or applications.
  • the training stage uses the ID field mining block 210 for identifying associated pattern fields composing event IDs in heterogeneous logs.
  • the automated event ID field analysis process 200 may include an event automata modeling block 220 .
  • the event automata modeling block 220 corresponds to the processes of profiling and summarizing event behaviors on log sequence sets grouped by ID content, which is defined by the event ID field content discovered in the ID field mining block 210 .
  • the automated event ID field analysis process 200 may include an event sequence anomaly detection block 230 .
  • the event sequence anomaly detection block 230 takes heterogeneous logs collected from the same system in the ID field mining block 210 for event sequence behavior testing.
  • the process uses the event automata learnt from the event automata modeling block 220 for profiling and detecting abnormal behaviors of log sequence sets grouped by ID content, which is defined by the event ID field content discovered in the ID field mining block 210 .
  • FIG. 3 further shows the ID field mining block 210 of FIG. 2 , in accordance with an embodiment of the present invention.
  • the ID field mining block 210 includes a log format pattern recognition block 210 A, a reverse index of training logs block 210 B, an associate rule mining on pattern field pairs block 210 C, and an output the detected ID field block 210 D.
  • a set of log formats matching the training logs can be provided by users directly, or generated automatically by a format recognition procedure on all the heterogeneous logs as follows in block 210 A 1 - 210 A 3 :
  • Block 210 A 1 tokenization, similarity, clustering
  • Block 210 A 2 alignment, log motif discovery/recognition
  • Block 210 A 3 assignment ID.
  • a 1 (tokenization; similarity; clustering), taking the input heterogeneous logs, a generic tokenization is processed is so as to generate semantically meaningful tokens from logs. After the heterogeneous logs are tokenized, a similarity measurement on heterogeneous logs is applied. This similarity measurement leverages both the log layout information and log content information, and it is specially tailored to arbitrary heterogeneous logs. Once the similarities among logs are captured, a log hierarchical clustering algorithm can be applied so as to generate and output a log cluster hierarchy. AEIFA allows users to plug in their favorite clustering algorithms.
  • the following steps are a pattern recognition from such motifs.
  • fields such as time stamps, Internet Protocol (IP) addresses and universal resource locators (URLs) are recognized.
  • IP Internet Protocol
  • URLs universal resource locators
  • Second, other fields which are highly conserved in the logs are recognized and organized in a data structure in the cluster hierarchy. Note that the above log motif discovery and pattern recognition is first done in the lowest level of the log hierarchy. After that, all the information is back propagated up to higher levels in the hierarchy, and fused with their local patterns to provide format pattern representations from the logs.
  • the recognized log format patterns are in form of regular expressions.
  • a 3 assign a filed ID for each variable field in a recognized log format pattern.
  • the field ID consists of two parts: the ID of the log format pattern that this field belongs to, and the sequence number of this field compared to other fields in the same log format pattern.
  • the log format pattern IDs can be assigned with the integer number ⁇ 1, 2, 3, . . . n ⁇ for a log pattern set of size N, and the field sequence order can be assigned with the integer number ⁇ 1, 2, 3, . . . k ⁇ for a log format with k variable fields.
  • FIG. 4 shows 6 log formats 400 recognized for the logs in FIG. 1 .
  • the log formats are assigned pattern IDs from 1 to 6 ; in Pattern 1 , the 3 variable fields are assigned the IDs P 1 F 1 , P 1 F 2 , and P 1 F 3 ; variable fields in other format patterns are assigned ID in a similar way. That is, FIG. 4 shows an example of log format pattern recognition block 210 A for the logs in FIG. 1 , in accordance with an embodiment of the present invention.
  • reverse index generation procedure based on the log formats generated in the log format pattern recognition block 210 A as follows in block 210 B 1 - 210 B 2 :
  • Block 210 B 1 initialization
  • Block 210 B 2 formatting and assignment.
  • FIG. 5 shows partial content of the inverse index hash table 500 for the logs in FIG. 1 with the log formats in FIG. 4 . That is, FIG. 5 shows an example of reverse index of training logs block 210 B for the logs in FIG. 1 with the log formats in FIG. 4 , in accordance with an embodiment of the present invention.
  • (A) initialize a hash table T where the key is a composite index key, and the value is an object set;
  • (C) initialize a hash table F where the key is a composite index key, and the value is an integer initialized as 0;
  • FIG. 6 further shows the event automata modeling block 220 of FIG. 2 , in accordance with an embodiment of the present invention.
  • the event automata modeling block 220 includes a log grouping based on ID content block 220 A and an event automata generation based on long groups block 220 B.
  • set of logs grouped based on ID content as follows in block 220 A 1 - 220 A 2 :
  • Block 220 A 1 initialization
  • Block 220 A 2 build key and sort.
  • a 1 initialize a hash table G where key is a composite index key, and value is an ordered object list.
  • a 2 (build key and sort), for each log L i in the training logs, repeat the following: (i) if the log format P j matching this log contains any ID field discovered in the ID field mining block 210 , build a composite key k which consists of the log content matching those ID fields, and insert into the hash table G as G(k).insert((time_stamp(L i ), IDs(P j ))), where time_stamp(L i ) is the time stamp of L i , and the ordered object list is sorted by the time stamps; IDs(P j ) is the ID fields of the log format P j , otherwise, (ii) continue.
  • Block 220 B 1 initialization
  • Block 220 B 2 order
  • Block 220 B 3 output models.
  • FIG. 7 shows an event automata model 700 for the logs in FIG. 1 . That is, FIG. 7 shows an example of the event automata modeling block 220 for the logs in FIG. 1 , in accordance with an embodiment of the present invention.
  • FIG. 8 further shows the event sequence anomaly detection block 230 of FIG. 2 , in accordance with an embodiment of the present invention.
  • the event sequence anomaly detection block 230 includes initializing a hash table for active event automata instances block 230 A, a log grouping based on ID content block 230 B, and event automata matching on log sequence groups block 230 C.
  • the hash table E uses ID content as the key, and active automata instances as the value. Initially it is empty.
  • Block 230 C 1 alert missing instance
  • Block 230 C 2 insert new instance
  • Block 230 C 3 active instance.
  • an exemplary computer system 900 which may represent a server or a network device, in accordance with an embodiment of the present invention.
  • the computer system 900 includes at least one processor (CPU) 905 operatively coupled to other components via a system bus 902 .
  • a cache 906 operatively coupled to the system bus 902 .
  • ROM Read Only Memory
  • RAM Random Access Memory
  • I/O input/output
  • sound adapter 930 operatively coupled to the system bus 902 .
  • network adapter 990 operatively coupled to the system bus 902 .
  • user interface adapter 950 operatively coupled to the system bus 902 .
  • a first storage device 922 and a second storage device 929 are operatively coupled to system bus 902 by the I/O adapter 920 .
  • the storage devices 922 and 929 can be any of a disk storage device (e.g., a magnetic or optical disk storage device), a solid state magnetic device, and so forth.
  • the storage devices 922 and 929 can be the same type of storage device or different types of storage devices.
  • a speaker 932 may be operatively coupled to system bus 902 by the sound adapter 930 .
  • a transceiver 995 is operatively coupled to system bus 902 by network adapter 990 .
  • a display device 962 is operatively coupled to system bus 902 by display adapter 960 .
  • a first user input device 952 , a second user input device 959 , and a third user input device 956 are operatively coupled to system bus 902 by user interface adapter 950 .
  • the user input devices 952 , 959 , and 956 can be any of a sensor, a keyboard, a mouse, a keypad, a joystick, an image capture device, a motion sensing device, a power measurement device, a microphone, a device incorporating the functionality of at least two of the preceding devices, and so forth. Of course, other types of input devices can also be used, while maintaining the spirit of the present invention.
  • the user input devices 952 , 959 , and 956 can be the same type of user input device or different types of user input devices.
  • the user input devices 952 , 959 , and 956 are used to input and output information to and from system 900 .
  • the computer system 900 may also include other elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements.
  • various other input devices and/or output devices can be included in computer system 900 , depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art.
  • various types of wireless and/or wired input and/or output devices can be used.
  • additional processors, controllers, memories, and so forth, in various configurations can also be utilized as readily appreciated by one of ordinary skill in the art.
  • a flow chart for a heterogeneous log anomaly detection method 1000 is illustratively shown, in accordance with an embodiment of the present invention.
  • block 1020 generate an automata model by profiling event behaviors of the plurality of event sequences, the plurality of event sequences grouped in the automata model by combinations of one or more pattern fields and one or more event identifiers from among the plurality of event identifiers, wherein for a given combination, the one or more event identifiers therein must be respectively comprised in a same one of the one or more pattern fields with which it is combined.
  • block 1040 control an anomaly-initiating one of the network devices based on the anomaly.
  • Embodiments described herein may be entirely hardware, entirely software or including both hardware and software elements.
  • the present invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
  • Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system.
  • a computer-usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports the program for use by or in connection with the instruction execution system, apparatus, or device.
  • the medium can be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium.
  • the medium may include a computer-readable storage medium such as a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk, etc.
  • Each computer program may be tangibly stored in a machine-readable storage media or device (e.g., program memory or magnetic disk) readable by a general or special purpose programmable computer, for configuring and controlling operation of a computer when the storage media or device is read by the computer to perform the procedures described herein.
  • the inventive system may also be considered to be embodied in a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner to perform the functions described herein.
  • a data processing system suitable for storing and/or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus.
  • the memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code to reduce the number of times code is retrieved from bulk storage during execution.
  • I/O devices including but not limited to keyboards, displays, pointing devices, etc. may be coupled to the system either directly or through intervening I/O controllers.
  • Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks.
  • Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Debugging And Monitoring (AREA)
US15/429,849 2016-03-22 2017-02-10 Automated event ID field analysis on heterogeneous logs Expired - Fee Related US10237295B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/429,849 US10237295B2 (en) 2016-03-22 2017-02-10 Automated event ID field analysis on heterogeneous logs
PCT/US2017/017869 WO2017165018A1 (fr) 2016-03-22 2017-02-15 Analyse de champ d'identifiants d'évènement automatique sur des journaux hétérogènes

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662311436P 2016-03-22 2016-03-22
US15/429,849 US10237295B2 (en) 2016-03-22 2017-02-10 Automated event ID field analysis on heterogeneous logs

Publications (2)

Publication Number Publication Date
US20170279840A1 US20170279840A1 (en) 2017-09-28
US10237295B2 true US10237295B2 (en) 2019-03-19

Family

ID=59898843

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/429,849 Expired - Fee Related US10237295B2 (en) 2016-03-22 2017-02-10 Automated event ID field analysis on heterogeneous logs

Country Status (2)

Country Link
US (1) US10237295B2 (fr)
WO (1) WO2017165018A1 (fr)

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10664501B2 (en) * 2016-10-11 2020-05-26 International Business Machines Corporation Deriving and interpreting users collective data asset use across analytic software systems
US10462170B1 (en) * 2016-11-21 2019-10-29 Alert Logic, Inc. Systems and methods for log and snort synchronized threat detection
US10698926B2 (en) * 2017-04-20 2020-06-30 Microsoft Technology Licensing, Llc Clustering and labeling streamed data
US20190251138A1 (en) * 2018-02-09 2019-08-15 Banjo, Inc. Detecting events from features derived from multiple ingested signals
US10581945B2 (en) 2017-08-28 2020-03-03 Banjo, Inc. Detecting an event from signal data
US11025693B2 (en) 2017-08-28 2021-06-01 Banjo, Inc. Event detection from signal data removing private information
US10313413B2 (en) 2017-08-28 2019-06-04 Banjo, Inc. Detecting events from ingested communication signals
US10970184B2 (en) 2018-02-09 2021-04-06 Banjo, Inc. Event detection removing private information
US10585724B2 (en) 2018-04-13 2020-03-10 Banjo, Inc. Notifying entities of relevant events
US10261846B1 (en) 2018-02-09 2019-04-16 Banjo, Inc. Storing and verifying the integrity of event related data
US11195115B2 (en) 2018-02-21 2021-12-07 Red Hat Israel, Ltd. File format prediction based on relative frequency of a character in the file
WO2019195674A1 (fr) * 2018-04-06 2019-10-10 Banjo, Inc. Détection d'événements à partir de caractéristiques dérivées de multiples signaux ingérés
US11194813B2 (en) * 2018-07-06 2021-12-07 Open Text Sa Ulc Adaptive big data service
US11017085B2 (en) * 2018-07-06 2021-05-25 Percepio AB Methods and nodes for anomaly detection in computer applications
CN109086186B (zh) * 2018-07-24 2022-02-15 中国联合网络通信集团有限公司 日志检测方法及装置
CN109711656B (zh) * 2018-08-20 2023-10-03 云上广济(贵州)信息技术有限公司 多系统关联预警方法、装置、设备及计算机可读存储介质
US10764149B2 (en) * 2018-09-12 2020-09-01 The Mitre Corporation Cyber-physical system evaluation
CN110321371B (zh) * 2019-07-01 2024-04-26 腾讯科技（深圳）有限公司 日志数据异常检测方法、装置、终端及介质
US11537498B2 (en) * 2020-06-16 2022-12-27 Microsoft Technology Licensing, Llc Techniques for detecting atypical events in event logs
CN114301768A (zh) * 2020-09-23 2022-04-08 中国移动通信集团广东有限公司 网络功能虚拟化nfv设备的异常检测方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120137367A1 (en) * 2009-11-06 2012-05-31 Cataphora, Inc. Continuous anomaly detection based on behavior modeling and heterogeneous information analysis
US20130067575A1 (en) * 2003-04-04 2013-03-14 Juniper Networks, Inc. Detection of network security breaches based on analysis of network record logs
US20140344622A1 (en) * 2013-05-20 2014-11-20 Vmware, Inc. Scalable Log Analytics
US20160261482A1 (en) * 2015-03-04 2016-09-08 Fisher-Rosemount Systems, Inc. Anomaly detection in industrial communications networks

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101414255A (zh) * 2007-10-19 2009-04-22 国际商业机器公司 日志事件处理方法与设备
US10114148B2 (en) * 2013-10-02 2018-10-30 Nec Corporation Heterogeneous log analysis
US20150242431A1 (en) * 2014-02-25 2015-08-27 Ca, Inc. Computer system log file analysis based on field type identification

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130067575A1 (en) * 2003-04-04 2013-03-14 Juniper Networks, Inc. Detection of network security breaches based on analysis of network record logs
US20120137367A1 (en) * 2009-11-06 2012-05-31 Cataphora, Inc. Continuous anomaly detection based on behavior modeling and heterogeneous information analysis
US20140344622A1 (en) * 2013-05-20 2014-11-20 Vmware, Inc. Scalable Log Analytics
US20160261482A1 (en) * 2015-03-04 2016-09-08 Fisher-Rosemount Systems, Inc. Anomaly detection in industrial communications networks

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Xu, W. et al., "Detecting Large-Scale System Problems by Mining Console Logs" In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (SOSP '09) (Oct. 2009) pp. 117-132.

Also Published As

Publication number Publication date
US20170279840A1 (en) 2017-09-28
WO2017165018A1 (fr) 2017-09-28

Similar Documents

Publication Publication Date Title
US10237295B2 (en) Automated event ID field analysis on heterogeneous logs
US11971898B2 (en) Method and system for implementing machine learning classifications
US10353756B2 (en) Cluster-based processing of unstructured log messages
US10585908B2 (en) Method and system for parameterizing log file location assignments for a log analytics system
CN111639027B (zh) 一种测试方法、装置及电子设备
US11727025B2 (en) Method and system for implementing a log parser in a log analytics system
US10331441B2 (en) Source code mapping through context specific key word indexes and fingerprinting
US20220171800A1 (en) Clustering using natural language processing
CN109086182A (zh) 数据库自动告警的方法及终端设备
CN105786950A (zh) 一种处理数据的方法及装置
US20220222266A1 (en) Monitoring and alerting platform for extract, transform, and load jobs
US11676345B1 (en) Automated adaptive workflows in an extended reality environment
CN113672692B (zh) 数据处理方法、装置、计算机设备和存储介质
CN110008462A (zh) 一种命令序列检测方法及命令序列处理方法
JP7274162B2 (ja) 異常操作検知装置、異常操作検知方法、およびプログラム
CN106462629A (zh) 搜索中的直接答案触发
CN111858617A (zh) 用户查找方法和装置、计算机可读存储介质、电子设备
JP5206268B2 (ja) ルール作成プログラム、ルール作成方法及びルール作成装置
CN110059491A (zh) 数据导入监控方法、装置、设备及可读存储介质
CN105653533A (zh) 一种更新分类关联词集合的方法和装置
CN114579136A (zh) 代码处理方法、装置、计算机设备和存储介质
CN114253920A (zh) 一种交易重新排序方法、装置、设备及可读存储介质
CN110737662A (zh) 一种数据分析方法、装置、服务器及计算机存储介质
CN110275863A (zh) 文件移动方法、装置及存储介质
JP6212373B2 (ja) 操作ログ管理装置及び方法

Legal Events

Date Code Title Description
AS Assignment

Owner name: NEC LABORATORIES AMERICA, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHANG, HUI;JIANG, GUOFEI;REEL/FRAME:041226/0997

Effective date: 20170206

AS Assignment

Owner name: NEC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEC LABORATORIES AMERICA, INC.;REEL/FRAME:048078/0539

Effective date: 20190114

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20230319