CN100383784C - On-line analysing and treating system and method - Google Patents
On-line analysing and treating system and method Download PDFInfo
- Publication number
- CN100383784C CN100383784C CNB2004100002075A CN200410000207A CN100383784C CN 100383784 C CN100383784 C CN 100383784C CN B2004100002075 A CNB2004100002075 A CN B2004100002075A CN 200410000207 A CN200410000207 A CN 200410000207A CN 100383784 C CN100383784 C CN 100383784C
- Authority
- CN
- China
- Prior art keywords
- dimension
- log
- daily record
- module
- analytical processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The present invention relates to an online analysis treatment system and an online analysis treatment method, which is used for carrying out the online treatment of logs generated by devices. The online treatment analysis system comprises a log collection module, a log analysis module, a log preservation module, an intermediate treatment module and an online analysis treatment module, wherein the log collection module is used for collecting raw data and dynamically collecting logs sent by the devices; the log analysis module is used for analyzing the collected logs; the log preservation module is used for storing the logs after analysis; the intermediate treatment module is used for carrying out the intermediate treatment of the logs in the log preservation module, and the intermediate treatment module periodically generates and preserves intermediate results; the online analysis treatment module pretreats and displays the preliminarily existing intermediate results. The corresponding online analysis treatment method comprises the steps of log collection and analysis, log preservation, intermediate result periodical generation and online analysis treatment. With the used of the intermediate results, the present invention realizes the online analysis treatment, quickens response speed, and improves treatment efficiency. Because a special preparatory phase of the intermediate results does not exist and the dynamic collection can be carried out, problems can be instantaneously found.
Description
Technical field
The present invention relates to counting statistics machine information security fields, refer to a kind of on-line analysing processing system and method especially.
Background technology
Along with the development of infotech, data volume increases rapidly, and the accumulation of data is increasing.At information security field, many equipment (as, fire wall, system for monitoring intrusion, router, server etc.) all producing daily record all the time every day.When daily record amount less relatively (hundreds of bar even still less), experienced keeper can grasp these historical records with experience intuitively, finds wherein unusual to count data, finds the event log of being concerned about.But often equipment will produce several ten thousand even up to a million records every day, and such quantity has exceeded the receptible scope of keeper, and the keeper only can't handle these data with experience intuitively in the time that limits.Therefore, carry out on-line analytical processing (the On-lineAnalytical Processing of daily record, be called for short OLAP) be very important, on-line analytical processing is a kind of analytical approach, have gather, merging and polymerizable functional, and from the ability of different angle observed informations, the true cause that can in a large amount of log informations, pinpoint the problems by this method, and then carry out fault diagnosis, eliminating and system's reparation.
Thereby operations such as the common each polymerization of on-line analytical processing instrument all need the raw data from fact table to begin counting statistics causes the delay of dealing with problems, the hope that this is with the user pinpoints the problems as early as possible deviates from mutually, and is all the more so during particularly in the face of the daily record of magnanimity.For example, when hundreds thousand of capable daily record data, if having to travel through fact table with the counting statistics answer, then asking certain to gather need take a long time, and the client is difficult to tolerate its response and on-line analytical processing formation speed as a result.
For addressing this problem, for example Microsoft SQL Server of a kind of on-line analytical processing instrument appears
This on-line analytical processing instrument carries out on-line analytical processing from middle result, it is need be before the carrying out on-line analytical processing special preparatory stage that but the content of its data area in preparation generates, and what face must be a stable data storehouse, can't dynamically carry out on-line analytical processing while collecting data, and then can not carry out the on-line analytical processing of data immediately, can't find problem rapidly.
At the speed that how to improve on-line analytical processing, also there are some solutions:
What (1) No. 6567796 patent of the U.S. " System and method for management of anautomatic OLAP report broadcast system " realized is efficiently the on-line analytical processing report to be passed to plurality of devices simultaneously, its objective is the handling capacity and the speed that strengthen on-line analytical processing report distribution, and be convenient to the keeper distribution procedure is controlled.Though the technical scheme of this patent improves on-line analytical processing report distribution speed, on-line analytical processing result one reports that promptly the speed that generates is not enhanced.
(2) content of No. 6567804 patents of the U.S. " Shared computation of user-defined metrics in anon-line analytic processing system " be produce on-line analytical processing as a result the time in order to raise the efficiency, reuse and the public part of sharing complex expression and polymerization.Though the technical scheme of this patent can improve the efficient of on-line analytical processing, fundamentally do not improve on-line analytical processing result's formation speed yet.
In sum, on-line analytical processing is because of carrying out counting statistics from raw data in the prior art, and formation speed is slow as a result to cause on-line analytical processing, thereby incurs loss through delay the existence of finding the problem in the daily record data and may miss the preferably opportunity of dealing with problems.
Even utilize intermediate result to carry out on-line analytical processing, but need the stable data storehouse to reach the preparatory stage special before carrying out on-line analytical processing, can't find immediately that equally the problem in the daily record data exists.
Summary of the invention
The problem that the present invention solves is in the various operations of on-line analytical processing, avoids beginning counting statistics from raw data, thereby improves the speed of on-line analytical processing, pinpoints the problems immediately.
For addressing the above problem, on-line analysing processing system of the present invention, in-line processing is carried out in the daily record that is used for some equipment is produced, and comprising: the log collection module, be used for the collection of raw data, dynamic collection is carried out in the daily record that equipment sends over; Log analyzing module is used for the daily record of collecting is resolved; Log saving module is used for the daily record after the storing and resolving; Intermediate process module is used for the daily record of log saving module is carried out intermediate treatment and regularly produced and preserve intermediate result; The on-line analytical processing module, the intermediate result that utilization is pre-existing in is handled and is shown.
Described log collection module, intermediate process module and the concurrent working of on-line analytical processing module.Log saving module comprises the log sheet of preserving daily record and the dictionary table of dimension data, and dictionary table is corresponding with the external key in the log sheet by the major key of dictionary table with log sheet.Intermediate process module is every carries out polymerization once through one section preset time to log sheet, dictionary table, usually gathers to merge according to the low-dimensional level in each dimension and preserves intermediate result.
Correspondingly, on-line analytical processing method of the present invention may further comprise the steps: log collection and analyzing step are used for the dynamic collection daily record and will collect daily record resolving corresponding dimension data; Step is preserved in daily record, is used for preserving resolving the back daily record, usually daily record is kept in the log sheet, and dimension data is kept in the dictionary table; Regularly generate the intermediate result step, whenever log sheet, dictionary table are carried out according to the dimension polymerization once, and preserve intermediate result in middle table through one section preset time; The on-line analytical processing step obtains intermediate result in the intermediate list, and handles accordingly.
Described on-line analytical processing step further may further comprise the steps:
50) determine the on-line analytical processing condition, the condition of the definite scope of input and dimension, dimension hierarchical level information;
51) calculate the result of the dimension level of selected dimension by the intermediate result in the middle table, and show; 52) judge whether to finish on-line analytical processing? if decision finishes on-line analytical processing, then finish, otherwise change step 53);
53) select one of this dimension level to tie up the member, excite its corresponding link, indicate it is descended to bore;
Is not 54) judge arriving low-dimensional level? if not low-dimensional level, then can on this dimension, proceed to bore down, change step 55); Otherwise change step 56);
55) will tie up level adjusts to down one deck dimension level and will tie up level and adjust to the one deck below it and change step 58);
56) judge whether to specify other dimension in this dimension back? if specify other dimension in current dimension back, then can carry out the conversion of dimension, change step 57); Otherwise demonstration current results;
57) dimension of selecting appointment is as new current dimension, and with the ground floor dimension level of this dimension as current dimension level;
58) scope is adjusted in the scope of selecting this dimension member, determined that new on-line analytical processing condition adjusts to scope in this dimension of selection member's the scope, determine new on-line analytical processing condition, and forward 51 to).
In addition, log collection and analyzing step, regularly generate intermediate result step and the concurrent working of on-line analytical processing step.
Compared with prior art, the present invention has the following advantages:
On-line analysing processing system of the present invention and method by utilizing the intermediate result that regularly produces in advance, realize instant on-line analytical processing, improve response speed and treatment effeciency;
On-line analysing processing system of the present invention and method be not because there is the special preparatory stage of intermediate result, and can carry out dynamic collection, but collections, intermediate result generation, on-line analytical processing concurrent working in addition, so can pinpoint the problems immediately;
In the on-line analytical processing step, (Slice) operation of will cutting into slices merges with following brill (Drill Down) operation, and has realized the automatic conversion of dimension, thus more convenient user's use.
Description of drawings
Fig. 1 is an on-line analysing processing system block diagram of the present invention.
Fig. 2 is an on-line analytical processing method flow diagram of the present invention.
Fig. 3 is the refinement process flow diagram that step is preserved in daily record among Fig. 2.
Fig. 4 is the process flow diagram of refinement among Fig. 3.
Fig. 5 is the refinement process flow diagram that regularly generates the intermediate result step among Fig. 2.
Fig. 6 is that on-line analytical processing method of the present invention is with dimension hierarchical structure block diagram in the specific embodiment.
Fig. 7 is the refinement process flow diagram of on-line analytical processing step among Fig. 2.
Embodiment
Counting statistics combined data and to generate intermediate result be the fast basis of response of on-line analytical processing in advance.Need not carry out data and gather at polymerization stage at converging operation, this polymerization can be carried out counting statistics and needn't carry out counting statistics from raw data from middle result, thereby further fast reaction speed, but also to guarantee that the result of on-line analytical processing is understandable for the user and uses.Do not gather if do not carry out these data in advance, when then having the daily record of magnanimity, response speed is that the user can not put up with at all.If counting statistics is good in advance but be used to respond the data of this converging operation, then almost can make an immediate response.
In order to the following describes conveniently, introduce the term in several on-line analytical processings here.Cube is the main object in the on-line analytical processing, it is a data set, usually by the subset construction of data warehouse (DW) or database, and tissue and be aggregated into a multidimensional structure by one group of dimension and metric definition, cube extracts its raw data from data warehouse.Cube is regarded data as data cube (Data Cube) form, and this data cube allows with multidimensional to data modeling and observation, by the true definition of peacekeeping.Each data cube is called a square body (Cuboid) or piece (Chunk), deposits the square body that the bottom gathers and is called basic side's body (Base Cuboid).
Each cube all has a framework, and this framework is the respectively set of table (fact table and dictionary table) that has connected in the data warehouse.Core table in the framework is fact table (storage raw data, the log sheet in the present embodiment), and fact table is the source of cube metric.In addition, dimension table is a kind of special dictionary table, is the source of cube dimension.Each cube framework includes a fact table and one or more dimension table.The metric of cube comes from the row in the fact table, and the dimension of cube comes from the row in the dimension table.
Concentrate at multidimensional data, metric is a class value, and these values are listed as based on one in the fact table of cube, and is generally numeral.In addition, metric is the central value of the cube analyzed.That is, metric is final user's numerical data that emphasis is checked when browsing cube.Dimension then is the architectural characteristic of cube, is the special angle of people's observation problem, promptly is used for attribute or property value are organized into different level of abstractions, for example time dimension, geographical dimension, product dimension.Can also have the different a plurality of descriptions aspect of level of detail according to certain dimension, this a plurality of descriptions aspect is the level of dimension, promptly ties up level, for example, and time dimension: year, season, month, day.A value of dimension is become a dimension member of this dimension, if dimension is poly-have multi-level, then tieing up the member is exactly the combination of different dimensional level value, for example, geographical dimension has three levels " province, city, county ", and then " Changyi county, Weifang City, Shandong Province " just constitutes one of geographical dimension dimension member, the dimension member might not be on each level of dimension values all, for example, Shandong Province, Weifang City, Shandong Province all are the dimension members of geographical dimension.
---back will be called dimension data---can also be divided into two kinds for the data in the dimension table: a kind of is stable, protocol type for example, it can not change along with the change of time, another kind of is constantly to change, for example name is attacked in invasion, user name and virus name etc., when each equipment uses, because user's additions and deletions, the upgrading of virus characteristic storehouse and invasion intrusion feature database, dimension data in these dimension tables is constantly to change, also can be called as dynamic dimension data, therefore for the dimension data in the stable dimension table, how to obtain and the dimension data managed in these dimension tables that constantly change is a very important job.
Following to bore (Drill Down) operation be to obtain by the data of the overview data to details, and following brill can be by downward along the data hierarchy of dimension or introduce new dimension and realize.Section (Slice) operation is to select on a dimension of given data cube.Stripping and slicing (Dice) operation is selected by two or more dimensions are carried out.
Please refer to shown in Figure 1, on-line analysing processing system of the present invention can utilize the characteristics that daily record is added up to accelerate the speed of massive logs on-line analytical processing to being that the content of the massive logs collected of a kind of equipment 200 is carried out on-line analytical processing from plurality of devices 200 (raw data is provided) simultaneously.Equipment 200 of the present invention includes but not limited to that fire wall, system for monitoring intrusion, router and server etc. generate the equipment of daily record.
On-line analysing processing system of the present invention is used for in-line processing is carried out in the daily record that some equipment 200 are produced, and comprising: log collection module 110, log analyzing module 120, database 100, on-line analytical processing module 300 and interface 310.
Described log collection unit 110, be used for the collection of raw data, dynamic collection is carried out in the i.e. daily record that equipment 200 is sended over, and send daily record resolution unit 120 to, mainly be daily record to be sent to daily record resolution unit 120 from a plurality of equipment 200 in the present embodiment by Syslog or snmp protocol etc.
Described daily record resolution unit 120, be used for the parsing of raw data, promptly real time parsing is carried out in the daily record of equipment 200, parse corresponding attribute information (include but not limited to dimension data, the dimension data in the present embodiment comprises static dimension data and dynamic dimension data).Usually, the daily record that equipment 200 sends is a character string that certain format is arranged, and daily record resolution unit 120 receives after these character strings, these character strings of interpretation of rules according to form, therefrom analyze desired content, in the present embodiment, the method for parsing adopts the technology based on plug-in unit.
Described database 100 is used for the intermediate treatment of daily record after the storing and resolving and daily record and regularly produces intermediate result, comprises that solid will preserves module 130 (including log sheet and dictionary table) and intermediate process module 140.
Log sheet in the described log saving module 130 is exactly a fact table, the daily record correspondence that each bar of daily record is received a record in the log sheet, set up a log sheet for one type daily record, the alarm log of for example all invasion attacks all is kept in the invasion attack logs table;
Described dictionary table is used for illustrating the attribute in a certain respect of daily record, when the daily record with daily record is saved in the log sheet, the correlation attribute information that parses is saved in the dictionary table.May have a plurality of attributes for a daily record, then each property value is stored in the corresponding dictionary table, promptly has a plurality of dictionary tables.For example for the alarm log of invading attack, just there are many dictionary tables, just listed all protocol types as the agreement dictionary table, just can represent this attribute as long as in log sheet, there is a field to preserve any bar record that each bar daily record correspondence the agreement dictionary table.Dictionary table is corresponding with the external key in the log sheet by the major key of dictionary table with log sheet, hereinafter can continue to describe.
Described intermediate process module 140 is regularly handled the intermediate result that generates on-line analytical processing with the daily record of collecting (every spend one period scheduled period), and is saved in the middle table.In the on-line analytical processing process, middle table is exactly the data area in preparation.
Described on-line analytical processing module 300 then utilizes the intermediate result that is pre-existing in the middle table to handle and be presented on the interface 310, comprises dimension conversion process, brill processing down, operations such as stripping and slicing, section.
In addition, the intermediate process module and the on-line analytical processing module of log collection module, generation intermediate result can be concurrent workings.
Each module details content of on-line analysing processing system of the present invention please refer to the respective description in the on-line analytical processing method.
Please refer to Fig. 2, on-line analytical processing method of the present invention comprises:
Log collection and analyzing step D1, be used for the dynamic collection daily record and daily record is resolved, from some equipment collector journals, and will collect daily record and resolve, corresponding attribute information (include but not limited to dimension data, the dimension data in the present embodiment comprises static dimension data and dynamic dimension data);
Daily record is preserved step D2, is used for preserving and resolves the back daily record, usually daily record is kept in the log sheet as raw data, and attribute information is kept in the dictionary table;
Regularly generate intermediate result step D3, whenever, log sheet, dictionary table are carried out polymerization once through one section preset time, usually gathering polymerization and preserve intermediate result in middle table according to the low-dimensional level in each dimension, can certainly not be from low-dimensional level as required;
On-line analytical processing step D4 obtains intermediate result in the intermediate list, and handles accordingly.
Log collection and analyzing step D1, regularly generate intermediate result step D3, on-line analytical processing step D4 can concurrent working.
For example for the original log of collecting:
"＜1〉LX-NIDS[11174]: | web-attacks_6|WEB attacks: id command attempt[Classification; Web Application Attack] [Priority:0]: TCP}202.210.0.1:1090-〉10.50.10.8:80 " described log collection and analyzing step D1 can parse following information:
Invasion is attacked name: WEB and is attacked: id command attempt
Host IP address: 202.210.0.1 is attacked in invasion
Invasion attacked port: 1090
Invaded attack host IP address: 10.50.10.8
Invaded attacked port: 80
Event level: 0
Protocol type: TCP.
Please refer to Fig. 3, is the process flow diagram that step D2 is preserved in daily record, and daily record is preserved step D2 and further comprised the steps:
Step S1, after receiving the daily record of an equipment of having resolved, according to the definite log sheet that will preserve of the content of daily record, log sheet can exist a plurality of according to the difference of the kind of handling daily record, for example invades attack logs table, virus event log sheet and user access logs table or the like;
Step S2, before concrete log content is saved in log sheet, (it is just passable only need to set the dictionary table that a content remains unchanged for static dimension data in system to collect dynamic dimension data, then need constantly from daily record, to collect for dynamic dimension data), non-existent dynamic dimension data in the dictionary table is preserved;
Please in the lump with reference to Fig. 4, step S2 further comprises the steps: at first execution in step S21, judges in the daily record after resolving whether also have untreated dynamic dimension data; If there is no then finish,, obtain following first untreated dynamic dimension data if exist then execution in step S22; Step S23, with this untreated dynamic dimension data be kept at corresponding dictionary table in the dynamic dimension data collected compare; If had this untreated dynamic dimension data in the dictionary table, then returned step S21; If the dynamic dimension data that does not exist this daily record to comprise in the dictionary table, then implementation step S24 should be saved in the dictionary table by dynamic dimension data, and returns step S21 and preserve until finishing all dynamic dimension datas;
Step S3, the daily record for after resolving deposits log sheets different in the database respectively in;
Step S4 judges whether to continue to handle the daily record after the parsing, if continue then return step S1 and preserve up to all daily records and finish, if do not continue then finish.
For clearer on-line analytical processing method of the present invention is described, be that example is described with the statistics of alarm log below to the invasion attack, but this method is not limited to the statistics to the alarm log of invasion attack, can also be applied to the statistics to other daily record (as virus event alarm log, user access logs).
The alarm log of all invasion attacks all will be kept in the invasion attack logs table, and dictionary table is corresponding with the external key in the log sheet by the major key of dictionary table with log sheet, and wherein the structure of this log sheet is as shown in table 1.
Table 1 invasion attack logs list structure
| Field name | Explanation |
| LIndex | Automatically increasing since 0, is the major key of log sheet, is used for uniquely in log sheet determining a daily record. |
| GTime | Is described, if should the time in the daily record then insert as time proximity with the time of receiving daily record the time that present event takes place. |
| AlertTitleID | The ID of warning theme is used for illustrating that name is attacked in the invasion of this attack.It is an external key, illustrates this is attacked corresponding which invasion of invading in the attack famous-brand clock and attacks name, and is promptly corresponding with the major key AlertTitleID that invades the attack famous-brand clock, illustrates that name is attacked in the invasion of each bar daily record. |
| SrcIP | Original ip address just initiates to invade the IP address of the main frame of attacking |
| SrcPort | Original port points out that this attack is the attack of attacking which port initiation of main frame from invasion. |
| DestIP | Purpose IP address, the IP address of the main frame of being attacked is attacked in this invasion. |
| DestPort | Destination interface is pointed out port under attack. |
| IPProtoID | Protocol type, this attack by which procotol is undertaken.It is an external key, and explanation is that agreement in the IP protocol tables, and promptly the major key IPProtoID with the IP protocol tables is corresponding, and the protocol type of each bar daily record is described. |
| EventLevelID | The order of severity of this attack.It is an external key, and explanation is which rank in the event level table, and explanation is that agreement in the event level table, and promptly the major key EventLevelID with the event level table is corresponding, and the event level of each bar daily record is described. |
| RepeatNum | The same log occurrence number that this record is corresponding, default is 1 |
| OriginalMsg | The content of original log |
Relevant dictionary table has three, comprises invasion attack famous-brand clock, IP protocol tables and event level table, and its structure is respectively shown in table 2, table 3 and table 4.
The name list structure is attacked in table 2 invasion
| Field name | Explanation |
| AlertTitleID | Major key is used for correspondingly with AlertTitleID in the log sheet, illustrates that name is attacked in the invasion of each bar daily record correspondence. |
| AlertTitle | Name is attacked in invasion. |
Table 3IP protocol tables structure
| Field name | Explanation |
| IPProtoID | Major key is used for correspondingly with IPProtoID in the log sheet, and each bar daily record corresponding protocols name is described. |
| IPProtoName | The agreement name. |
Table 4 event level list structure
| Field name | Explanation |
| EventLevelID | Major key is used for correspondingly with the EventLevelID of log sheet, and the order of severity of the corresponding incident of each bar daily record is described. |
| EventLevelName | The title of this order of severity. |
Wherein, invading and attacking the famous-brand clock original state is sky, replenishes by constantly collect AlertTtile in the daily record of receiving, maintenance invasion attack famous-brand clock is consistent with daily record.And because the kind of agreement and the value of event level determine that so the content of IP protocol tables and event level table do not need to collect, the content of these two kinds of dictionary tables remains constant.
Suppose that during two o'clock in the morning to three on January 1st, 2003 log statistic system received the alarm log of as shown in table 5 ten invasion attacks, do not receive the alarm log that any invasion is attacked in other working time.
The daily record that table 5 is received
| Receive the time of daily record | The log content of receiving |
| 2003-01-01?02:10:11 | <70>LX-NIDS[1876]：spp_bo：Back?Orifice?Traffic?detected(key：31337) {UDP}202.210.0.5:1125-＞10.50.10.8:31337 |
| 2003-01-01?02:11:51 | <69>LX-NIDS[1876]：spp_bo：Back?Orifice?Traffic?detected(key：31337) {UDP}202.210.0.5:1125-＞10.50.10.8:31337 |
| 2003-01-01?02:12:11 | <68>LX-NIDS[1876]：spp_bo：Back?Orifice?Traffic?detected(key：31337) {UDP}202.210.0.5:1125-＞10.50.10.8:31337 |
| 2003-01-01?02:12:32 | <67>LX-NIDS[1876]：spp_bo：Back?Orifice?Traffic?detected(key：31337) {UDP}202.210.0.2:1125-＞10.50.10.8:31337 |
| 2003-01-01?02:15:11 | <66>LX-NIDS[1876]：spp_bo：Back?Orifice?Traffic?detected(key：31337) |
| {UDP}202.210.0.1:1125-＞10.50.10.8:31337 | |
| 2003-01-01?02:20:17 | ＜1〉LX-NIDS[11174]: | web-attacks_6|WEB attacks: id command attempt [Classification:Web Application Attack] [Priority:0]: TCP}202.210.0.1:1090-＞10.50.10.8:80 |
| 2003-01-01?02:30:19 | ＜2〉LX-NIDS[3350]: | web-cgi_45|WEB attacks: files.pl access[Classification:Attempted Information Leak] [Priority:0]: { TCP}202.210.0.1:1090-＞10.50.10.8:80 |
| 2003-01-01?02:35:10 | ＜3〉LX-NIDS[3350]: | web-coldfusion_31|WEB attacks: onrequestend.cfm access [Classification:Attempted Information Leak] [Priority:0]: { TCP} 202.210.0.1:1090-＞10.50.10.8:80 |
| 2003-01-01?02:40:19 | ＜1〉LX-NIDS[11174]: | web-attacks_6|WEB attacks: id command attempt [Classification:Web Application Attack] [Priority:0]: TCP}202.210.0.1:1090-＞10.50.10.8:80 |
| 2003-01-01?02:50:22 | ＜1〉LX-NIDS[11174]: | web-attacks_6|WEB attacks: id command attempt [Classification:Web Application Attack] [Priority:0]: TCP}202.210.0.1:1090-＞10.50.10.8:80 |
After 2003-01-01 02:10:11 receives that alarm log is attacked in article one invasion, name Back Orifice is attacked in the invasion of its correspondence to be saved in the invasion attack famous-brand clock, and five day of the second to the of back invasion attack alarm log reports that it all is Back Orifice that name is attacked in invasion, therefore be not that name is attacked in new invasion, attack name WEB attack and in the 6th invasion attack alarm log, collect new invasion: idcommand attempt, adding it to invasion attacks in the famous-brand clock, the processing of back is similar, and this does not give unnecessary details.It is as shown in table 6 that the famous-brand clock content is attacked in the invasion that forms at last.
Famous-brand clock is attacked in table 6 invasion
| AlertTitleID | AlertTitle |
| 0 | Back?Orifice |
| 1 | WEB attacks: id command attempt |
| 2 | WEB attacks: files.pl access |
| 3 | WEB attacks: onrequestend.cfm access |
Wherein, AlertTitleID is the sign type in invasion attack famous-brand clock, and just system generates sequence number value automatically, the delegation in this sequence number value unique identification table.AlertTitleID in AlertTitleID and the log sheet has corresponding relation.
Because the kind of agreement and the value of event level determine, so the content of IP protocol tables and event level table do not need to collect, and the content of these two kinds of dictionary tables remains constant, respectively shown in table 7 and table 8.
Table 7IP protocol tables
| IPProtoID (hexadecimal representation) | ?IPProtoName | Explanation |
| ?0x00?00?00?01 | ?ICMP | The ICMP agreement |
| ?0x00?00?00?02 | ?TCP | Transmission Control Protocol |
| ?0x00?00?00?04 | ?UDP | Udp protocol |
| ?0x00?00?00?08 | ?SIPP-ESP | The SIPP-ESP agreement |
| ?0x00?00?00?10 | ?SIPP-AH | The SIPP-AH agreement |
| ?0x00?00?00?20 | ?IGRP | The IGRP agreement |
| ?0x00?00?00?40 | ?OSPFIGP | The OSPFIGP agreement |
| ?0x00?00?00?80 | Other | Other agreements |
Table 8 event level table
| EventLevelID | EventLevelName |
| 0x01 | Do not have |
| 0x02 | Low |
| 0x04 | In |
| 0x08 | High |
The IP address all is to deposit with the form of integer in database, is stored as 83940042 as 202.210.0.5, promptly hexadecimal 0x0500D2CA.For convenience of description, set up the IP address that occurs in this example and the corresponding relation of integer below, conversion between the two all has corresponding function under development environment.
| Integer | The IP address |
| 83940042 | 202.210.0.5 |
| 33608394 | 202.210.0.2 |
| 16831178 | 202.210.0.1 |
| 134885898 | 10.50.10.8 |
| 118108682 | 10.50.10.7 |
| 101331466 | 10.50.10.6 |
After daily record was preserved step D2 and finished, aforementioned ten daily records were saved in the invasion attack logs table in the database, and the content of preservation is as shown in table 9.
Utilizing the mode that regularly generates the middle table data to carry out polymerization is owing to have two characteristics for daily record, the first, and daily record is according to unidirectional increase of time; The second, the summarized results of daily record on-line analytical processing all is (Distributive) or (Algebraic) of algebraically that distributes usually.
Summarized results distributes, and is meant that it can calculate by following distribution mode: establish data set and be divided into n set, can utilize the operating function of this result of calculation to obtain a result of calculation on each part.The result of calculation of all parts is used the result of same operating function acquisition and all data is directly utilized coming to the same thing of this operating function acquisition.
Summarized results is an algebraically, if it can have an algebraic function with M parameter to calculate (wherein M is a bounded integer), and each parameter all is the result of a distribution.
Therefore, generate intermediate result step D3 regularly, though the time is continuous, but because selected normally significant time period when carrying out log statistic, therefore determine a minimum time section that statistical significance is arranged as time statistics dimension, daily record is once calculated the daily record in the last minimum time section every a minimum time section, just daily record is divided set according to time and other dimension data condition, each set is calculated the parameter of each distribution of the statistics of the statistics of each distribution and algebraically respectively, and this intermediate result is saved in middle table, when carrying out log statistic, just can utilize these intermediate results directly to carry out statistical computation, thereby greatly raise the efficiency.Directly utilize aggregate function to calculate for the summarized results that distributes; Summarized results for algebraically then calculates its each parameter that distributes earlier, utilizes the calculation of parameter result when carrying out on-line analytical processing again.
Please refer to Fig. 5, is the process flow diagram that regularly generates intermediate result step D3 in the on-line analytical processing method of the present invention, regularly generates intermediate result step D3 and further comprises the steps:
Step S5, at first judge between start time and concluding time intermediate result whether as calculated? because allow manual this flow process of carrying out, therefore can recomputate, and replace original result of calculation with new result of calculation to the good as calculated time period.If therefore the intermediate result between start time and concluding time as calculated a part then change step S6, otherwise change step S7.When regularly carrying out, start time and concluding time are respectively the time that finished a last time of carrying out and a last time period.
Step S6 is deletion existing intermediate result between start time and concluding time, and purpose is in order to deposit new intermediate result in.
Step S7 is first minimum time section after the acquisition start time, and what exist between permission start time and concluding time in this flow process is not a minimum time section, and comprises the situation of a plurality of minimum time sections.
Implementation step S8 subsequently, the statistics that set that each statistics dimension in this minimum time section is formed utilizes log sheet to calculate to distribute and the parameter of algebraically statistics, i.e. intermediate result, and it is saved in the corresponding middle table.
Step S9 judges whether to reach the concluding time, if arrive, and process ends then; If no, then enter step S10, obtain to calculate the next minimum time section of the time period of finishing, and jump to step S8 and calculate.
Alarm log with above-mentioned invasion attack is that example describes equally, attacks three dimensions of name according to time of being taken place by object of attack, invasion attack and invasion and carries out on-line analytical processing.Wherein attacked main frame by the low-dimensional level of object of attack, be made into group and attack host groups, group can comprise the group or the main frame of subordinate; And low-dimensional level of the time that the invasion attack takes place is one hour; Invasion is attacked name and is had only one deck notion.
The tolerance of the alarm log of invasion attack is the number of times that the invasion attack takes place.The middle table form of the on-line analytical processing of invasion attack is as follows: the alarm log with aforementioned invasion attack is an example, the result of as if statistics is included in the total degree that the invasion attack takes place in a certain designated duration and invades the number percent that the invasion attack of attacking name accounts for all invasion attacks with each, and then the structure of middle table can be as shown in table 10.
The structure of table 10 middle table
| Field name | Explanation |
| HourID | Since the hourage of reference time as 01 month 00:00:00 on the 01st in 2003 |
| DestIP | Purpose IP address, the IP address of the main frame of being attacked is attacked in this invasion. |
| AlertTitleID | The ID of warning theme is used for illustrating that name is attacked in the invasion of this attack.It is an external key, illustrates this is attacked corresponding which invasion of invading in the attack famous-brand clock and attacks name. |
| CountNum | Tolerance is just found the number of times that invasion is attacked under these conditions. |
Please refer to organization chart between group shown in Figure 6, above-mentioned and main frame, group represents by two tables that with the relation of main frame what the main frame table reflected is the information of each main frame, and its structure is as shown in table 11:
Table 11 main frame list structure
| Field name | Explanation |
| HostIP | The IP address of main frame, major key, if DestIP=HostIP in the present embodiment, the main frame attacked of the invasion attack of this log record is exactly this main frame so |
| HostName | The title of main frame |
| GroupID | Affiliated ID number of organizing of this main frame, external key is corresponding with the GroupID of group. |
What the group table reacted is the information of each group, its structure such as table 12:
Table 12 group list structure
| Field name | Explanation |
| GroupID | Organize ID number, major key is used for group of unique appointment in this table, notices that 0 is not effectively to organize ID number. |
| GroupName | Group name claims |
| ParentGroupID | The ID of corresponding higher level's group if there is not the higher level to organize, then is 0 |
In this example, main frame table content such as table 13:
Table 13 main frame table content
| HostIP | HostName | GroupID |
| 134885898 | host_10.50.10.8 | 2 |
| 118108682 | host_10.50.10.7 | 2 |
| 101331466 | host_10.50.10.6 | 3 |
And the content of group table is a table 14:
Table 13 group table content
| GroupID | GroupName | ParentGroupID |
| 1 | Group 61 | 0 |
| 2 | Group 62 | 1 |
| 3 | Group 63 | 1 |
The total degree of wherein invading the attack generation is the statistics that distributes, can be by each minimum time section and each invasion attack name be divided set to data, regularly add up the statistics of these set again and they are saved in the middle table as intermediate result, after the time period of having determined final statistics, just can calculate final statistics by these middle table.
The number percent that the invasion attack that name is attacked in each invasion accounts for all invasion attacks then is the statistics of an algebraically, number of times of attacking the invasion attack of name because of the invasion total degree of attack and each invasion all is the result who distributes, and this number percent can obtain divided by the total degree of invasion attack by number of times that the invasion attack of name is attacked in each invasion.
If the minimum time section of definition is one hour, be that 2003-01-0103:00:00 will be to handling in the daily record that 2003-01-01 02:00:00～2003-01-01 02:59:59 receives so after one hour, we are reference time with 2003-01-01 00:00:00, so above-mentioned 2003-01-0102:00:00～2003-01-01 02:59:59 should be corresponding be 01 minimum time section, 2003-01-0100:00:00～2003-01-01 01:59:59 correspondence be 00 minimum time section.
First minimum time section of obtaining among the step S7 is 01 minimum time section.
Among the step S8, can insert the record of each hour by SQL statement in middle table, the content of the middle table of generation is as shown in table 14.
The content of table 14 middle table
| HourID | AlertTitleID | CountNum | DestIP |
| 1 | 0 | 5 | 134885898 |
| 1 | 1 | 3 | 134885898 |
| 1 | 2 | 1 | 134885898 |
| 1 | 3 | 1 | 134885898 |
Intermediate result in utilizing middle table realizes on-line analytical processing step D4, utilize intermediate result to show the result of on-line analytical processing, and realized that section (Slice) operation merges with following brill (Drill Down) operation and the conversion of dimension, thereby more convenient user's use.
Usually as condition, and drill down operator is to carry out on the basis of the piece (Chunk) that stripping and slicing operation and sectioning form, and uses very inconvenient user's understanding with stripping and slicing operation and sectioning in traditional on-line analytical processing.The present invention operates the piece that forms preliminary on-line analytical processing as condition except utilizing stripping and slicing, and presents according to the form of this dimension with report.
Further operation then is to carry out drill down operator according to the dimension member that the user selects, simultaneously thereby the dimension member of this dimension is realized a complete report that realizes by drill down operator, user friendly understanding as the condition of sectioning on the basis of new sub-piece.Simultaneously, when having arrived the low-dimensional level of certain dimension, the common hope of user can be automatically brought to the understanding of other dimension to make things convenient for the user that each metric is gathered.
Still the alarm log with top invasion attack is an example, and being that dimension describes by object of attack: group 61 is tieed up level topmost by object of attack, and group 62, group 63 are second layer dimension levels, is attacked main frame and be exactly low-dimensional level.If the user selects to be tieed up level topmost by object of attack, and according to time range is that 2003-01-01 02:00:00～2003-01-01 02:59:59, all invasion attacks condition by name are carried out the stripping and slicing operation, then at last that its result is as follows, wherein the number of times of being attacked be according to HourID in the middle table be 1, according to specified main frame all in group 61 of DestIP CountNum's and (directly the intermediate result of intermediate list obtains), as shown in Table 15:
The content of table 15 stripping and slicing form
| By object of attack | The number of times of invaded attack |
| Group 61 | 10 |
(Slice) operation will merge with following brill (Drill Down) operation owing to will cut into slices, therefore after " group 61 " link above clicking, just will be according to the condition section that is limited to by firing area in group 61 the scope, and according to by object of attack dimension brill down, shown in table 16:
The content of table 16 time brill form
| By object of attack | The number of times of invaded attack |
| Group 62 | 10 |
| Group 63 | 0 |
Wherein organize 62 by number of times of attack, be by middle table with all HourID be 1, is attacked that main frame writes down in group 62 CountNum's and; Group 63 by number of times of attack, be by middle table with all HourID be 1, is attacked that main frame writes down in group 63 CountNum's and.When the user clicks " group 62 ", will trigger corresponding link, according to the section of the condition in the scope that is limited to group 62 by firing area, and according to being descended to bore by the object of attack dimension, owing to do not organized below, just can be directly with its directly under main frame list, and utilize report tool to form form (seeing Table 17):
Table 17
| By object of attack | The number of times of invaded attack |
| host_10.50.10.8 | 10 |
| host_10.50.10.7 | 0 |
Wherein the number of times of invaded attack is by middle table, is 1 with HourID, DestIP be respective host IP address record CountNum and.After the user clicks " host_10.50.10.8 ", triggered corresponding link, at this moment arrived the low-dimensional level of " by object of attack " dimension.Carry out the conversion (convert to what dimension can set) of dimension then, dimension is converted to invasion attacks the name dimension, show so just according to the section of the condition in the scope that is limited to host_10.50.10.8 by firing area, and according to invasion attack name, see Table 18:
Table 18
| Name is attacked in invasion | The number of times that invasion is attacked |
| Back?Orifice | 5 |
| WEB attacks: id command attempt | 3 |
| WEB attacks: files.pl access | 1 |
| WEB attacks: onrequestend.cfm access | 1 |
Wherein the number of times attacked of invasion all is to obtain by middle table, is 1 with HourID, and DestIP is that 134885898-is the IP address of host_10.50.10.8, invasion attack record that this row invasion by name attacks name CountNum's and.
Please refer to shown in Figure 7ly, in sum, on-line analytical processing step D4 further comprises the steps:
Step S40 determines the on-line analytical processing condition, for example, and the condition of the definite scope of input and dimension, dimension hierarchical information;
Step 41 is calculated the result of the dimension level of selected dimension by the intermediate result in the middle table, and is shown.The user can carry out stripping and slicing by the range of condition of input, and according to dimension that provides and dimension level display result.This calculating can realize by storing process, and the result that will obtain passes to report tool, utilizes report tool to present to the user with the form of form, user friendly understanding and use;
Does step 42 judge whether to finish on-line analytical processing? if decision finishes on-line analytical processing, then finish, otherwise change step S43;
Step S43 selects certain value (promptly tieing up the member) of this dimension level, excites its corresponding link, indicates it is descended to bore;
Step S44, is not judge arriving low-dimensional level? if not low-dimensional level, then can on this dimension, proceed to bore down and change step S45, the dimension level is adjusted to the one deck below it and changeed step S49; Otherwise just need forward other dimension to, change step S46;
Does step S46 judge whether to specify other dimension in this dimension back? if specified dimension in current dimension back, then can carry out the conversion of dimension, change step 47; Otherwise show current results (seeing S48);
Step S47, the dimension of selecting appointment is as new current dimension, and with the ground floor dimension level of this dimension as current dimension level;
Step S49 adjusts to scope in the scope of selecting this dimension member, determines new on-line analytical processing condition, and forwards step S41 to.
In sum, on-line analysing processing system of the present invention and method by utilizing the intermediate result that regularly produces in advance, realize instant on-line analytical processing, improve response speed and treatment effeciency.
Utilize middle table realize on-line analytical processing, the present invention will cut into slices (Slice) operation with bore (Drill Down) operation merging down, and realized the automatic conversion of dimension, thus more convenient user's use.Usually as condition, and drill down operator is to carry out on the basis of the piece (Chunk) that stripping and slicing operation and sectioning form, and uses very inconvenient user's understanding with stripping and slicing operation and sectioning in traditional on-line analytical processing.
Therefore, the present invention forms the piece of preliminary on-line analytical processing except utilizing the stripping and slicing operation as condition, and according to this dimension level with the report form present, the further operation of subsequent user then is to carry out drill down operator according to a value of the dimension level of user's selection, thereby the value that will tie up level simultaneously realizes a complete report that realizes by drill down operator, user friendly understanding as the condition of sectioning on the basis of new piece.
Simultaneously, when having arrived the low-dimensional level of certain dimension, the common hope of user can be automatically brought to the understanding of other dimension to make things convenient for the user that each metric is gathered, and has realized the automatic conversion of this dimension, and present to the user with the form of complete report, made things convenient for user's understanding and use.
Claims (10)
1. on-line analysing processing system, in-line processing is carried out in the daily record that is used for some equipment is produced, and comprising:
The log collection module is used for the collection of raw data, and dynamic collection is carried out in the daily record that equipment sends over;
Log analyzing module is used for the daily record of collecting is resolved;
Log saving module is used for the daily record after the storing and resolving;
Intermediate process module is used for the daily record of log saving module is carried out intermediate treatment and regularly produced and preserve intermediate result;
The on-line analytical processing module, the intermediate result that utilization is pre-existing in is handled and is shown.
2. on-line analysing processing system as claimed in claim 1 is characterized in that, log collection module, intermediate process module and the concurrent working of on-line analytical processing module.
3. on-line analysing processing system as claimed in claim 1, it is characterized in that, log saving module comprises the log sheet of preserving daily record and the dictionary table of dimension data, and dictionary table is corresponding with the external key in the log sheet by the major key of dictionary table with log sheet, intermediate process module is every carries out polymerization once through one section preset time to log sheet, dictionary table, usually gathers to merge according to the low-dimensional level in each dimension and preserves intermediate result.
4. an on-line analytical processing method is characterized in that, this method may further comprise the steps:
Log collection and analyzing step are used for the dynamic collection daily record and will collect daily record resolving, corresponding dimension data;
Step is preserved in daily record, is used for preserving resolving the back daily record, usually daily record is kept in the log sheet, and dimension data is kept in the dictionary table;
Regularly generate the intermediate result step, whenever log sheet, dictionary table are carried out according to the dimension polymerization once, and preserve intermediate result in middle table through one section preset time;
The on-line analytical processing step obtains intermediate result in the intermediate list, and handles accordingly.
5. on-line analytical processing method as claimed in claim 4 is characterized in that, regularly generates the intermediate result step and carries out regular polymerization according to the low-dimensional level of dimension and produce intermediate result, further comprises the steps:
41) as calculated whether the intermediate result of judgement between start time and concluding time? if the intermediate result between start time and concluding time as calculated a part change step 42), otherwise change step 43);
42) deletion existing intermediate result between start time and concluding time;
43) first minimum time section after the acquisition start time;
44) set that each statistics dimension in this minimum time section is formed utilizes log sheet to calculate the statistics of distribution and the parameter of algebraically statistics, obtains intermediate result, and it is saved in the corresponding middle table;
45) judge whether to reach the concluding time, if arrive, process ends then; If no, then enter step 46);
46) obtain the next minimum time section of time period, and jump to step 44) calculate, finish until flow process.
6. as claim 4,5 described on-line analytical processing methods, it is characterized in that the on-line analytical processing step further comprises the steps:
50) determine the on-line analytical processing condition, the condition of the definite scope of input and dimension, dimension hierarchical level information;
51) calculate the result of the dimension level of selected dimension by the intermediate result in the middle table, and show;
52) judge whether to finish on-line analytical processing? if decision finishes on-line analytical processing, then finish, otherwise change step 53);
53) select one of this dimension level to tie up the member, excite its corresponding link, indicate it is descended to bore;
Is not 54) judge arriving low-dimensional level? if not low-dimensional level, then can on this dimension, proceed to bore down, change step 55); Otherwise change step 56);
55) will tie up level adjusts to down one deck dimension level and will tie up one deck that level adjusts to below it and open commentaries on classics step 58);
56) judge whether to specify other dimension in this dimension back? if specify other dimension in current dimension back, then can carry out the conversion of dimension, change step 57); Otherwise demonstration current results;
57) dimension of selecting appointment is as new current dimension, and with the ground floor dimension level of this dimension as current dimension level;
58) scope is adjusted in the scope of selecting this dimension member, determined new on-line analytical processing condition, and forward 51 to).
7. on-line analytical processing method as claimed in claim 6 is characterized in that, daily record is preserved step and further comprised the steps:
71) receive the daily record of an equipment of having resolved after, determine the log sheet that will preserve according to the content of daily record;
72) before concrete log content is saved in log sheet, collects dynamic dimension data and non-existent dynamic dimension data in the dictionary table is preserved;
73) for the daily record after resolving, deposit definite log sheet in;
74) judge whether to continue to handle daily record after the parsing, if continue then return step 71) preserve up to all daily records and finish, if do not continue then finish.
8. on-line analytical processing method as claimed in claim 7 is characterized in that step 72) further comprise the steps
721) whether also exist untreated dynamic dimension data if there is no then to finish in the daily record after judgement is resolved, if exist then execution in step 722);
722) obtain following first untreated dynamic dimension data;
723) will this untreated dynamic dimension data be kept at corresponding dictionary table in the dynamic dimension data collected compare; If had this untreated dynamic dimension data in the dictionary table, then returned step 721), if the dynamic dimension data that does not exist this daily record to comprise in the dictionary table, then implementation step 724);
724) should be saved in the dictionary table by dynamic dimension data, and return step 721 and preserve until finishing all dynamic dimension datas.
9. on-line analytical processing method as claimed in claim 4 is characterized in that, dictionary table is corresponding with the external key in the log sheet by the major key of dictionary table with log sheet.
10. on-line analytical processing method as claimed in claim 4 is characterized in that, log collection and analyzing step, regularly generates intermediate result step and the concurrent working of on-line analytical processing step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100002075A CN100383784C (en) | 2004-01-02 | 2004-01-02 | On-line analysing and treating system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100002075A CN100383784C (en) | 2004-01-02 | 2004-01-02 | On-line analysing and treating system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1641637A CN1641637A (en) | 2005-07-20 |
| CN100383784C true CN100383784C (en) | 2008-04-23 |
Family
ID=34866677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100002075A Expired - Fee Related CN100383784C (en) | 2004-01-02 | 2004-01-02 | On-line analysing and treating system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100383784C (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101192227B (en) * | 2006-11-30 | 2011-05-25 | 阿里巴巴集团控股有限公司 | Log file analytical method and system based on distributed type computing network |
| CN101018121B (en) * | 2007-03-15 | 2011-03-09 | 杭州华三通信技术有限公司 | Log convergence processing method and convergence processing device |
| KR101210258B1 (en) * | 2010-05-25 | 2012-12-10 | 주식회사 잉카인터넷 | method for displaying information about hack tool usage in online game |
| CN103177369A (en) * | 2011-12-20 | 2013-06-26 | 触动多媒体技术（上海）有限公司 | Remote analytic method for log information |
| CN105389352A (en) * | 2015-10-30 | 2016-03-09 | 北京奇艺世纪科技有限公司 | Log processing method and apparatus |
| CN107729500B (en) * | 2017-10-20 | 2021-01-05 | 锐捷网络股份有限公司 | Data processing method and device for online analysis processing and background equipment |
| CN108763573A (en) * | 2018-06-06 | 2018-11-06 | 众安信息技术服务有限公司 | A kind of OLAP engines method for routing and system based on machine learning |
| CN112738087A (en) * | 2020-12-29 | 2021-04-30 | 杭州迪普科技股份有限公司 | Attack log display method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5930785A (en) * | 1995-03-31 | 1999-07-27 | International Business Machines Corporation | Method for detecting and optimizing queries with encoding/decoding tables |
| WO2003083721A1 (en) * | 2002-04-03 | 2003-10-09 | Koninklijke Philips Electronics N.V. | Reshaped uddi for intranet use |
-
2004
- 2004-01-02 CN CNB2004100002075A patent/CN100383784C/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5930785A (en) * | 1995-03-31 | 1999-07-27 | International Business Machines Corporation | Method for detecting and optimizing queries with encoding/decoding tables |
| WO2003083721A1 (en) * | 2002-04-03 | 2003-10-09 | Koninklijke Philips Electronics N.V. | Reshaped uddi for intranet use |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1641637A (en) | 2005-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100518076C (en) | Journal accounting method and system | |
| Cao et al. | Efficient top-k query calculation in distributed networks | |
| US20180129579A1 (en) | Systems and Methods with a Realtime Log Analysis Framework | |
| CN104618343B (en) | A kind of method and system of the website threat detection based on real-time logs | |
| CN101582817B (en) | Method for extracting network interactive behavioral pattern and analyzing similarity | |
| US7921459B2 (en) | System and method for managing security events on a network | |
| CN103930887B (en) | The inquiry stored using raw column data collects generation | |
| CN106371986A (en) | Log treatment operation and maintenance monitoring system | |
| CN110691070B (en) | Network abnormity early warning method based on log analysis | |
| CN101335643B (en) | Method and apparatus for SDH equipment alarm correlativity analysis | |
| CN100383784C (en) | On-line analysing and treating system and method | |
| CN104917627A (en) | Log cluster scanning and analysis method used for large-scale server cluster | |
| CN113051147A (en) | Database cluster monitoring method, device, system and equipment | |
| CN114637989A (en) | APT attack tracing method and system based on distributed system and storage medium | |
| CN113660209B (en) | DDoS attack detection system based on sketch and federal learning and application | |
| CN101719847A (en) | High-performance monitoring method for DNS traffic | |
| CN106682206A (en) | Method and system for big data processing | |
| CN113849513A (en) | Data center operation and maintenance alarm information merging method and system | |
| CN115277113A (en) | Power grid network intrusion event detection and identification method based on ensemble learning | |
| CN107832333A (en) | Method and system based on distributed treatment and DPI data structure user network data fingerprint | |
| CN114116872A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN112288317B (en) | Industrial big data analysis platform and method based on multi-source heterogeneous data governance | |
| CN113645238A (en) | DDoS defense method oriented to Handle identification system | |
| Dolberg et al. | Efficient multidimensional aggregation for large scale monitoring | |
| KR101484186B1 (en) | Indexing apparatus and method for search of security monitoring data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080423 Termination date: 20210102 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |