CN117955883A - Detection method and system for identifying periodical surge of data item frequency - Google Patents
Detection method and system for identifying periodical surge of data item frequency Download PDFInfo
- Publication number
- CN117955883A CN117955883A CN202410352596.5A CN202410352596A CN117955883A CN 117955883 A CN117955883 A CN 117955883A CN 202410352596 A CN202410352596 A CN 202410352596A CN 117955883 A CN117955883 A CN 117955883A
- Authority
- CN
- China
- Prior art keywords
- field value
- field
- result
- value
- array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 81
- 230000000737 periodic effect Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims description 21
- 238000003860 storage Methods 0.000 claims description 18
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000003491 array Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 7
- 230000035939 shock Effects 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 13
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application relates to the field of network data analysis, in particular to a detection method and a detection system for identifying the periodic surge of a data item frequency, which are used for detecting the Burst item condition of an adjacent time window by utilizing a socket array to obtain the time interval of two adjacent Burst items, and carrying out hash operation on a stream tag and the time interval by utilizing a Top-k structure to obtain the periodic surge item of Top-k so as to realize rapid and accurate periodic Burst item detection under the condition of limited memory.
Description
Technical Field
The invention relates to the field of network flow measurement, in particular to a detection method and a detection system for identifying periodic rapid increase of frequency of a data item.
Background
Network traffic measurement refers to measuring the frequency, base, HEAVY HITTER, entropy, etc. of network flows to find potential abnormal patterns, and provide important reference information for network management and network security. Burst is a new class of data stream patterns, which refers to the situation where the stream suddenly increases in adjacent time windows. When the traffic increase value is greater than a certain threshold, we consider this as a burst, and an abnormal increase in traffic generally means the occurrence of an abnormal event or potential attack. Periodicity (Periodicity) is an important feature of the data stream, meaning that data items arrive at fixed time intervals, and is of great importance for predicting the behavior pattern of future data streams.
Traditional flow measurement techniques mainly focus on measuring the cardinal number and size characteristics of a data stream, and detect abnormal flow by storing flow information using a compact data structure. Many research efforts have emerged in this area, such as ELASTIC SKETCH, PYRAMID SKETCH, SPREADSKETCH, etc., in addition to which more recent research has tended to explore new patterns of data flow, including burstiness, periodicity, and time progression. BurstSketch is focused on researching the burstiness mode of the data flow, and proposes a data flow mode characterized by sudden increase and sudden drop, and potential burstiness projects are monitored and captured in real time through a snapshot technology. PeriodicSketch is directed to studying the periodic pattern of a data stream, aiming at identifying data items that occur at fixed time intervals and reporting Top-k periodic items. STAIRSKETCH is focused on a time progressive study, based on the observation that "recent data items have greater value", recent data items are memorized using a mechanism like a sliding window.
However, the above-described studies have focused on only one of the two data stream modes, either bursty or periodic, and have not focused on the case of burstiness with periodic modes, i.e., periodic bursts of data item frequencies. Furthermore, combining the two data stream patterns will create a more valuable pattern, effectively identifying such traffic as it appears periodically during a time window when an abnormal activity that suddenly increases in traffic has an important role in predicting the time of occurrence of the future abnormal activity. For example: it can be applied to the detection of regular advanced persistent threats (ADVANCED PERSISTENT THREATS, APT) (APT is a hidden and persistent means of attack often with regular features). Thus, combining burstiness with periodicity helps to mine potential periodic anomaly patterns, providing a reference for network management and security.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is to overcome the situation that the research in the prior art only focuses on one of two data stream modes, namely burstiness or periodicity, and does not concentrate the two modes in the burstiness with the periodic mode, namely, the periodic sudden increase of the frequency of the data item.
To solve the above technical problem, a first aspect of the present invention provides a detection method for identifying periodic surge in frequency of a data item, the method comprising:
Acquiring a data packet; the data packet comprises a flow label;
Carrying out hash calculation on the stream tag, and mapping the stream tag to a storage position of a socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
Traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, and generating a first judging result so as to generate a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
Performing Burst item detection on the socket array according to the Cur field value and the Pre field value to generate a Burst item detection result;
generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result;
storing the flow label and the time interval to a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
Calculating the splicing result of the stream label and the time interval according to a hash function, and mapping the splicing result to a storage position of a second unit array;
Judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, and generating a second judging result so as to generate a second ID field value, an Interval field value and a Frequency field value according to the second judging result;
Traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
In one embodiment of the present invention, the step of generating the first ID field value, the Pre field value, the Cur field value, the TimePre field value, and the TimeCur field value according to the first determination result includes:
And if the first judging result is that the first ID field is consistent with the flow label, acquiring a first unit array corresponding to the first ID field, and adding one to the Cur field value in the first unit array, wherein the TimeCur field value is set as a current time window.
In one embodiment of the present invention, the step of generating the first ID field value, the Pre field value, the Cur field value, the TimePre field value, and the TimeCur field value according to the first determination result further includes:
If the first judging result is that the first ID field is inconsistent with the flow label, judging whether an empty first unit array exists in the socket array or not;
If yes, inserting the flow label into an empty first unit array, setting a Pre field value in the first unit array to be 0, setting a Cur field value to be 1, setting a TimePre field value to be 0, and setting a TimeCur field value to be a current time window;
if not, screening a first unit array with the minimum Cur field value;
And replacing a first ID field value in the first unit array through the flow label, setting the Pre field value to 0, the Cur field value to 1, the TimePre field value to 0 and the TimeCur field value to the current time window.
In one embodiment of the present invention, the step of performing Burst item detection on the Burst array according to the Cur field value and the Pre field value, and generating a Burst item detection result includes:
judging whether the quotient of the Cur field value divided by the Pre field value is larger than a preset threshold value or not, and whether the Cur field value and the Pre field value are both larger than 0 or not;
If yes, the detection result of the Burst item is that the detection is passed;
if not, the Burst item detection result is that the detection is not passed.
In one embodiment of the present invention, the step of generating the time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result includes:
if the Burst item detection result is that the detection passes, judging whether the TimePre field value is 0;
If not, calculating the difference value between the TimeCur field value and the TimePre field value, generating the time interval, and setting the TimePre field value as a current time window;
If so, the time interval is 0.
In one embodiment of the present invention, the step after the time interval for generating the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result includes:
Exchanging the Cur field value in the Cur field with the Pre field value in the Pre field, and clearing the Cur field, wherein the TimeCur field value is the current time window.
In one embodiment of the present invention, the step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result includes:
And if the second judging result is that the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, acquiring a second unit array corresponding to the second ID field, and adding one to the Frequency field value in the second unit array.
In one embodiment of the present invention, the step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result further includes:
If the second judging result is that the combined result of the flow label and the time Interval is inconsistent with the combined result of the second ID field and the Interval field, judging whether an empty second unit array exists;
If yes, selecting the second unit array which is empty, setting a second ID field value as the flow label, setting an Interval field value as the time Interval, and setting a Frequency field value as 1;
If not, the second ID field value is set as the flow label, the Interval field value is set as the time Interval, and the Frequency field value is set as 1.
A second aspect of the present invention provides a detection system for identifying periodic bursts of data item frequency, the system comprising: the device comprises a first calculation module, a judgment module, a detection module and a second calculation module;
The computing module is configured to: acquiring a data packet; the data packet comprises a flow label; carrying out hash calculation on the stream tag to generate a storage position of the mapped socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
The judgment module is configured to: traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, generating a first judging result, and generating a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
The detection module is configured to: performing Burst item detection on the socket array according to the Cur field value and the Pre field value to generate a Burst item detection result; generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result; storing the flow label and the time interval to a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
The second computing module is configured to: calculating the combined result of the flow label and the time interval according to a hash function to generate a mapped storage position of a second unit array; judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, generating a second judging result, and generating a second ID field value, an Interval field value and a Frequency field value according to the second judging result; traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
Compared with the prior art, the technical scheme of the invention has the following advantages:
According to the detection method and the system for identifying the periodic surge of the data item frequency, the Burst item condition of the adjacent time window is detected by utilizing the socket array, the time interval between two adjacent Burst items is obtained, and the periodic surge item of the Top-k is obtained by utilizing the Top-k structure to hash the flow label and the time interval, so that the rapid and accurate periodic Burst item detection is realized under the condition of limited memory.
Drawings
In order that the invention may be more readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof that are illustrated in the appended drawings.
FIG. 1 is a flow chart of a method and system for detecting periodic bursts of identification data item frequency provided by the invention;
FIG. 2 is a flow chart of another embodiment of a method and system for detecting periodic bursts of identification data item frequency provided by the invention;
fig. 3 is a diagram of a method and a system for detecting periodic surge in frequency of identification data items according to the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and specific examples, which are not intended to be limiting, so that those skilled in the art will better understand the invention and practice it.
In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art based on embodiments of the application without making any inventive effort, fall within the scope of the application.
Referring to fig. 1 and 2, the present invention provides a detection method for identifying periodic surge in frequency of data items, the method comprising:
s100, acquiring a data packet; the data packet comprises a flow label;
in step S100, the acquired data packet is uniquely identified by a flow label, which selects a source IP address in the present application.
S200, carrying out hash calculation on the stream tag, and mapping the stream tag to a storage position of a socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
In step S200, the first portion of socket array structure is composed of w Buckets arrays, each socket array is composed of d first cell arrays (cells), and each first cell array (cell) stores 5 fields, which are respectively: a first ID field, a Pre field, a Cur field, timePre field, timeCur field. Wherein the first ID field stores a flow label for the item (the flow label may be a source IP address, a destination IP address, a source port number, a destination port number, a protocol, and combinations thereof, the present application taking the source IP address as an example). The Pre field stores the item frequency count of the last time window record, the Cur field stores the frequency count of the current time window, the TimePre field stores the time window in which the Burst item occurred before, and the TimeCur field stores the current time window. Wherein the Burst item is defined as: the current time window item frequency exceeds k times the frequency of the last time window item (k is taken to be 2 in the present application). And carrying out hash operation on the stream label by using a hash function in the socket array, and positioning the storage position of the stream.
When the measurement period starts, initializing the socket array, and setting all fields in the socket array to 0. When a stream with a stream tag f 1 arrives (assuming that f 1 is the source IP address of the data packet, different source IP addresses form different data items), the Bucket array maps the stream tag f 1 to one of the row of Bucket arrays through a hash function, and the calculation formula of the position mapped to the ith Bucket array is as follows:
（1）；
Wherein, H can be any hash function with good randomness, f 1 is a flow label, and w is the number of columns of the socket array. To the ith Bucket array mapped to.
S300, traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, and generating a first judging result so as to generate a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
In step S300, the step of generating the first ID field value, the Pre field value, the Cur field value, the TimePre field value, and the TimeCur field value according to the first determination result includes: and if the first judging result is that the first ID field is consistent with the flow label, acquiring a first unit array corresponding to the first ID field, and adding one to the Cur field value in the first unit array, wherein the TimeCur field value is set as a current time window.
The step of generating the first ID field value, the Pre field value, the Cur field value, the TimePre field value, and the TimeCur field value according to the first determination result further includes: if the first judging result is that the first ID field is inconsistent with the flow label, judging whether an empty first unit array exists in the socket array or not; if yes, inserting the flow label into an empty first unit array, setting a Pre field value in the first unit array to be 0, setting a Cur field value to be 1, setting a TimePre field value to be 0, and setting a TimeCur field value to be a current time window; if not, screening a first unit array with the minimum Cur field value; and replacing a first ID field value in the first unit array through the flow label, setting the Pre field value to 0, the Cur field value to 1, the TimePre field value to 0 and the TimeCur field value to the current time window.
In an actual application scene, traversing the mapped array of the socket array, judging whether a first ID field stored in the mapped array of the socket array is consistent with a flow label f 1, if the first ID field is consistent with the flow label f 1, finding a first cell array (cell) corresponding to the first ID field, adding one to the Cur field value in the first cell array (cell), setting the TimeCur field value as a current time window, and keeping other field values unchanged.
If the first ID field is inconsistent with the flow tag f 1, at this time, it needs to be determined whether there is an empty first unit array in the socket array, if there is an empty first unit array, then the flow tag f 1 is directly inserted into the empty first unit array that is first matched, that is, the flow tag f 1 is inserted into the first ID field of the first unit array, and meanwhile, the Pre field value is set to 0, the Cur field value is set to 1, the TimePre field value is set to 0, and the TimeCur field value is set to the current time window. If there is no empty first cell array, then item replacement occurs, and the replacement policy adopted is to directly replace the first cell array (cell) with the smallest count value, that is, screen all the first cell arrays (cells) currently, select the first cell array (cell) with the smallest Cur field value for replacement, replace the first ID field value of the first cell array (cell) with the flow label f 1, and set the Pre field to 0, the Cur field to 1, the timepre field to 0, and the timecur to the current time window.
S400, performing Burst item detection on the Burst array according to the Cur field value and the Pre field value to generate a Burst item detection result;
In step S400, the step of performing Burst item detection on the Burst array according to the Cur field value and the Pre field value, and generating a Burst item detection result includes: judging whether the quotient of the Cur field value divided by the Pre field value is larger than a preset threshold value or not, and whether the Cur field value and the Pre field value are both larger than 0 or not; if yes, the detection result of the Burst item is that the detection is passed; if not, the Burst item detection result is that the detection is not passed.
In an actual application scene, the Burst array is detected by the Cur field value and the Pre field value at the end of the time window, and the detection process is as follows: traversing a mapped-to socket arrayChecking whether the quotient of the Cur field value divided by the Pre field value is greater than a preset threshold, wherein the preset threshold is 2 in the application, if the quotient of the Cur field value divided by the Pre field value is greater than the preset threshold, the item is considered as a Burst item, the detection result of the Burst item is detection passing, and if the quotient of the Cur field value divided by the Pre field value is not greater than the preset threshold, the item is not considered as the Burst item, and the detection result of the Burst item is detection failing. In addition to the threshold constraint, the condition that both the Pre field and the Cur field are greater than zero needs to be satisfied because a case occurs: a stream occurs in the first time window but does not occur in the second time window, where the Cur field is zero and the detect Burst array operation is not performed. When the time window is 1, since this is the first time window, only the Cur field in the Bucket array is set to a value, and the Pre field is set to a value of 0, and the operation of detecting the Burst item is not performed at this time.
S500, generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result;
In step S500, the step of generating the time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result includes: if the Burst item detection result is that the detection passes, judging whether the TimePre field value is 0; if not, calculating the difference value between the TimeCur field value and the TimePre field value, generating the time interval, and setting the TimePre field value as a current time window; if so, the time interval is 0, and the stream tag of the first unit array and the calculated time interval are sent to the second part.
In an actual application scenario, a field value TimePre is stored in a fourth field of the socket array, a field value TimeCur is stored in a fifth field, if the detection result of the Burst item is that the detection is passed, whether the field value TimePre is 0 needs to be judged, if the field value TimePre is not 0, the field value TimePre is subtracted by the field value TimeCur to obtain a time interval of occurrence of the Burst item, and if the field value TimePre is 0, it is indicated that the time interval is the time interval of occurrence of the Burst item for the first time, so that there is no time interval, that is, calculation of the time interval is not needed, and the time interval is 0.
S501, exchanging the Cur field value in the Cur field with the Pre field value in the Pre field, and clearing the Cur field, wherein the TimeCur field value is the current time window.
In step S501, the step after generating the time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result includes: at the end of the time window, a cleaning operation is required, and at this time, the Cur field value in the socket array needs to be exchanged to the Pre field value. Meanwhile, the Cur field value is cleared, the TimePre field value is kept unchanged, the TimeCur field value is set to be a new time window, and the next window data stream processing is started.
When a new time window is opened, if a new stream arrives at this time, a hash operation is performed on the stream label, the updating process is consistent with the steps, the Burst item is detected at the end of the time window, and since the count value of the last time window is stored in the Pre field value, the quotient of dividing the Cur field value by the Pre field value is compared with a preset threshold value to detect the Burst item, and meanwhile, the time interval at which the Burst item occurs is calculated and transmitted to the second part, and finally, the operation of cleaning the data structure is performed.
S600, storing the flow label and the time interval into a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
In step S600, the Top-k structure of the second portion may be regarded as a hash table, and is composed of m second element arrays, where each second element array stores three fields, respectively: the device comprises a second ID field, an Interval field and a Frequency field, wherein the second ID field is a flow label corresponding to an item, the Interval field is a time Interval for Burst, and the Frequency field is the number of times of occurrence of the time Interval. Initially, all fields in the second cell array are initialized to 0.
S700, calculating the combination result of the flow label and the time interval according to a hash function, and mapping the combination result to a storage position of a second unit array;
In step S700, when the first portion passes through the incoming stream tag f 1 and the corresponding Burst occurrence time interval, the stream tag and time interval are combined as a key, using a hash function And carrying out hash calculation on the key, and mapping the key to a storage position of the second cell array, namely finding the corresponding second cell array to store corresponding information. The calculation formula of the storage location mapped to the second cell array is as follows:
（2）；
Wherein, Can be any hash function with good randomness, m is the length of the Top-k structure,Is the ith second cell array position of the mapped Top-k structure.
S800, judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, and generating a second judging result so as to generate a second ID field value, an Interval field value and a Frequency field value according to the second judging result;
In step S800, the step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result includes: and if the second judging result is that the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, acquiring a second unit array corresponding to the second ID field, and adding one to the Frequency field value in the second unit array. The step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result further includes: if the second judging result is that the combined result of the flow label and the time Interval is inconsistent with the combined result of the second ID field and the Interval field, judging whether an empty second unit array exists; if yes, selecting the second unit array which is empty, setting a second ID field value as the flow label, setting an Interval field value as the time Interval, and setting a Frequency field value as 1; if not, the second ID field value is set as the flow label, the Interval field value is set as the time Interval, and the Frequency field value is set as 1.
In an actual application scene, judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, if the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, finding a second unit array corresponding to the second ID field, and adding one to the Frequency field value in the second unit array, wherein other field values are unchanged. If the combination result of the flow label and the time Interval is inconsistent with the combination result of the second ID field and the Interval field, at this time, it needs to be judged whether an empty second unit array exists in the hash table, if the empty second unit array exists, the empty second unit array is found, the second ID field value in the empty second unit array is set as the flow label f 1, the Interval field value is set as the corresponding time Interval, and the Frequency field value is set as 1. If there is no empty second unit array, then a replacement occurs, and the replacement policy adopts a direct replacement mode, sets the second ID field value to be the flow label f 1, sets the Interval field value to be the corresponding time Interval, and sets the Frequency field value to be 1.
S900, traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
In step S900, after all the flow labels are inserted, the Top-k structure records the condition of the periodic items, traverses all the second element arrays, reports the Top-k periodic items and the time intervals of occurrence thereof, i.e. returns the (ID, interval) element with the maximum frequency of Top-k, which can reflect the occurrence of the most periodic flow, and can obtain which Burst items are periodically occurring by using the information.
In a second aspect, and with reference to fig. 3, the present application provides a detection system for identifying periodic bursts of data item frequency, the system comprising: the device comprises a first computing module 100, a judging module 200, a detecting module 300 and a second computing module 400;
The first computing module 100 is configured to: acquiring a data packet; the data packet comprises a flow label; carrying out hash calculation on the stream tag to generate a storage position of the mapped socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
The judging module 200 is configured to: traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, generating a first judging result, and generating a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
The detection module 300 is configured to: performing Burst item detection on the socket array according to the Cur field value and the Pre field value to generate a Burst item detection result; generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result; storing the flow label and the time interval to a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
The second computing module 400 is configured to: calculating the combined result of the flow label and the time interval according to a hash function to generate a mapped storage position of a second unit array; judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, generating a second judging result, and generating a second ID field value, an Interval field value and a Frequency field value according to the second judging result; traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
The effects of the above system when the above method is applied may be referred to the description in the foregoing method embodiment, and will not be repeated here.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations and modifications of the present invention will be apparent to those of ordinary skill in the art in light of the foregoing description. It is not necessary here nor is it exhaustive of all embodiments. And obvious variations or modifications thereof are contemplated as falling within the scope of the present invention.
Claims (9)
1. A method of detecting a periodic surge in frequency of a data item, the method comprising:
Acquiring a data packet; the data packet comprises a flow label;
Carrying out hash calculation on the stream tag, and mapping the stream tag to a storage position of a socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
Traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, and generating a first judging result so as to generate a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
Performing Burst item detection on the socket array according to the Cur field value and the Pre field value to generate a Burst item detection result;
generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result;
storing the flow label and the time interval to a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
calculating the combined result of the stream tag and the time interval according to a hash function, and mapping the combined result to a storage position of a second unit array;
Judging whether the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, and generating a second judging result so as to generate a second ID field value, an Interval field value and a Frequency field value according to the second judging result;
Traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
2. The method of claim 1, wherein the step of generating a first ID field value, a Pre field value, a Cur field value, a TimePre field value, and a TimeCur field value according to the first determination result comprises:
And if the first judging result is that the first ID field is consistent with the flow label, acquiring a first unit array corresponding to the first ID field, and adding one to the Cur field value in the first unit array, wherein the TimeCur field value is set as a current time window.
3. The method of claim 1, wherein the step of generating a first ID field value, a Pre field value, a Cur field value, a TimePre field value, and a TimeCur field value according to the first determination result further comprises:
If the first judging result is that the first ID field is inconsistent with the flow label, judging whether an empty first unit array exists in the socket array or not;
If yes, inserting the flow label into an empty first unit array, setting a Pre field value in the first unit array to be 0, setting a Cur field value to be 1, setting a TimePre field value to be 0, and setting a TimeCur field value to be a current time window;
if not, screening a first unit array with the minimum Cur field value;
And replacing a first ID field value in the first unit array through the flow label, setting the Pre field value to 0, the Cur field value to 1, the TimePre field value to 0 and the TimeCur field value to the current time window.
4. The method for detecting the periodic surge of the frequency of the identification data item according to claim 3, wherein the step of performing Burst item detection on the Burst array according to the Cur field value and the Pre field value and generating a Burst item detection result comprises the steps of:
Judging whether the quotient of the Cur field value divided by the Pre field value is larger than a preset threshold value or not, and whether the Cur field value and the Pre field value are both larger than 0 or not;
If yes, the detection result of the Burst item is that the detection is passed;
if not, the Burst item detection result is that the detection is not passed.
5. The method for detecting periodic bursts of frequency of data items as in claim 4, wherein said step of generating a Burst item time interval based on said TimePre field values and said TimeCur field values in said Burst item detection result comprises:
if the Burst item detection result is that the detection passes, judging whether the TimePre field value is 0;
If not, calculating the difference value between the TimeCur field value and the TimePre field value, generating the time interval, and setting the TimePre field value as a current time window;
If so, the time interval is 0.
6. The method for detecting the periodic surge of the frequency of the identified data item according to claim 1, wherein the step after the time interval of generating the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result comprises:
Exchanging the Cur field value in the Cur field with the Pre field value in the Pre field, and clearing the Cur field, wherein the TimeCur field value is the current time window.
7. The method of claim 1, wherein the step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result comprises:
And if the second judging result is that the combined result of the flow label and the time Interval is consistent with the combined result of the second ID field and the Interval field, acquiring a second unit array corresponding to the second ID field, and adding one to the Frequency field value in the second unit array.
8. The method of claim 1, wherein the step of generating the second ID field value, the Interval field value, and the Frequency field value according to the second determination result further comprises:
If the second judging result is that the combined result of the flow label and the time Interval is inconsistent with the combined result of the second ID field and the Interval field, judging whether an empty second unit array exists;
If yes, selecting the second unit array which is empty, setting a second ID field value as the flow label, setting an Interval field value as the time Interval, and setting a Frequency field value as 1;
If not, the second ID field value is set as the flow label, the Interval field value is set as the time Interval, and the Frequency field value is set as 1.
9. A detection system for identifying periodic bursts of data items, the system comprising: the device comprises a first calculation module, a judgment module, a detection module and a second calculation module;
The computing module is configured to: acquiring a data packet; the data packet comprises a flow label; carrying out hash calculation on the stream tag to generate a storage position of the mapped socket array; the socket array comprises a plurality of first unit arrays; the first element array comprises a first ID field, a Pre field, a Cur field, a TimePre field and a TimeCur field;
The judgment module is configured to: traversing the mapped socket array, judging whether the first ID field is consistent with the flow label, generating a first judging result, and generating a first ID field value, a Pre field value, a Cur field value, timePre field values and TimeCur field values according to the first judging result;
The detection module is configured to: performing Burst item detection on the socket array according to the Cur field value and the Pre field value to generate a Burst item detection result; generating a time interval of the Burst item according to the TimePre field value and the TimeCur field value in the Burst item detection result; storing the flow label and the time interval to a hash table; the hash table comprises a second array of cells; the second unit array comprises a second ID field, an Interval field and a Frequency field;
The second computing module is configured to: calculating the splicing result of the stream tag and the time interval according to a hash function to generate a mapped storage position of a second unit array; judging whether the flow label is consistent with the second ID field, generating a second judging result, and generating a second ID field value, an Interval field value and a Frequency field value according to the second judging result; traversing the hash table, generating a detection result according to the second ID field value and the Interval field value, and reporting a periodic shock item of the hash table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410352596.5A CN117955883B (en) | 2024-03-26 | 2024-03-26 | Detection method and system for identifying periodical surge of data item frequency |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410352596.5A CN117955883B (en) | 2024-03-26 | 2024-03-26 | Detection method and system for identifying periodical surge of data item frequency |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117955883A true CN117955883A (en) | 2024-04-30 |
| CN117955883B CN117955883B (en) | 2024-06-07 |
Family
ID=90798449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410352596.5A Active CN117955883B (en) | 2024-03-26 | 2024-03-26 | Detection method and system for identifying periodical surge of data item frequency |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117955883B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040064737A1 (en) * | 2000-06-19 | 2004-04-01 | Milliken Walter Clark | Hash-based systems and methods for detecting and preventing transmission of polymorphic network worms and viruses |
| CN105447430A (en) * | 2015-11-19 | 2016-03-30 | 中南大学 | Label and method for recognizing label through employing HARN protocol |
| CN113132180A (en) * | 2021-03-11 | 2021-07-16 | 武汉大学 | Cooperative type large flow detection method facing programmable network |
| CN117115582A (en) * | 2023-07-27 | 2023-11-24 | 南京行者易智能交通科技有限公司 | Target detection method combining reinforcement learning of human feedback |
-
2024
- 2024-03-26 CN CN202410352596.5A patent/CN117955883B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040064737A1 (en) * | 2000-06-19 | 2004-04-01 | Milliken Walter Clark | Hash-based systems and methods for detecting and preventing transmission of polymorphic network worms and viruses |
| CN105447430A (en) * | 2015-11-19 | 2016-03-30 | 中南大学 | Label and method for recognizing label through employing HARN protocol |
| CN113132180A (en) * | 2021-03-11 | 2021-07-16 | 武汉大学 | Cooperative type large flow detection method facing programmable network |
| CN117115582A (en) * | 2023-07-27 | 2023-11-24 | 南京行者易智能交通科技有限公司 | Target detection method combining reinforcement learning of human feedback |
Non-Patent Citations (1)
| Title |
|---|
| 赵月爱;彭新光;: "异或和取模运算的负载均衡算法", 计算机工程与设计, no. 06, 28 March 2007 (2007-03-28) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117955883B (en) | 2024-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108494810B (en) | Attack-oriented network security situation prediction method, device and system | |
| US7596810B2 (en) | Apparatus and method of detecting network attack situation | |
| CN111259204B (en) | APT detection correlation analysis method based on graph algorithm | |
| US8191149B2 (en) | System and method for predicting cyber threat | |
| US7352280B1 (en) | System and method for intruder tracking using advanced correlation in a network security system | |
| CN112422537B (en) | Behavior prediction method of network attack knowledge graph generated based on honeypot actual combat | |
| Tadda et al. | Realizing situation awareness within a cyber environment | |
| CN112544059A (en) | Method, device and system for network traffic analysis | |
| Khosravi et al. | Alerts correlation and causal analysis for APT based cyber attack detection | |
| US7376090B2 (en) | Method of detecting distributed denial of service based on grey theory | |
| Gaikwad et al. | Anomaly based intrusion detection system using artificial neural network and fuzzy clustering | |
| CN110191004A (en) | A kind of port detecting method and system | |
| CN114679327B (en) | Network attack level determination method, device, computer equipment and storage medium | |
| CN116471124A (en) | Computer network safety prediction system for analyzing based on big data information | |
| Huang et al. | Network forensic analysis using growing hierarchical SOM | |
| Ourston et al. | Coordinated internet attacks: responding to attack complexity | |
| CN117955883B (en) | Detection method and system for identifying periodical surge of data item frequency | |
| Gyanchandani et al. | Intrusion detection using C4. 5: performance enhancement by classifier combination | |
| CN114726623B (en) | Advanced threat attack assessment method and device, electronic equipment and storage medium | |
| Wang et al. | Virtual indexing based methods for estimating node connection degrees | |
| CN113709097A (en) | Network risk perception method and defense method | |
| Parra-Arnau et al. | Dataveillance and the False-Positive Paradox | |
| Li et al. | Discovering novel multistage attack strategies | |
| Parekh et al. | A Hybrid Approach to Protect Server from IP Spoofing Attack | |
| Ge et al. | On effective sampling techniques for host-based intrusion detection in MANET |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |