CN109600345A - Abnormal data flow rate testing methods and device - Google Patents
Abnormal data flow rate testing methods and device Download PDFInfo
- Publication number
- CN109600345A CN109600345A CN201710938412.3A CN201710938412A CN109600345A CN 109600345 A CN109600345 A CN 109600345A CN 201710938412 A CN201710938412 A CN 201710938412A CN 109600345 A CN109600345 A CN 109600345A
- Authority
- CN
- China
- Prior art keywords
- data flow
- abnormal data
- abnormal
- benchmark
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of abnormal data flow rate testing methods and devices.By being based on preset data flow benchmark, being monitored to abnormal data flow in the access data traffic in default monitoring time section in default monitoring time section；The abnormal data flow monitored is labeled according to abnormal data flow grade is preset；The abnormal data flow of mark is subjected to data characterization.Preset data flow benchmark is calculated by study in advance and sampling in the present invention and provides benchmark for the subsequent mark for carrying out abnormal data flow, in order to which access data traffic is monitored with preset data flow benchmark during marking abnormal data flow, data traffic will be accessed and preset data flow benchmark is compared automatically and identified abnormal data, and then complete the mark and data characterization to different grades of abnormal data flow.It realizes the complexity of reduction abnormal data flow detection, improves the detection efficiency of abnormal data flow and the purpose of recall rate.
Description
Technical field
The present invention relates to network technique fields, more specifically to a kind of abnormal data flow rate testing methods and device.
Background technique
With the fast development of internet, people obtain different information or resource by accessing various websites.Current
Website amount of access in, in addition to user access generate flow other than, it is some waste site resources abnormal data flows, such as
Crawler flow, advertisement cheating flow etc. also account for considerable proportion.
Currently, monitoring of the website for abnormal data flow, is still leading progress with artificial and relevant rule.
Wherein, the detection of Artificial Anomalies data traffic extremely relies on related personnel's professional knowledge, and inefficiency.And relevant rule is pair
The normalized by definition of artificial experience, rule is more and cumbersome, can only find few a part of abnormal data flow, and is very easy to lose
Effect.In addition, there is also significant controversials for a large amount of rules, rule application is caused to be difficult to promote.
Therefore, abnormal data flow detection complexity can be reduced by needing one kind at present, improve the inspection of abnormal data flow
Survey the scheme of efficiency and recall rate.
Summary of the invention
In view of this, reducing abnormal number this application provides a kind of abnormal data flow rate testing methods and device to realize
According to flow detection complexity, the detection efficiency of abnormal data flow and the purpose of recall rate are improved.
To achieve the goals above, it is proposed that scheme it is as follows:
First aspect present invention discloses a kind of abnormal data flow rate testing methods, comprising:
In default monitoring time section, it is based on the preset data flow benchmark, in the default monitoring time section
Abnormal data flow is monitored in access data traffic；
The abnormal data flow monitored is labeled according to abnormal data flow grade is preset；
The abnormal data flow of mark is subjected to data characterization.
Preferably, the default process of the preset data flow benchmark includes:
Acquire the access data traffic of Website front-end；
The abnormal data flow for including in the access data traffic is searched and rejected, normal data flow is obtained；
Data base is constructed using the normal data flow, obtains preset data flow benchmark.
Preferably, the access data traffic of the acquisition Website front-end, comprising:
The access data traffic from different web sites front end in the sampling time is acquired, the sampling time specifically determines with demand
Fixed, the Website front-end includes and is not limited to browser version, operating system version, province, city, screen color depth, flash editions
The general acquisition data in this grade front end.
Preferably, described to search and reject the abnormal data flow for including in the access data traffic, obtain normal number
According to flow, comprising:
The data distribution benchmark in the access data traffic, the data distribution benchmark are determined using data mining algorithm
It is used to indicate the data dimension distribution benchmark in the access data traffic from different front-end technology；
Outlier Detection Algorithm based on Gaussian Profile calculates the data distribution benchmark, determines abnormal data flow；
The determining abnormal data flow is rejected from the access data traffic, obtains normal data flow.
Preferably, the method also includes:
Generate the abnormal data flow of data characterization and the comparison diagram of the preset data flow benchmark.
Preferably, the comparison diagram includes curve control figure, and one in multidimensional data cube comparison diagram and column comparison diagram
Kind is a variety of.
Preferably, the foundation is preset abnormal data flow grade and is marked to the abnormal data flow monitored
Note；
Based on supervision algorithm to the abnormal data flow monitored, according to the differentiation pair of default abnormal data flow grade
The different grades of abnormal data flow is answered, the default abnormal data flow grade includes at least two-stage.
The abnormal data flow different grades of to correspondence is labeled, and the abnormal data flow of same grade uses phase
Same mark.
Second method of the present invention discloses a kind of abnormal data flow detector, comprising:
Monitoring unit, for the preset data flow benchmark being based on, to the default prison in default monitoring time section
Abnormal data flow is monitored in the access data traffic surveyed in the period；
Unit is marked, for marking according to default abnormal data flow grade to the abnormal data flow monitored
Note；
Characterization unit, the abnormal data flow for that will mark carry out data characterization.
Third aspect present invention discloses a kind of storage medium, and the storage medium includes the program of storage, wherein in institute
Equipment where controlling the storage medium when stating program operation executes the inspection of the abnormal data flow as disclosed in first aspect present invention
Survey method.
Fourth aspect present invention discloses a kind of processor, and the processor is for running program, wherein described program fortune
The abnormal data flow rate testing methods as disclosed in first aspect present invention are executed when row.
As can be seen from the above technical solutions, the present invention discloses a kind of abnormal data flow rate testing methods and device.By
In default monitoring time section, it is based on the preset data flow benchmark, to the access data flow in the default monitoring time section
Abnormal data flow is monitored in amount；According to preset abnormal data flow grade to the abnormal data flow monitored into
Rower note；The abnormal data flow of mark is subjected to data characterization.In the present invention by learning and sampling meter in advance
Calculation obtains preset data flow benchmark and provides benchmark for the subsequent mark for carrying out abnormal data flow, in order to mark abnormal number
Access data traffic is monitored with preset data flow benchmark during according to flow, i.e., will access data traffic and is preset
Data traffic benchmark is compared automatically and is identified abnormal data flow, and then is completed to different grades of abnormal data flow
Mark and data characterization.Realize reduction abnormal data flow detection complexity, improve abnormal data flow detection efficiency and
The purpose of recall rate.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow diagram of analysis of advertising results method disclosed by the embodiments of the present invention；
Fig. 2 is a kind of display side of element relevant to targeted advertisements material disclosed by the embodiments of the present invention in video
Formula；
Fig. 3 is a kind of detection flows mixed distribution result comprising abnormal flow disclosed by the embodiments of the present invention；
After Fig. 4 is the rejecting Fig. 3 data exception flow disclosed by the embodiments of the present invention realized by data mining algorithm, just
The distributed effect of normal flow；
Fig. 5 a is the comparison diagram of mile abnormality data disclosed by the embodiments of the present invention and preset data flow benchmark；
Fig. 5 b is the comparison diagram of moderate abnormality data disclosed by the embodiments of the present invention and preset data flow benchmark；
Fig. 5 c is the comparison diagram of severely subnormal data disclosed by the embodiments of the present invention and preset data flow benchmark；
Fig. 6 is a kind of structural schematic diagram of abnormal data flow detector disclosed by the embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
It can be seen from background technology that use artificial screening abnormal data flow, the artificial side for marking abnormal data flow at present
Formula is not only complicated cumbersome, and the detection efficiency and recall rate for abnormal data flow are also very low.Therefore, the invention discloses one kind
Abnormal flow data detection method improves the detection of abnormal data flow to realize the complexity of reduction abnormal data flow detection
The purpose of efficiency and recall rate.
As shown in Figure 1, be a kind of flow diagram of abnormal data flow rate testing methods disclosed by the embodiments of the present invention, it is main
Include:
Step S101: in default monitoring time section, it is based on the preset data flow benchmark, when to the default monitoring
Between in access data traffic in section abnormal data flow be monitored.
During executing step S101, the default process for the preset data flow benchmark being based on is as shown in Fig. 2, main
Include:
Step S201: the access data traffic of Website front-end is acquired.
Specifically execute step S201 during, the acquisition of related data flow mainly pass through front end JavaScript with
Background data base interaction is realized.
Optionally, the access data traffic from different web sites front end in the acquisition sampling time.The sampling time usual feelings
It is 24 hours under condition, but is not limited in 24 hours, can also be greater than 24 hours or less than 24 hours.Different websites are then
Including at least any two kinds in browser version, operating system version, province, city, screen color depth, flash version.Such as figure
It is a large amount of 24 hours distribution maps of separate sources data shown in 3.
Step S202: searching and rejects the abnormal data flow for including in the access data traffic, obtains normal data
Flow.
During specifically executing step S202, following process is specifically included:
Firstly, determining the data distribution benchmark in the access data traffic using data mining algorithm.Wherein, the data
Distribution benchmark is used to indicate the data dimension distribution benchmark in the access data traffic from different front-end technology.
That is, according to access data traffic separate sources, in different ways to corresponding data traffic into
Row parsing.Such as corresponding browser version, operating system version are parsed according to user_agent；By ip parse click come
Source city, province etc..Abnormal data flow is determined using related resolution field.
Secondly, the Outlier Detection Algorithm based on Gaussian Profile calculates the data distribution benchmark, abnormal data flow is determined.
Finally, the determining abnormal data flow is rejected from the access data traffic, normal flow is obtained
Amount.
Step S203: data base is constructed using the normal data flow, obtains preset data flow benchmark.
Because in collected access data traffic necessarily also including various noise flows.Therefore the present invention is based on Gausses
Abnormality detection (Anomaly Detection) algorithm of distribution picks various noise flows and relevant abnormalities data traffic
It removes.Its theory being based on are as follows: abnormal data flow it is always relatively fewer and dispersion, normal data flow flow it is always more and
The characteristics of aggregation, carries out.After being rejected abnormal data flow using Anomaly Detection algorithm, obtain as shown in Figure 4
Preset data flow benchmark.
Optionally, Anomaly Detection algorithm include and be not limited to simple average, clustering algorithm,
The tradition such as OneClassSVM, IsolationForest or emerging Anomaly Detection algorithm.Pass through benefit in the present invention
With any of the above-described kind of algorithm from a large amount of access data traffic, the dimension for obtaining related data flow is distributed benchmark.
Optionally, which is subjected to data characterization.
The data traffic of a large amount of separate sources is obtained by different front end websites in the present invention so that it is subsequent from
The determination that dynamicization data traffic is distributed benchmark and abnormal data flow not only includes single dimension index, also includes multiple dimensions
Index.
Step S102: the abnormal data flow monitored is labeled according to abnormal data flow grade is preset.
During specifically executing step S102, firstly, based on supervision algorithm to the abnormal data stream monitored
Amount distinguishes the corresponding different grades of abnormal data flow according to default abnormal data flow grade.Here default exception
Data traffic grade includes at least two-stage.
Then, the abnormal data flow different grades of to correspondence is labeled, the abnormal data flow of same grade
Using identical mark.Different grades of abnormal data flow can use different marks, for difference.
Optionally, three kinds of mile abnormality, moderate abnormality and severely subnormal grades can be divided by presetting abnormal data flow grade
Not.
Here supervision algorithm refers to the supervision algorithm in any machine learning.Including and be not limited to decision tree, random
Forest, GBDT, logistic regression, xgboost and neural network (deep learning) method.
Step S103: the abnormal data flow of mark is subjected to data characterization.
During specifically executing step S103, the abnormal data flow of mark is subjected to data characterization processing, it should
Data characterization refers to the general characteristic of target class data or summarizing for feature.The output of data characterization can use diversified forms
It provides, such as pie chart, item figure, curve, multi-dimensional data cube and the multi-dimensional table including crosstab.As a result describing can also be with
It is provided with broad sense relationship or rule (referred to as characterization rules) form.
Further, preset data flow benchmark is also subjected to data characterization, then generates the described of data characterization
The comparison diagram of abnormal data flow and the preset data flow benchmark.
The comparison diagram includes curve control figure, one of multidimensional data cube comparison diagram and column comparison diagram or a variety of.
For example, being followed successively by mile abnormality, moderate abnormality and severely subnormal and preset data as shown in Fig. 5 a, Fig. 5 b, Fig. 5 c
The comparison diagram of Flow datum.Fig. 5 a, Fig. 5 b, it is embodied respectively with spatial abnormal feature and normal distribution benchmark in Fig. 5 c.
The present invention discloses a kind of abnormal data flow rate testing methods and device.By being based in default monitoring time section
The preset data flow benchmark supervises abnormal data flow in the access data traffic in the default monitoring time section
Control；The abnormal data flow monitored is labeled according to abnormal data flow grade is preset；By the described different of mark
Regular data flow carries out data characterization.Preset data flow benchmark is calculated by study and sampling in advance in the present invention
For it is subsequent carry out abnormal data flow mark benchmark is provided, in order to during marking abnormal data flow by access number
It is monitored according to flow and preset data flow benchmark, i.e., will access data traffic and the progress of preset data flow benchmark is automatic right
Than and identify abnormal data flow, and then complete the mark and data characterization to different grades of abnormal data flow.It realizes
Reduce the complexity of abnormal data flow detection, improves the detection efficiency of abnormal data flow and the purpose of recall rate.
Based on abnormal data flow rate testing methods disclosed in the embodiments of the present invention, the embodiment of the present invention is also corresponding open
A kind of abnormal data flow detector, as shown in fig. 6, the abnormal data flow detector 600 specifically includes that
Monitoring unit 601, for the preset data flow benchmark being based on, to described pre- in default monitoring time section
If abnormal data flow is monitored in the access data traffic in monitoring time section.
Mark unit 602, for according to preset abnormal data flow grade to the abnormal data flow monitored into
Rower note.
Characterization unit 603, the abnormal data flow for that will mark carry out data characterization.
Further, in the abnormal data flow detector further include: default unit 604.The default unit 605 wraps
It includes:
Acquisition unit, for acquiring the access data traffic of Website front-end.
Culling unit obtains normal for searching and rejecting the abnormal data flow for including in the access data traffic
Data traffic.
Construction unit obtains preset data flow benchmark for constructing data base using the normal data flow.
It the specific principle of each unit in abnormal data flow detector disclosed in the embodiments of the present invention and holds
Row process, it is identical as abnormal data flow rate testing methods disclosed in the embodiments of the present invention, reference can be made to aforementioned present invention is implemented
Corresponding part in abnormal data flow rate testing methods disclosed in example, is not discussed here.
Based on abnormal data flow detector disclosed in the embodiments of the present invention, above-mentioned each unit can pass through one
The hardware device that kind is made of processor and memory is realized.Specifically: above-mentioned each unit and module are deposited as program unit
It is stored in memory, above procedure unit stored in memory is executed by processor to realize the inspection of abnormal data flow
It surveys.
Wherein, include kernel in processor, gone in memory to transfer corresponding program unit by kernel.Kernel can be set
One or more realizes the detection to abnormal data flow by adjusting kernel parameter.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, if read-only memory (ROM) or flash memory (flash RAM), memory include that at least one is deposited
Store up chip.
Further, the embodiment of the invention provides a kind of processors, and the processor is for running program, wherein institute
Abnormal data flow rate testing methods are executed when stating program operation.
Further, the embodiment of the invention provides a kind of equipment, equipment includes processor, memory and is stored in storage
On device and the program that can run on a processor, processor perform the steps of in default monitoring time section when executing program,
Based on the preset data flow benchmark, to abnormal data flow in the access data traffic in the default monitoring time section into
Row monitoring；The abnormal data flow monitored is labeled according to abnormal data flow grade is preset；By the institute of mark
It states abnormal data flow and carries out data characterization.
Wherein, the default process of the preset data flow benchmark includes: to acquire the access data traffic of Website front-end；It looks into
The abnormal data flow for including in the access data traffic is looked for and rejected, normal data flow is obtained；Utilize the normal number
Data base is constructed according to flow, obtains preset data flow benchmark.
Specifically, the access data traffic of the acquisition Website front-end, comprising: come from different web sites in the acquisition sampling time
The access data traffic of front end, the sampling time be more than or equal to 24 hours, the Website front-end include at least browser version,
Operating system version, province, city, screen color depth, flash version.
Specifically, described search and reject the abnormal data flow for including in the access data traffic, normal number is obtained
According to flow, comprising: determine the data distribution benchmark in the access data traffic, the data distribution using data mining algorithm
Benchmark is used to indicate the data dimension distribution benchmark in the access data traffic from different front-end technology；Based on Gaussian Profile
Outlier Detection Algorithm calculate the data distribution benchmark, determine abnormal data flow；By the determining abnormal data flow
It is rejected from the access data traffic, obtains normal data flow.
The abnormal data flow monitored is marked specifically, the foundation presets abnormal data flow grade
Note, including to the abnormal data flow monitored, being distinguished pair according to default abnormal data flow grade based on supervision algorithm
The different grades of abnormal data flow is answered, the default abnormal data flow grade includes at least two-stage.To corresponding different
The abnormal data flow of grade is labeled, and the abnormal data flow of same grade uses identical mark.
Further, further includes: generate the abnormal data flow and the preset data flow base of data characterization
Quasi- comparison diagram.Wherein, the comparison diagram includes curve control figure, and one in multidimensional data cube comparison diagram and column comparison diagram
Kind is a variety of.
Equipment disclosed in the embodiment of the present invention can be server, PC, PAD, mobile phone etc..
Further, the embodiment of the invention also provides a kind of storage medium, it is stored thereon with program, the program is processed
Abnormal data flow rate testing methods are realized when device executes.
Present invention also provides a kind of computer program products, when executing on data processing equipment, are adapted for carrying out just
The program of beginningization there are as below methods step: in default monitoring time section, it is based on the preset data flow benchmark, to described pre-
If abnormal data flow is monitored in the access data traffic in monitoring time section；According to default abnormal data flow grade pair
The abnormal data flow monitored is labeled；The abnormal data flow of mark is subjected to data characterization.
Wherein, the default process of the preset data flow benchmark includes: to acquire the access data traffic of Website front-end；It looks into
The abnormal data flow for including in the access data traffic is looked for and rejected, normal data flow is obtained；Utilize the normal number
Data base is constructed according to flow, obtains preset data flow benchmark.
Specifically, the access data traffic of the acquisition Website front-end, comprising: come from different web sites in the acquisition sampling time
The access data traffic of front end, the sampling time be more than or equal to 24 hours, the Website front-end include at least browser version,
Operating system version, province, city, screen color depth, flash version.
Specifically, described search and reject the abnormal data flow for including in the access data traffic, normal number is obtained
According to flow, comprising: determine the data distribution benchmark in the access data traffic, the data distribution using data mining algorithm
Benchmark is used to indicate the data dimension distribution benchmark in the access data traffic from different front-end technology；Based on Gaussian Profile
Outlier Detection Algorithm calculate the data distribution benchmark, determine abnormal data flow；By the determining abnormal data flow
It is rejected from the access data traffic, obtains normal data flow.
The abnormal data flow monitored is marked specifically, the foundation presets abnormal data flow grade
Note, including to the abnormal data flow monitored, being distinguished pair according to default abnormal data flow grade based on supervision algorithm
The different grades of abnormal data flow is answered, the default abnormal data flow grade includes at least two-stage.To corresponding different
The abnormal data flow of grade is labeled, and the abnormal data flow of same grade uses identical mark.
Further, further includes: generate the abnormal data flow and the preset data flow base of data characterization
Quasi- comparison diagram.Wherein, the comparison diagram includes curve control figure, and one in multidimensional data cube comparison diagram and column comparison diagram
Kind is a variety of.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable Jie
The example of matter.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including element
There is also other identical elements in process, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above is only embodiments herein, are not intended to limit this application.To those skilled in the art,
Various changes and changes are possible in this application.It is all within the spirit and principles of the present application made by any modification, equivalent replacement,
Improve etc., it should be included within the scope of the claims of this application.
Claims (10)
1. a kind of abnormal data flow rate testing methods characterized by comprising
In default monitoring time section, it is based on the preset data flow benchmark, to the access in the default monitoring time section
Abnormal data flow is monitored in data traffic；
The abnormal data flow monitored is labeled according to abnormal data flow grade is preset；
The abnormal data flow of mark is subjected to data characterization.
2. the method according to claim 1, wherein the default process of the preset data flow benchmark includes:
Acquire the access data traffic of Website front-end；
The abnormal data flow for including in the access data traffic is searched and rejected, normal data flow is obtained；
Data base is constructed using the normal data flow, obtains preset data flow benchmark.
3. according to the method described in claim 2, it is characterized in that, the access data traffic of the acquisition Website front-end, comprising:
The access data traffic from different web sites front end in the sampling time is acquired, the sampling time is determined by actual demand,
The Website front-end includes and is not limited to browser version, operating system version, province, city, screen color depth, flash version
The equal general acquisition data in front ends.
4. according to the method described in claim 3, it is characterized in that, the lookup and rejecting in the access data traffic and including
Abnormal data flow, obtain normal data flow, comprising:
Determine that the data distribution benchmark in the access data traffic, the data distribution benchmark are used for using data mining algorithm
Indicate the data dimension distribution benchmark in the access data traffic from different front-end technology；
Outlier Detection Algorithm based on Gaussian Profile calculates the data distribution benchmark, determines abnormal data flow；
The determining abnormal data flow is rejected from the access data traffic, obtains normal data flow.
5. the method according to claim 1, wherein further include:
Generate the abnormal data flow of data characterization and the comparison diagram of the preset data flow benchmark.
6. according to the method described in claim 5, multidimensional data is vertical it is characterized in that, the comparison diagram includes curve control figure
One of square comparison diagram and column comparison diagram are a variety of.
7. method according to claim 1 to 6, which is characterized in that the foundation presets abnormal data flow etc.
Grade is labeled the abnormal data flow monitored, comprising:
Based on supervision algorithm to the abnormal data flow monitored, distinguishes and corresponded to not according to default abnormal data flow grade
The abnormal data flow of ad eundem, the default abnormal data flow grade include at least two-stage；
The abnormal data flow different grades of to correspondence is labeled, and the abnormal data flow of same grade is using identical
Mark.
8. a kind of abnormal data flow detector characterized by comprising
Monitoring unit, for the preset data flow benchmark being based on, when to the default monitoring in default monitoring time section
Between in access data traffic in section abnormal data flow be monitored；
Unit is marked, for being labeled according to default abnormal data flow grade to the abnormal data flow monitored；
Characterization unit, the abnormal data flow for that will mark carry out data characterization.
9. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment execute such as abnormal data flow detection side of any of claims 1-7
Method.
10. a kind of processor, which is characterized in that the processor is for running program, wherein executed such as when described program is run
Abnormal data flow rate testing methods of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710938412.3A CN109600345A (en) | 2017-09-30 | 2017-09-30 | Abnormal data flow rate testing methods and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710938412.3A CN109600345A (en) | 2017-09-30 | 2017-09-30 | Abnormal data flow rate testing methods and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109600345A true CN109600345A (en) | 2019-04-09 |
Family
ID=65956151
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710938412.3A Pending CN109600345A (en) | 2017-09-30 | 2017-09-30 | Abnormal data flow rate testing methods and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109600345A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287322A (en) * | 2019-06-27 | 2019-09-27 | 有米科技股份有限公司 | Moisture flow processing method, system and the equipment of social media flow |
| CN110781494A (en) * | 2019-10-22 | 2020-02-11 | 武汉极意网络科技有限公司 | Data abnormity early warning method, device, equipment and storage medium |
| CN110995459A (en) * | 2019-10-12 | 2020-04-10 | 平安科技（深圳）有限公司 | Abnormal object identification method, device, medium and electronic equipment |
| CN111080651A (en) * | 2019-12-12 | 2020-04-28 | 西南科技大学 | Automatic monitoring method for petroleum drilling polluted gas based on water flow segmentation |
| CN111314869A (en) * | 2020-02-18 | 2020-06-19 | 中国联合网络通信集团有限公司 | Flow quota distribution method and device, electronic equipment and storage medium |
| CN111404835A (en) * | 2020-03-30 | 2020-07-10 | 北京海益同展信息科技有限公司 | Flow control method, device, equipment and storage medium |
| CN111835696A (en) * | 2019-04-23 | 2020-10-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting abnormal request individuals |
| CN112000538A (en) * | 2019-05-10 | 2020-11-27 | 百度在线网络技术（北京）有限公司 | Page content display monitoring method, device and equipment and readable storage medium |
| CN113343064A (en) * | 2021-06-18 | 2021-09-03 | 北京百度网讯科技有限公司 | Data processing method, device, equipment, storage medium and computer program product |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7814548B2 (en) * | 2005-09-13 | 2010-10-12 | Honeywell International Inc. | Instance based learning framework for effective behavior profiling and anomaly intrusion detection |
| CN103886068A (en) * | 2014-03-20 | 2014-06-25 | 北京国双科技有限公司 | Data processing method and device for Internet user behavior analysis |
| CN105071985A (en) * | 2015-07-24 | 2015-11-18 | 四川大学 | Server network behavior description method |
| CN105208037A (en) * | 2015-10-10 | 2015-12-30 | 中国人民解放军信息工程大学 | DoS/DDoS attack detecting and filtering method based on light-weight intrusion detection |
| CN105577679A (en) * | 2016-01-14 | 2016-05-11 | 华东师范大学 | Method for detecting anomaly traffic based on feature selection and density peak clustering |
| CN105847283A (en) * | 2016-05-13 | 2016-08-10 | 深圳市傲天科技股份有限公司 | Information entropy variance analysis-based abnormal traffic detection method |
| CN106060043A (en) * | 2016-05-31 | 2016-10-26 | 北京邮电大学 | Abnormal flow detection method and device |
| CN106713324A (en) * | 2016-12-28 | 2017-05-24 | 北京奇艺世纪科技有限公司 | Flow detection method and device |
-
2017
- 2017-09-30 CN CN201710938412.3A patent/CN109600345A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7814548B2 (en) * | 2005-09-13 | 2010-10-12 | Honeywell International Inc. | Instance based learning framework for effective behavior profiling and anomaly intrusion detection |
| CN103886068A (en) * | 2014-03-20 | 2014-06-25 | 北京国双科技有限公司 | Data processing method and device for Internet user behavior analysis |
| CN105071985A (en) * | 2015-07-24 | 2015-11-18 | 四川大学 | Server network behavior description method |
| CN105208037A (en) * | 2015-10-10 | 2015-12-30 | 中国人民解放军信息工程大学 | DoS/DDoS attack detecting and filtering method based on light-weight intrusion detection |
| CN105577679A (en) * | 2016-01-14 | 2016-05-11 | 华东师范大学 | Method for detecting anomaly traffic based on feature selection and density peak clustering |
| CN105847283A (en) * | 2016-05-13 | 2016-08-10 | 深圳市傲天科技股份有限公司 | Information entropy variance analysis-based abnormal traffic detection method |
| CN106060043A (en) * | 2016-05-31 | 2016-10-26 | 北京邮电大学 | Abnormal flow detection method and device |
| CN106713324A (en) * | 2016-12-28 | 2017-05-24 | 北京奇艺世纪科技有限公司 | Flow detection method and device |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111835696B (en) * | 2019-04-23 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Method and device for detecting abnormal request individuals |
| CN111835696A (en) * | 2019-04-23 | 2020-10-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting abnormal request individuals |
| CN112000538B (en) * | 2019-05-10 | 2023-09-15 | 百度在线网络技术（北京）有限公司 | Page content display monitoring method, device and equipment and readable storage medium |
| CN112000538A (en) * | 2019-05-10 | 2020-11-27 | 百度在线网络技术（北京）有限公司 | Page content display monitoring method, device and equipment and readable storage medium |
| CN110287322B (en) * | 2019-06-27 | 2021-04-16 | 有米科技股份有限公司 | Water flow processing method, system and equipment for social media flow |
| CN110287322A (en) * | 2019-06-27 | 2019-09-27 | 有米科技股份有限公司 | Moisture flow processing method, system and the equipment of social media flow |
| CN110995459A (en) * | 2019-10-12 | 2020-04-10 | 平安科技（深圳）有限公司 | Abnormal object identification method, device, medium and electronic equipment |
| CN110781494A (en) * | 2019-10-22 | 2020-02-11 | 武汉极意网络科技有限公司 | Data abnormity early warning method, device, equipment and storage medium |
| CN111080651A (en) * | 2019-12-12 | 2020-04-28 | 西南科技大学 | Automatic monitoring method for petroleum drilling polluted gas based on water flow segmentation |
| CN111314869A (en) * | 2020-02-18 | 2020-06-19 | 中国联合网络通信集团有限公司 | Flow quota distribution method and device, electronic equipment and storage medium |
| CN111404835A (en) * | 2020-03-30 | 2020-07-10 | 北京海益同展信息科技有限公司 | Flow control method, device, equipment and storage medium |
| CN113343064A (en) * | 2021-06-18 | 2021-09-03 | 北京百度网讯科技有限公司 | Data processing method, device, equipment, storage medium and computer program product |
| CN113343064B (en) * | 2021-06-18 | 2023-07-28 | 北京百度网讯科技有限公司 | Data processing method, apparatus, device, storage medium, and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109600345A (en) | Abnormal data flow rate testing methods and device | |
| TWI727202B (en) | Method and system for identifying fraudulent publisher networks | |
| US10243818B2 (en) | User interface that provides a proactive monitoring tree with state distribution ring | |
| US20190377463A1 (en) | User interface that facilitates node pinning for a proactive monitoring tree | |
| Huang | Patent portfolio analysis of the cloud computing industry | |
| US20150213631A1 (en) | Time-based visualization of the number of events having various values for a field | |
| US20140325058A1 (en) | Proactive monitoring tree with severity state sorting | |
| CN105913273A (en) | Intelligent recommendation all-terminal display method and device | |
| CN108710654B (en) | Public opinion data visualization method and equipment | |
| CN103514304A (en) | Project recommendation method and device | |
| CN109561052A (en) | The detection method and device of website abnormal flow | |
| CN109561326A (en) | A kind of data query method and device | |
| US20140006421A1 (en) | Collaborative filtering of a graph | |
| Rui et al. | Network-constrained and category-based point pattern analysis for Suguo retail stores in Nanjing, China | |
| US20110173046A1 (en) | Social network marketing plan comparison method and system | |
| US20160042366A1 (en) | System and method for monitoring competitive performance of brands | |
| US20220383168A1 (en) | Method and system for reducing risk values discrepancies between categories | |
| Pizzol et al. | Identifying marginal supplying countries of wood products via trade network analysis | |
| CN108388509A (en) | A kind of method for testing software, computer readable storage medium and terminal device | |
| CN110458615A (en) | Pass through the method and device of internet information assessment customer satisfaction | |
| Pruyt et al. | On generating and exploring the behavior space of complex models | |
| CN103870541A (en) | Social network user interest mining method and system | |
| Gunawardena et al. | Visual complexity analysis using taxonomic diagrams of figures and backgrounds in Japanese residential streetscapes | |
| US20160269857A1 (en) | System and method of creating abstractions of real and virtual environments and objects subject to latency constraints | |
| CN110493218B (en) | Situation awareness virtualization method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100080 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190409 |