CN104486116A - Multidimensional query method and multidimensional query system of flow data - Google Patents
Multidimensional query method and multidimensional query system of flow data Download PDFInfo
- Publication number
- CN104486116A CN104486116A CN201410767486.1A CN201410767486A CN104486116A CN 104486116 A CN104486116 A CN 104486116A CN 201410767486 A CN201410767486 A CN 201410767486A CN 104486116 A CN104486116 A CN 104486116A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- stream
- flows
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention provides a multidimensional query method and a multidimensional query system of flow data. The method comprises the following steps: acquiring flow data; gathering flow data containing same five-element set in the acquired flow data to flow, wherein the five-element set comprises a source IP (Internet Protocol), a destination IP, a source interface, a destination interface and a service type mark TOS queue; acquiring flow information corresponding to the gathered flow, wherein the flow information comprises a source product line, a destination product line, a source network section, a destination network section, a source computer room and a destination computer room; correspondingly storing the acquired flow information and the five-element set of the flow in a database which supports online analytical processing; and carrying out multidimensional query of the flow data by virtue of the database which supports online analytical processing. By virtue of the multidimensional query method and the multidimensional query system of flow data, the multidimensional query of mass flow data is effectively implemented by the database which supports online analytical processing.
Description
Technical field
The present invention relates to the communications field, specifically, relate to the method and system of various dimensions query flows data.
Background technology
Along with the expansion of Network scale, in-house traffic transport gets more and more, and nonstandard traffic transport can cause across machine room flow congestion, and finally cause Operational Visit impaired, traffic affecting is normally accessed.Therefore the inquiry fast magnanimity data on flows being carried out to various dimensions is needed for network management personnel, to grasp the real time status of network traffics, in net, application and different business are in the service condition of different time sections, traffic profile in certain time period of quick displaying, grasp across network of computer room load state, unreasonable with Timeliness coverage network configuration, quick position congested flow belongs to, and avoids the network bandwidth and server bottleneck problem.But, usually relevant database is used to carry out various dimensions inquiry in the prior art, but the query relation type database for large-scale data on flows is difficult to support, in addition for the conjunctive query of multiple dimension, comprises distributed data base and is also difficult to satisfy the demands.
Summary of the invention
For effectively solving the problems of the technologies described above, the invention provides a kind of method and system of various dimensions query flows data.
On the one hand, embodiments of the present invention provide a kind of method of various dimensions query flows data, and described method comprises:
Gather data on flows;
In the described data on flows collected, the data on flows containing identical five-tuple is pooled stream, wherein said five-tuple comprises: source IP, object IP, source interface, object interface, TOS queue;
Obtain the flow information corresponding with the described stream converged out, wherein said flow information comprises: product-derived line, object product line, the source network segment, destination network segment, source machine room, object machine room;
The five-tuple of the described flow information that gets and described stream is stored to the database supporting on-line analytical processing accordingly;
By the database of described support on-line analytical processing, various dimensions inquiry is carried out to described data on flows.
On the other hand, embodiments of the present invention provide a kind of system of various dimensions query flows data, and described system comprises:
Acquisition module, for gathering data on flows;
Convergence module, in the data on flows that collects at described acquisition module, the data on flows containing identical five-tuple is pooled stream, and wherein said five-tuple comprises: source IP, object IP, source interface, object interface, TOS queue;
Acquisition module, for obtain with described convergence module flow information corresponding to the stream that converges out, wherein said flow information comprises: product-derived line, object product line, the source network segment, destination network segment, source machine room, object machine room;
Memory module, for being stored to the five-tuple of the flow information accessed by described acquisition module and described stream the database supporting on-line analytical processing accordingly;
Various dimensions enquiry module, the database for the support on-line analytical processing be stored to by described memory module carries out various dimensions inquiry to described data on flows.
The method and system implementing various dimensions query flows data provided by the invention can inquire about magnanimity data on flows by various dimensions more efficiently by using the database supporting on-line analytical processing.
Accompanying drawing explanation
Fig. 1 is the flow chart of the method for various dimensions query flows data according to embodiment of the present invention;
Fig. 2 shows a kind of execution mode of the step S130 shown in Fig. 1;
Fig. 3 shows a kind of execution mode of the step S140 shown in Fig. 1;
Fig. 4 shows the frame diagram of the various dimensions query flows data of embodiment of the present invention;
Fig. 5 is the structural representation of the system of various dimensions query flows data according to embodiment of the present invention.
Embodiment
For making the object of embodiments of the invention, technical scheme and advantage clearly, below in conjunction with accompanying drawing, the present invention is described in further detail.
Fig. 1 is the flow chart of the method for various dimensions query flows data according to embodiment of the present invention.See Fig. 1, the method comprises:
S110: gather data on flows.
Wherein, in embodiments of the present invention, described data on flows can be gathered by the statistical technique of stream information Network Based, such as (provide counting messages function by Netstream, it distinguishes stream information according to the object ip address of message, destination slogan, ip address, source, source port number, protocol number and TOS (the COS marks of three layer data bags), and carries out independently data statistics for different stream informations) board gathers.
S120: in the described data on flows collected, the data on flows containing identical five-tuple is pooled stream, wherein said five-tuple comprises: source IP, object IP, source interface, object interface, TOS queue.Wherein stream is also called network flow, refer within a period of time, the unidirectional message flow transmitted between a source IP address and object IP address, all messages have identical source port number, destination slogan, protocol number and source, object IP address, and namely five-tuple content is identical.
S130: obtain the flow information corresponding with the described stream converged out, wherein said flow information can include, but are not limited to: product-derived line, object product line, the source network segment, destination network segment, source machine room, object machine room.
Wherein, as shown in Figure 2, step S130 can be realized in the following manner in embodiments of the present invention:
S131: according to the source IP of described stream, obtains the described source network segment;
S132: according to the described source network segment that gets and network element data, obtain described source machine room;
S133: according to described source IP and service metadata, obtains described product-derived line;
S134: according to the object IP of described stream, obtains described destination network segment;
S135: according to the described destination network segment that gets and network element data, obtain described object machine room;
S136: according to described object IP and service metadata, obtains described object product line.
Certainly, above-mentioned implementation procedure is only exemplary, for obtaining product-derived line, object product line, the source network segment, destination network segment, source machine room, can the changing accordingly as required smoothly of object machine room.
Further, in embodiments of the present invention, above-mentioned flow information can also comprise flow value.Can also comprise in the specific implementation of therefore this step S130: after execution step S120, the data on flows containing identical five-tuple is added up, and using the flow value of described cumulative result as described stream.
S140: the five-tuple of the described flow information that gets and described stream is stored to the database supporting on-line analytical processing accordingly.
Wherein, in embodiments of the present invention, exemplarily, the database of this support on-line analytical processing can be such as Palo database, it is the Large-scale Database System towards analysis of company of a Baidu, it is mainly used in supporting & stablizing, online, interactively data sheet and OLAP (On-LineAnalytical Processing, on-line analytical processing) (on-line analytical processing is shared multidimensional information in service, for the online process access of particular problem and the Fast Software technology of analysis, it is by carrying out fast to the multiple possible observation form of information, the access of stable and consistent and interactivity, administrative decision personnel are allowed to carry out deep observation to data.Decision data is multidimensional data, and multidimensional data is exactly the main contents of decision-making), the characteristic that of Palo database is very large is: meet the demand that this two class of form and olap analysis is different.Palo bottom layer realization is based on HDFS file system.
Further, be the efficiency improving various dimensions inquiry, one of the present invention preferred embodiment in, before the described step S140 of execution, can according to query feature, obtain inquiry field combination; Upper volume table is set up according to the described inquiry field combination got.Wherein going up volume table is the Aggregation Table set up some conventional dimension combination, can reduce data volume and improve the search efficiency to conventional dimension data.
As shown in Figure 3, in embodiments of the present invention, this step S140 can realize particularly in the following manner:
S141: corresponding with the five-tuple of described stream for the described flow information got is saved to text, and (Hadoop is a distributed system architecture developed by Apache fund club described text to be imported to Hadoop, the design that the framework of Hadoop is most crucial is exactly: HDFS (HadoopDistributed File System, a kind of distributed file system) and Map Reduce (a kind of programming model, the concurrent operation for large-scale dataset (being greater than 1TB)).HDFS is that the data of magnanimity provide storage, then Map Reduce is that the data of magnanimity provide calculating);
S142: this Hadoop to split described text and to be loaded into the different nodes of Palo database.
Wherein, in embodiments of the present invention, such as can be split described text by hash algorithm (can according to the quantity of described node).
S150: various dimensions inquiry is carried out to described data on flows by the database of described support on-line analytical processing.
Below in conjunction with concrete example, embodiments of the present invention are specifically described.
Fig. 4 shows the frame diagram of the various dimensions query flows data of embodiment of the present invention.See Fig. 4, idiographic flow of the present invention is as follows:
1) by network core switch ports themselves data on flows by netstream board mirror image out, to collect network flow data efficiently, its self-defined configuration sampling ratio and network flow relevant parameter, and with netflow (a kind of data exchange ways, its operation principle is: first IP bag data of stream, generate netflow buffer memory, data same are subsequently transmitted in same data flow based on cache information, no longer mate the strategies such as relevant access control, netflow buffer memory contains the statistical information of subsequent data stream simultaneously) v9 protocol form transmits, wherein can according to the quantity configuration netstream board of core switch, to obtain good autgmentability, support the process of more massive data on flows,
2) netflow receiver module flowagent (netflow data on flows receiver module, network equipment netflow flow data is resolved for receiving) receive netstream board data, resolve netflow message, according to five-tuple, data on flows is pooled stream, can carry out converging according to minute granularity and the data on flows of identical five-tuple be carried out cumulative to obtain data volume, and to netstream sampled data according to sampling than reducing, and preserve into DI file, and be transferred to data preprocessing module;
3) data preprocessing module by the data on flows received in conjunction with network element data (for obtaining the corresponding relation of the network segment (being defaulted as C section IP address) and machine room) and service metadata (for obtaining the corresponding relation of IP address and product line information), obtain flowing corresponding source, object product line information, source, destination network segment information, source, object computer room information, and preserve into text (five-tuple of each stream of text file physical record and the source of stream correspondence, object product line information, source, destination network segment information, source, object computer room information, the information such as the time of data volume and five-tuple), text file is imported to Hadoop,
4) Hadoop is loaded into the different nodes of Palo database after carrying out computing fractionation (can be split by hash algorithm) to text file, wherein Palo database can be set up according to the query feature of upper layer application in advance and roll up table (field combination choosing often inquiry is set up), the correlation inquiry that more than ten plant dimension can be realized, effectively improve search efficiency;
5), after Palo node loads data, the concurrent real-time query to mass data is provided according to SQL protocol form;
6) data query scheme adopts UDA (Universal Data Access, universal data access interface), the key (major key) unique to all query generations, and by result cache in redis (a kind of high performance key-value database) the inside, its life span is set to 3600s.
Because Palo database adopts efficient distributed data to import, bottom adopts Hadoop to carry out data prediction, therefore large-scale data inquiry can be realized relative to traditional relevant database or distributed relational database, various dimensions statistical analysis can also be carried out simultaneously, calculate multiple statistical indicator, with more deep awareness network flow distribution.Network auditing system can be provided further based on this, network cost is shared, service deployment is planned, network infrastructure is extended, service traffics monitor, comprehensively network traffics are analyzed and manages.
Fig. 5 is the structural representation of the system of various dimensions query flows data according to embodiment of the present invention.See Fig. 5, this system 100 comprises:
Acquisition module 110, for gathering data on flows.
Wherein, in embodiments of the present invention, this acquisition module 110 can comprise: statistical technique collecting unit, for gathering described data on flows by the statistical technique of stream information Network Based, particularly, this statistical technique collecting unit such as can adopt above-mentioned netstream board.
Convergence module 120, for in the data on flows that collects at described acquisition module 110, data on flows containing identical five-tuple is pooled stream, wherein said five-tuple comprises: source IP, object IP, source interface, object interface, TOS queue, wherein, this convergence module 120 such as can adopt above-mentioned netflow receiver module flowagent.
Acquisition module 130, for obtain with described convergence module 120 flow information corresponding to the stream that converges out, wherein said flow information comprises: product-derived line, object product line, the source network segment, destination network segment, source machine room, object machine room.
Wherein, in embodiments of the present invention, this acquisition module 130 can include, but are not limited to:
Source network segment acquiring unit, for the source IP according to described stream, obtains the described source network segment;
Source machine room acquiring unit, for the source network segment that gets according to described source network segment acquiring unit and network element data, obtains described source machine room;
Product-derived line acquiring unit, for according to described source IP and service metadata, obtains described product-derived line;
Destination network segment acquiring unit, for the object IP according to described stream, obtains described destination network segment;
Object machine room acquiring unit, for the destination network segment that gets according to described destination network segment acquiring unit and network element data, obtains described object machine room;
Object product line acquiring unit, for according to described object IP and service metadata, obtains described object product line.
Further, in embodiments of the present invention, above-mentioned flow information can also comprise flow value.Therefore this acquisition module 130 can also comprise: flow value acquiring unit, for adding up to the data on flows containing identical five-tuple, using the flow value of described cumulative result as described stream.
Memory module 140, for being stored to the five-tuple of the flow information accessed by described acquisition module 130 and described stream the database supporting on-line analytical processing accordingly.
Wherein, in embodiments of the present invention, the database of exemplarily this support on-line analytical processing can be such as Palo database.
Further, in working of an invention mode, this memory module 140 can include, but are not limited to:
Storage unit, for being saved to text by corresponding with the five-tuple of described stream for the described flow information got;
Loading unit, text for being saved to described storage unit splits and is loaded into the different nodes of described Palo database, wherein this loading unit can comprise: Hash splits assembly, for being split described text by hash algorithm.
Various dimensions enquiry module 150, the database for the support on-line analytical processing be stored to by described memory module 140 carries out various dimensions inquiry to described data on flows.
For improve various dimensions inquiry efficiency, one of the present invention preferred embodiment in, this system 100 can also comprise:
Inquiry field combination acquisition module, for according to query feature, obtains inquiry field combination;
Module set up by upper volume table, and the inquiry field combination for getting according to described inquiry field combination acquisition module sets up upper volume table.
Below be only an example of embodiment of the present invention; the present invention is not limited thereto; under the prerequisite not departing from protection scope of the present invention; the modules of described system 100 can carry out combining or merging in every way; module after merging is unified realizes the modules function separately before merging, such as, this acquisition module 130 and memory module 140 can merge into above-mentioned data preprocessing module.
The method and system implementing various dimensions query flows data provided by the invention can by supporting the database various dimensions inquiry more efficiently magnanimity data on flows of on-line analytical processing.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode of software combined with hardware platform, can certainly all be implemented by hardware.Based on such understanding, what technical scheme of the present invention contributed to background technology can embody with the form of software product in whole or in part, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprising some instructions in order to make a computer equipment (can be personal computer, server, smart mobile phone or the network equipment etc.) perform the method described in some part of each embodiment of the present invention or embodiment.
The term used in specification of the present invention and wording, just to illustrating, are not meaned and are formed restriction.It will be appreciated by those skilled in the art that under the prerequisite of the general principle not departing from disclosed execution mode, can various change be carried out to each details in above-mentioned execution mode.Therefore, scope of the present invention is only determined by claim, and in the claims, except as otherwise noted, all terms should be understood by the most wide in range rational meaning.
Claims (22)
1. a method for various dimensions query flows data, is characterized in that, described method comprises:
Gather data on flows;
In the described data on flows collected, the data on flows containing identical five-tuple is pooled stream;
Obtain the flow information corresponding with the described stream converged out;
The five-tuple of the described flow information that gets and described stream is stored to the database supporting on-line analytical processing accordingly;
By the database of described support on-line analytical processing, various dimensions inquiry is carried out to described data on flows.
2. the method for claim 1, is characterized in that,
Described five-tuple comprises: source IP, object IP, source interface, object interface and COS mark TOS queue.
3. the method for claim 1, is characterized in that,
Described flow information comprises: product-derived line, object product line, the source network segment, destination network segment, source machine room and object machine room.
4. the method for claim 1, is characterized in that, described collection flow packet is drawn together:
Described data on flows is gathered by the statistical technique of stream information Network Based.
5. method as claimed in claim 2, is characterized in that, obtains the flow information corresponding with the described stream converged out and comprises:
According to the source IP of described stream, obtain the described source network segment;
According to the described source network segment that gets and network element data, obtain described source machine room;
According to described source IP and service metadata, obtain described product-derived line;
According to the object IP of described stream, obtain described destination network segment;
According to the described destination network segment that gets and network element data, obtain described object machine room;
According to described object IP and service metadata, obtain described object product line.
6. method as claimed in claim 3, is characterized in that,
Described flow information also comprises flow value.
7. method as claimed in claim 6, is characterized in that, obtains the flow information corresponding with the described stream converged out and also comprises:
Execution described the data on flows containing identical five-tuple is pooled stream after, the data on flows containing identical five-tuple is added up,
Using the flow value of described cumulative result as described stream.
8. the method for claim 1, is characterized in that, described method also comprises:
Perform described the described flow information that gets is stored to the five-tuple of described stream the database supporting on-line analytical processing accordingly before, according to query feature, field combination is inquired about in acquisition;
Upper volume table is set up according to the described inquiry field combination got.
9., as the method in claim 1 ~ 8 as described in any one, it is characterized in that,
The database of described support on-line analytical processing comprises Palo database.
10. method as claimed in claim 9, is characterized in that, is stored to accordingly by the five-tuple of the described flow information that gets and described stream and supports that the database of on-line analytical processing comprises:
Corresponding with the five-tuple of described stream for the described flow information got is saved to text;
Described text is split and is loaded into the different nodes of Palo database.
11. methods as claimed in claim 10, is characterized in that, carry out fractionation comprise described text:
By hash algorithm, described text is split.
The system of 12. 1 kinds of various dimensions query flows data, is characterized in that, described system comprises:
Acquisition module, for gathering data on flows;
Convergence module, in the data on flows that collects at described acquisition module, pools stream by the data on flows containing identical five-tuple;
Acquisition module, for obtain with described convergence module flow information corresponding to the stream that converges out;
Memory module, for being stored to the five-tuple of the flow information accessed by described acquisition module and described stream the database supporting on-line analytical processing accordingly;
Various dimensions enquiry module, the database for the support on-line analytical processing be stored to by described memory module carries out various dimensions inquiry to described data on flows.
13. systems as claimed in claim 12, is characterized in that,
Described five-tuple comprises: source IP, object IP, source interface, object interface and COS mark TOS queue.
14. systems as claimed in claim 12, is characterized in that,
Described flow information comprises: product-derived line, object product line, the source network segment, destination network segment, source machine room and object machine room.
15. systems as claimed in claim 12, it is characterized in that, described acquisition module comprises:
Statistical technique collecting unit, for gathering described data on flows by the statistical technique of stream information Network Based.
16. systems as claimed in claim 13, it is characterized in that, described acquisition module comprises:
Source network segment acquiring unit, for the source IP according to described stream, obtains the described source network segment;
Source machine room acquiring unit, for the source network segment that gets according to described source network segment acquiring unit and network element data, obtains described source machine room;
Product-derived line acquiring unit, for according to described source IP and service metadata, obtains described product-derived line;
Destination network segment acquiring unit, for the object IP according to described stream, obtains described destination network segment;
Object machine room acquiring unit, for the destination network segment that gets according to described destination network segment acquiring unit and network element data, obtains described object machine room;
Object product line acquiring unit, for according to described object IP and service metadata, obtains described object product line.
17. systems as claimed in claim 14, is characterized in that,
Described flow information also comprises flow value.
18. systems as claimed in claim 17, it is characterized in that, described acquisition module also comprises:
Flow value acquiring unit, for adding up to the data on flows containing identical five-tuple, using the flow value of described cumulative result as described stream.
19. systems as claimed in claim 12, it is characterized in that, described system also comprises:
Inquiry field combination acquisition module, for according to query feature, obtains inquiry field combination;
Module set up by upper volume table, and the inquiry field combination for getting according to described inquiry field combination acquisition module sets up upper volume table.
20., as the system in claim 12 ~ 19 as described in any one, is characterized in that,
The database of described support on-line analytical processing comprises Palo database.
21. systems as claimed in claim 20, it is characterized in that, described memory module comprises:
Storage unit, for being saved to text by corresponding with the five-tuple of described stream for the described flow information got;
Loading unit, the text for being saved to described storage unit splits and is loaded into the different nodes of described Palo database.
22. systems as claimed in claim 21, it is characterized in that, described loading unit comprises:
Hash splits assembly, for being split described text by hash algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767486.1A CN104486116A (en) | 2014-12-12 | 2014-12-12 | Multidimensional query method and multidimensional query system of flow data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767486.1A CN104486116A (en) | 2014-12-12 | 2014-12-12 | Multidimensional query method and multidimensional query system of flow data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104486116A true CN104486116A (en) | 2015-04-01 |
Family
ID=52760624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410767486.1A Pending CN104486116A (en) | 2014-12-12 | 2014-12-12 | Multidimensional query method and multidimensional query system of flow data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104486116A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105515842A (en) * | 2015-12-01 | 2016-04-20 | 成都科来软件有限公司 | General query system and general query method based on network data |
| CN108268538A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Database aggregation processing method and device |
| CN108322403A (en) * | 2018-01-31 | 2018-07-24 | 杭州迪普科技股份有限公司 | A kind of Netflow flow shunts method and device |
| CN110022248A (en) * | 2019-04-19 | 2019-07-16 | 山东浪潮云信息技术有限公司 | Link flow statistical method and system, traffic statistics host and statistics request end |
| CN110505180A (en) * | 2018-05-17 | 2019-11-26 | 中国科学院声学研究所 | A kind of net flow assorted method and system |
| CN110795600A (en) * | 2019-11-05 | 2020-02-14 | 成都深思科技有限公司 | Aggregation dimension reduction statistical method for distributed network flow |
| CN110868422A (en) * | 2019-11-20 | 2020-03-06 | 杭州安恒信息技术股份有限公司 | Http site detection method, apparatus, device, and medium |
| CN111181799A (en) * | 2019-10-14 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Network traffic monitoring method and equipment |
| CN113542245A (en) * | 2021-07-02 | 2021-10-22 | 广州华多网络科技有限公司 | Data flow monitoring method and device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060159028A1 (en) * | 2005-01-20 | 2006-07-20 | Martin Curran-Gray | Monitoring system, method of sampling datagrams, and apparatus therefor |
| CN101668006A (en) * | 2009-10-12 | 2010-03-10 | 哈尔滨工程大学 | Self adaptive network traffic sampling method for anomaly detection |
| CN102495851A (en) * | 2011-11-17 | 2012-06-13 | 百度在线网络技术（北京）有限公司 | Method, system and device for storing and querying timing sequence data |
| CN102611626A (en) * | 2012-03-30 | 2012-07-25 | 北京英诺威尔科技股份有限公司 | System and method for analyzing network flow |
| CN104156389A (en) * | 2014-07-04 | 2014-11-19 | 重庆邮电大学 | Deep packet detecting system and method based on Hadoop platform |
-
2014
- 2014-12-12 CN CN201410767486.1A patent/CN104486116A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060159028A1 (en) * | 2005-01-20 | 2006-07-20 | Martin Curran-Gray | Monitoring system, method of sampling datagrams, and apparatus therefor |
| CN101668006A (en) * | 2009-10-12 | 2010-03-10 | 哈尔滨工程大学 | Self adaptive network traffic sampling method for anomaly detection |
| CN102495851A (en) * | 2011-11-17 | 2012-06-13 | 百度在线网络技术（北京）有限公司 | Method, system and device for storing and querying timing sequence data |
| CN102611626A (en) * | 2012-03-30 | 2012-07-25 | 北京英诺威尔科技股份有限公司 | System and method for analyzing network flow |
| CN104156389A (en) * | 2014-07-04 | 2014-11-19 | 重庆邮电大学 | Deep packet detecting system and method based on Hadoop platform |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105515842A (en) * | 2015-12-01 | 2016-04-20 | 成都科来软件有限公司 | General query system and general query method based on network data |
| CN105515842B (en) * | 2015-12-01 | 2018-12-07 | 成都科来软件有限公司 | A kind of general polling system and querying method based on network data |
| CN108268538A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Database aggregation processing method and device |
| CN108322403A (en) * | 2018-01-31 | 2018-07-24 | 杭州迪普科技股份有限公司 | A kind of Netflow flow shunts method and device |
| CN108322403B (en) * | 2018-01-31 | 2022-03-25 | 杭州迪普科技股份有限公司 | Netflow flow shunting method and device |
| CN110505180A (en) * | 2018-05-17 | 2019-11-26 | 中国科学院声学研究所 | A kind of net flow assorted method and system |
| CN110022248A (en) * | 2019-04-19 | 2019-07-16 | 山东浪潮云信息技术有限公司 | Link flow statistical method and system, traffic statistics host and statistics request end |
| CN111181799A (en) * | 2019-10-14 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Network traffic monitoring method and equipment |
| CN111181799B (en) * | 2019-10-14 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Network traffic monitoring method and equipment |
| CN110795600A (en) * | 2019-11-05 | 2020-02-14 | 成都深思科技有限公司 | Aggregation dimension reduction statistical method for distributed network flow |
| CN110868422A (en) * | 2019-11-20 | 2020-03-06 | 杭州安恒信息技术股份有限公司 | Http site detection method, apparatus, device, and medium |
| CN113542245A (en) * | 2021-07-02 | 2021-10-22 | 广州华多网络科技有限公司 | Data flow monitoring method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104486116A (en) | Multidimensional query method and multidimensional query system of flow data | |
| CN104769582B (en) | For the real time data releasing of power grid | |
| CN107943668A (en) | Computer server cluster daily record monitoring method and monitor supervision platform | |
| CN109951463A (en) | A kind of Internet of Things big data analysis method stored based on stream calculation and novel column | |
| CN110413599A (en) | Generating date and storage system and method | |
| CN105069025A (en) | Intelligent aggregation visualization and management control system for big data | |
| CN104216989A (en) | Method for storing transmission line integrated data based on HBase | |
| CN103414608B (en) | Rapid web flow collection statistical system and method | |
| CN105069113A (en) | Data flow real-time visualization method and data flow real-time visualization system | |
| CN104933136B (en) | Dynamic share-car method and system based on magnanimity license auto-recognition system data | |
| CN102611626B (en) | System and method for analyzing network flow | |
| CN101141370A (en) | Gridding service based electric power enterprise real-time data processing method | |
| CN106649770A (en) | Large data query method and system | |
| Wakamiya et al. | Crowd-sourced urban life monitoring: urban area characterization based crowd behavioral patterns from twitter | |
| CN108268569A (en) | The acquisition of water resource monitoring data and analysis system and method based on big data technology | |
| CN111258978A (en) | Data storage method | |
| CN105450997A (en) | Cloud storage based video monitoring system | |
| CN112134719A (en) | Method and system for analyzing base station security log | |
| Gaurav et al. | An outline on big data and big data analytics | |
| Chen et al. | Big data generation and acquisition | |
| CN115695216A (en) | Big data analysis method for internet traffic flow direction | |
| CN105446707A (en) | Data conversion method | |
| CN115374101A (en) | Rail transit station level data management system | |
| Ghahramani et al. | Analysis of mobile phone data under a cloud computing framework | |
| KR101345095B1 (en) | Method and system for bgp routing data processing based on cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150401 |