CN110266603A - Authentication business network flow analysis system and method based on http protocol - Google Patents
Authentication business network flow analysis system and method based on http protocol Download PDFInfo
- Publication number
- CN110266603A CN110266603A CN201910569202.0A CN201910569202A CN110266603A CN 110266603 A CN110266603 A CN 110266603A CN 201910569202 A CN201910569202 A CN 201910569202A CN 110266603 A CN110266603 A CN 110266603A
- Authority
- CN
- China
- Prior art keywords
- service node
- network
- service
- node
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0894—Packet rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention is a kind of system and method for authentication business network flow analysis based on http protocol, and the system comprises the authentication platform, communication interface, memory, Back end data analysis platform and the flow collection equipment that carry out communication connection;Wherein, the network flow based on http protocol is disposed in service node acquire equipment.The present invention passes through Business Stream, data flow and the network planning to authentication platform, the part for needing to pay close attention to network flow in authentication platform is distinguished in a manner of service node, then hardware net water flow collection device is disposed, pass through HANDOOP big data processing platform later, analysis, cleaning, the storage of acquisition data are completed, analysis and fault pre-alarming finally are carried out to network based on these data.By using reasonably memory technology and structure, the quantity for landing data and accessed number are reduced, authentication platform memory capacity and storage efficiency are improved.
Description
Technical field
The present invention relates to and computer software develop programming technique field, and in particular to the authentication based on http protocol
Business network flow analysis system and method.
Background technique
With the fast development of Internet technology, society extends to cyberspace depth, and a networked society has become existing
The important component of real society.Internet is brought to people's life also changed dramatically people's while earth-shaking variation
Behavior, the behavior and business that receiving and dispatching mail, social activity, shopping, bank transaction etc. only occurred in society in the past, rapidly
Occur in cyberspace and obtains high speed development.
In recent years, residential identity network authentication techniques in China's obtain rapidly, network size with number of users exponentially
Increase, it is also more and more huger.The platform and the network architecture for supporting residential identity network authentication techniques also become increasingly complex, to network
The requirement of troubleshooting, it is also higher and higher.How to sketch the circulation path of data packet in network, optimization network node receives and hair
The time for sending data packet, the problem of being likely to occur in advance to network node, carry out early warning, this be one it is very real be also very urgent
The problem of.The network flow analysis method based on http protocol is used in this patent, and traditional flow analysis method is used for body
In part authentication business, there are the data of the inflow, the outflow that obtain each service node network flow in real time, analyze performance of network equipments
Bottleneck, grasp the network operation the characteristics of, network is suitably adjusted in time, guarantee network normal table run.
(1) the acquisition interception and distributed storage of network message
Net flow assorted based on machine learning method mainly presses the flow point class that application type forms network layer message.
Message is the minimum unit in categorizing system.According to message five-tuple (source IP address, source port number, target ip address, target side
Slogan and IP agreement) definition, message (Packets) is divided into two-way TCP or UDP flow, extracts stream unrelated with agreement, port
Feature, formed feature vector, indicate to flow with feature vector.The feature of characterization network flow has very much, and Andrew Moore was once arranged
The 249 kinds of features flowed out, and therefrom feature set of the 37 kinds of best candidate features of selective discrimination granularity as sorting flow, specifically
The temporal characteristics of message number and size characteristic, stream including stream and the zone bit information feature of message.Based on features above
Statistics and calculating, need to only intercept preceding 128 bytes of message.In system design, using WinPcap tool Packet capturing report
Text, and preceding 128 bytes for intercepting message form dmp file.Secondly, message data is based on Hadoop tool distributed storage
To HDFS.The size that each piece of HDFS is 64M.In order to guarantee that the Block of each HDFS can store complete message number, test
It is middle that the partial information interception of dmp file header is saved in the NameNode of Hadoop.In this way, each Block in DataNode
The message number of storage is 64 × 1024 × 1024/128=524,288 data message informations.
(2) formation of the network flow based on MapReduce algorithm
Network flow forming process based on MapReduce algorithm is divided into two stages.First, the Map stage are applied to HDFS
Each piece of message analysis.According to the message information of 128 bytes, basic message data, including time, capture length, length are extracted
Degree, agreement, source IP address and target ip address etc..Second, the Reduce stage are divided into multilayer Reducer.First layer Reducer is negative
It blames the message information for extracting the Map stage and forms stream according to the definition of network flow, and calculate the candidate feature of stream;The second layer
Reducer is responsible for the stream for forming first layer Reducer and merges according to five-tuple information.
Network traffic analysis device has universality, has certain limitation in profession and orientation analysis, this patent is being set
When counting Back end data analysis system, also consult and with reference to many design datas.It is suitble to authentication currently without comparison is found
The network flow analysis method and system of platform.The mainly following reason of authentication platform:
1) amount of access of authentication platform and concurrent amount of access are all very big;
2) communications protocol is all using http protocol;
3) complicated network structure of authentication platform;
4) authentication platform uses more ground multicenter mode.
The above reason determines the network traffic analysis of authentication platform, not available ready-made network flow
Measure analysis system.
Summary of the invention
In view of the deficiencies in the prior art, the present invention intends to provide the authentication business network flow based on http protocol
Analysis system and method solve the technical method of authentication platform Network Traffic Data Collection and analysis using triangular web.
By Business Stream, data flow and the network planning to authentication platform, will need to pay close attention to network in authentication platform
The part of flow is distinguished in a manner of service node, then disposes hardware net water flow collection device, passes through the big number of HANDOOP later
According to processing platform, analysis, cleaning, the storage of acquisition data are completed, analysis and failure finally are carried out to network based on these data
Early warning.
By using reasonably memory technology and structure, the quantity for landing data and accessed number are reduced, improves and recognizes
Demonstrate,prove platform memory capacity and storage efficiency.
In order to solve the above technical problems, technical solution provided by the invention is as follows:
Authentication business network flow analysis method based on http protocol, the described method comprises the following steps:
S1 combed and identified by Business Stream, data flow and the network planning to authentication platform, will be in authentication platform
The part for needing to pay close attention to network flow is identified in a manner of service node;
S2 disposes the network flow based on http protocol in service node and acquires equipment;
The traffic mirroring of each node of S3 obtains the stream of each server node on every service link to flow collection equipment
Amount;
S4 completes analysis, cleaning, the storage of acquisition data by HANDOOP big data processing platform;
The case where S5 changes according to network flow data analyzes network operation state in real time;
S6 has found the bottleneck of performance of network equipments, grasps network according to the inflow of service node network flow, outflow data
The characteristics of operation, in time suitably adjusts network, guarantees the normal table operation of network;
S7 sets the threshold value of the service processing time of each node according to service node process performance, passes through analysis service section
The processing time of point, judge the service performance situation of service node;
S8 sets the threshold value of the service transmission time of each node according to service node transmission time, passes through analysis service section
The transmission time of point, judges the service status of service node.
It should be noted that the inflow of a service node, outflow data volume can be obtained in the step 2,
It should be noted that the sending that a service node calls another service node can be obtained in the step 2
Time, and obtain a service node to another service node transmission time.
It should be noted that another service node receiving time and another service node return to a service node
At the beginning of, and obtain the processing time of another service node.
As the authentication business network flow analysis system of the invention based on http protocol, the system comprises into
Authentication platform, communication interface, memory, Back end data analysis platform and the flow collection equipment of row communication connection;Wherein, it is taking
Business node deployment acquires equipment based on the network flow of http protocol.
The beneficial effects of the invention are that:
1) much sooner to the early warning of network failure
The case where being changed according to network flow data finds the problems in network, when network flow levels off to 0, shows to be supervised
Network failure may occur for the position of control, need to handle in time.
2) to the performance bottleneck in the network planning, can accomplish to find and adjust in time
According to the inflow of network flow, the analysis of outflow data, finds the bottleneck of performance of network equipments, grasp the network operation
The characteristics of, network is suitably adjusted in time, guarantees the normal table operation of network.
3) to the service performance of service node each in network, accomplish precisely to grasp
The threshold value that the service processing time of each node is set according to service node process performance, passes through analysis service node
The processing time, judge the service performance situation.
Detailed description of the invention
Fig. 1 is schematic diagram of the invention;
Fig. 2 is inventive network flow collection flow diagram.
Specific embodiment
The present invention is further illustrated below by the mode of attached drawing and embodiment, but does not therefore limit the present invention to institute
Among the scope of embodiments stated.
As shown in Figure 1, the present invention is the authentication business network flow analysis method based on http protocol, the method
The following steps are included:
S1 combed and identified by Business Stream, data flow and the network planning to authentication platform, will be in authentication platform
The part for needing to pay close attention to network flow is identified in a manner of service node;
S2 disposes the network flow based on http protocol in service node and acquires equipment;
The traffic mirroring of each node of S3 obtains the stream of each server node on every service link to flow collection equipment
Amount;
S4 completes analysis, cleaning, the storage of acquisition data by HANDOOP big data processing platform;
The case where S5 changes according to network flow data analyzes network operation state in real time;
S6 has found the bottleneck of performance of network equipments, grasps network according to the inflow of service node network flow, outflow data
The characteristics of operation, in time suitably adjusts network, guarantees the normal table operation of network;
S7 sets the threshold value of the service processing time of each node according to service node process performance, passes through analysis service section
The processing time of point, judge the service performance situation of service node;
S8 sets the threshold value of the service transmission time of each node according to service node transmission time, passes through analysis service section
The transmission time of point, judges the service status of service node.
It should be noted that the inflow of a service node, outflow data volume can be obtained in the step 2,
It should be noted that the sending that a service node calls another service node can be obtained in the step 2
Time, and obtain a service node to another service node transmission time.
It should be noted that another service node receiving time and another service node return to a service node
At the beginning of, and obtain the processing time of another service node.
As shown in Fig. 2, obtaining and taking on every service link the traffic mirroring of each node to flow collection equipment F
The flow of business device node, by the analysis of data on flows, is obtained when a business is called from service node A to service node B
Following content:
(1) inflow of B service node, outflow data volume;
(2) A service node calls the time for issuing the time, reaching B service of B service node, and A service is obtained by calculation
The transmission time serviced to B;
(2) it at the beginning of B service node receiving time and B service node return to A service node, is obtained by calculation
The processing time of B service.
According to the collected data, following analysis can also be obtained:
(1) analyze network operation state: the case where being changed according to network flow data finds the problems in network, works as network
Flow levels off to 0, shows that network failure may occur for monitored position, needs to handle in time;
(2) different duration network flow datas are analyzed: according to the inflow of network flow, outflow data, finding the network equipment
The characteristics of bottleneck of performance, the grasp network operation, network is suitably adjusted in time, guarantees the normal table fortune of network
Row;
(3) the analysis service node processing time: when setting the service processing of each node according to service node process performance
Between threshold value the service performance situation is judged by the processing time of analysis service node;
(4) the analysis service node-node transmission time: when setting the service transmission of each node according to service node transmission time
Between threshold value the service status is judged by the transmission time of analysis service node.
Although specific embodiments of the present invention have been described above, it will be appreciated by those of skill in the art that this is only
For example, protection scope of the present invention is to be defined by the appended claims.Those skilled in the art without departing substantially from
Under the premise of the principle and substance of the present invention, many changes and modifications may be made, but these change and
Modification each falls within protection scope of the present invention.
Claims (5)
1. the authentication business network flow analysis method based on http protocol, which is characterized in that the method includes following
Step:
S1 is combed and is identified by Business Stream, data flow and the network planning to authentication platform, will be needed in authentication platform
The part for paying close attention to network flow is identified in a manner of service node;
S2 disposes the network flow based on http protocol in service node and acquires equipment;
The traffic mirroring of each node of S3 obtains the flow of each server node on every service link to flow collection equipment;
S4 completes analysis, cleaning, the storage of acquisition data by HANDOOP big data processing platform;
The case where S5 changes according to network flow data analyzes network operation state in real time;
S6 has found the bottleneck of performance of network equipments, grasps the network operation according to the inflow of service node network flow, outflow data
The characteristics of, network is suitably adjusted in time, guarantees the normal table operation of network;
S7 sets the threshold value of the service processing time of each node according to service node process performance, passes through analysis service node
The time is handled, judges the service performance situation of service node;
S8 sets the threshold value of the service transmission time of each node according to service node transmission time, passes through analysis service node
Transmission time judges the service status of service node.
2. the authentication business network flow analysis method according to claim 1 based on http protocol, feature exist
In, can be obtained in the step 2 service node inflow, outflow data volume.
3. the authentication business network flow analysis method according to claim 1 based on http protocol, feature exist
In the sending time that a service node calls another service node can be obtained in the step 2, and obtain a service
Transmission time of the node to another service node.
4. the authentication business network flow analysis method according to claim 3 based on http protocol, feature exist
At the beginning of, another service node receiving time and another service node return to a service node, and obtain
The processing time of another service node.
5. a kind of authentication business network flow analysis method according to claim 1 based on http protocol is
System, which is characterized in that the system comprises the authentication platforms, communication interface, memory, Back end data analysis that carry out communication connection
Platform and flow collection equipment;Wherein, the network flow based on http protocol is disposed in service node acquire equipment.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910569202.0A CN110266603B (en) | 2019-06-27 | 2019-06-27 | System and method for analyzing network flow of identity authentication service based on HTTP (hyper text transport protocol) |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910569202.0A CN110266603B (en) | 2019-06-27 | 2019-06-27 | System and method for analyzing network flow of identity authentication service based on HTTP (hyper text transport protocol) |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110266603A true CN110266603A (en) | 2019-09-20 |
CN110266603B CN110266603B (en) | 2022-12-20 |
Family
ID=67922410
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910569202.0A Active CN110266603B (en) | 2019-06-27 | 2019-06-27 | System and method for analyzing network flow of identity authentication service based on HTTP (hyper text transport protocol) |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110266603B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
CN113542160A (en) * | 2021-05-27 | 2021-10-22 | 贵州电网有限责任公司 | SDN-based method and system for pulling east-west flow in cloud |
CN114244582A (en) * | 2021-11-29 | 2022-03-25 | 国网江西省电力有限公司电力科学研究院 | Low-distribution data acquisition terminal authentication method for data association in Internet of things |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7991827B1 (en) * | 2002-11-13 | 2011-08-02 | Mcafee, Inc. | Network analysis system and method utilizing collected metadata |
CN102148716A (en) * | 2010-02-05 | 2011-08-10 | 中国联合网络通信集团有限公司 | Point-to-point system network performance testing method and system thereof |
CN104753732A (en) * | 2013-12-27 | 2015-07-01 | 郭祖龙 | Distribution based network traffic analysis system and method |
CN105610983A (en) * | 2016-03-07 | 2016-05-25 | 北京荣之联科技股份有限公司 | Distributive network monitoring method and system |
CN105979532A (en) * | 2016-04-15 | 2016-09-28 | 北京思特奇信息技术股份有限公司 | Performance-capacity analysis early warning method and device for service processing system |
CN108833126A (en) * | 2018-04-02 | 2018-11-16 | 平安科技(深圳)有限公司 | Electronic device, data link method for prewarning risk and storage medium |
CN109586999A (en) * | 2018-11-12 | 2019-04-05 | 深圳先进技术研究院 | A kind of container cloud platform condition monitoring early warning system, method and electronic equipment |
-
2019
- 2019-06-27 CN CN201910569202.0A patent/CN110266603B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7991827B1 (en) * | 2002-11-13 | 2011-08-02 | Mcafee, Inc. | Network analysis system and method utilizing collected metadata |
CN102148716A (en) * | 2010-02-05 | 2011-08-10 | 中国联合网络通信集团有限公司 | Point-to-point system network performance testing method and system thereof |
CN104753732A (en) * | 2013-12-27 | 2015-07-01 | 郭祖龙 | Distribution based network traffic analysis system and method |
CN105610983A (en) * | 2016-03-07 | 2016-05-25 | 北京荣之联科技股份有限公司 | Distributive network monitoring method and system |
CN105979532A (en) * | 2016-04-15 | 2016-09-28 | 北京思特奇信息技术股份有限公司 | Performance-capacity analysis early warning method and device for service processing system |
CN108833126A (en) * | 2018-04-02 | 2018-11-16 | 平安科技(深圳)有限公司 | Electronic device, data link method for prewarning risk and storage medium |
CN109586999A (en) * | 2018-11-12 | 2019-04-05 | 深圳先进技术研究院 | A kind of container cloud platform condition monitoring early warning system, method and electronic equipment |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
CN112069021B (en) * | 2020-08-21 | 2024-02-20 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
CN113542160A (en) * | 2021-05-27 | 2021-10-22 | 贵州电网有限责任公司 | SDN-based method and system for pulling east-west flow in cloud |
CN114244582A (en) * | 2021-11-29 | 2022-03-25 | 国网江西省电力有限公司电力科学研究院 | Low-distribution data acquisition terminal authentication method for data association in Internet of things |
CN114244582B (en) * | 2021-11-29 | 2023-06-20 | 国网江西省电力有限公司电力科学研究院 | Authentication method for low-profile data acquisition terminal associated with data in Internet of things |
Also Published As
Publication number | Publication date |
---|---|
CN110266603B (en) | 2022-12-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102315974B (en) | Stratification characteristic analysis-based method and apparatus thereof for on-line identification for TCP, UDP flows | |
CN105871832B (en) | A kind of network application encryption method for recognizing flux and its device based on protocol attribute | |
CN101714952B (en) | Method and device for identifying traffic of access network | |
CN102035698B (en) | HTTP tunnel detection method based on decision tree classification algorithm | |
CN106815112B (en) | Massive data monitoring system and method based on deep packet inspection | |
EP1764951B1 (en) | Statistical trace-based method, apparatus, node and system for real-time traffic classification | |
CN106341337B (en) | Flow detection and control mechanism and method capable of realizing application awareness under SDN | |
CN104283897B (en) | Wooden horse communication feature rapid extracting method based on multiple data stream cluster analysis | |
CN104092756B (en) | A kind of resource dynamic distributing method of the cloud storage system based on DHT mechanism | |
CN108289104A (en) | A kind of industry SDN network ddos attack detection with alleviate method | |
CN110266603A (en) | Authentication business network flow analysis system and method based on http protocol | |
CN101841440B (en) | Peer-to-peer network flow identification method based on support vector machine and deep packet inspection | |
CN104102700A (en) | Categorizing method oriented to Internet unbalanced application flow | |
CN101924757A (en) | Method and system for reviewing Botnet | |
CN102202064A (en) | Method for extracting behavior characteristics of Trojan communication based on network data flow analysis | |
CN109271793A (en) | Internet of Things cloud platform device class recognition methods and system | |
CN110417729A (en) | A kind of service and application class method and system encrypting flow | |
CN101184000A (en) | Packet sampling and application signature based internet application flux identifying method | |
CN102571946B (en) | Realization method of protocol identification and control system based on P2P (peer-to-peer network) | |
CN110034970A (en) | The network equipment distinguishes method of discrimination and device | |
CN103298035A (en) | Congestion control method and device | |
CN106789242A (en) | A kind of identification application intellectual analysis engine based on mobile phone client software behavioral characteristics storehouse | |
CN106941517A (en) | Five-tuple ticket synthetic method and device under a kind of asymmetric condition | |
CN106452941A (en) | Network anomaly detection method and device | |
CN109088903A (en) | A kind of exception flow of network detection method based on streaming |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20210722 Address after: 100048 No. 1, South Road, capital gymnasium, Beijing, Haidian District Applicant after: THE FIRST Research Institute OF MINISTRY OF PUBLIC SECURITY Applicant after: Beijing ZHONGDUN Anxin Technology Development Co.,Ltd. Address before: 100048 No. 1, South Road, capital gymnasium, Beijing, Haidian District Applicant before: THE FIRST Research Institute OF MINISTRY OF PUBLIC SECURITY |
|
TA01 | Transfer of patent application right | ||
GR01 | Patent grant | ||
GR01 | Patent grant |