CN107147674A - The analytic method and router and device of a kind of network data - Google Patents
The analytic method and router and device of a kind of network data Download PDFInfo
- Publication number
- CN107147674A CN107147674A CN201710482223.XA CN201710482223A CN107147674A CN 107147674 A CN107147674 A CN 107147674A CN 201710482223 A CN201710482223 A CN 201710482223A CN 107147674 A CN107147674 A CN 107147674A
- Authority
- CN
- China
- Prior art keywords
- data
- resolved
- network data
- network
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/026—Capturing of monitoring data using flow identification
Abstract
The invention discloses a kind of analytic method of network data, the data link layer information in network data is retained into terminal MAC address information, network data to be resolved is re-packaged into；According to a default data collecting mechanism gathered data, store into the data pool of network data to be resolved, when the data pool data volume be more than the first pre-set threshold value when, or the acquisition time be more than the second pre-set threshold value when, stop collection；One field value of the network data to be resolved is designated to the packet length of network data to be resolved, obtain the numerical value of a predeterminated position field of the network data to be resolved, if the numerical value meets the field value of the same position in standard network data protocol architecture, then judge the network data field value to be resolved for the packet length, and the network data to be resolved is parsed layer by layer, improve the resolution speed of network data.
Description
Technical field
The invention belongs to the analytic method and router of communication technical field, more particularly to network data.
Background technology
With the development and the popularization of computer of internet, the quantity rapid development of the network user, network behavior gradually into
One of most important social phenomenon in human behavior.Understand and analyze the behavior of the network user in depth, bring very big to user
Facility.
Packet species is various, and the data storage rule of different types of packet is also not quite similar.But user network
The species of packet is various, and the data of the packet got are all to store in binary form, and storage rule is various, it is difficult to do
To the data parsing and extraction of accurate packet.When handling a large number of users packet information, if it is possible to carry out batch
Processing, the network behavior precision data of quick obtaining user, and user behavior data analysis is carried out, huge business valency will be produced
Value.
Therefore, how carry out fast resolving is carried out to packet, as urgent problem to be solved.
The content of the invention
The technical scheme that the present invention is provided is as follows：
The present invention provides a kind of analytic method of network data, it is characterised in that comprise the following steps：
S1, the data link layer information in network data retained into terminal MAC address information, and by the network data weight
Newly it is encapsulated as network data to be resolved；
S2, according to a default data collecting mechanism, network data to be resolved, is stored to be resolved described in continuous collecting
Network data data pool in, when the data volume of the data pool is more than the first pre-set threshold value, or the acquisition time
During more than the second pre-set threshold value, stop collection；
S3, the packet length that a field value of the network data to be resolved is designated to network data to be resolved, and
According to the packet length, the numerical value of a predeterminated position field of the network data to be resolved is obtained, if the numerical value meets
The field value of same position in standard network data protocol architecture, then judge the network data field value to be resolved for institute
Packet length is stated, and the network data to be resolved is parsed layer by layer；
S4, otherwise, repeats step S3.
Preferably, step S1 also includes：
The field of one or two of byte is added in the packet header of the network data to be resolved, the value of the field is used to set institute
State the packet length value of network data to be resolved.
It is preferred that, step S1 also includes：
It is described to wait to solve if the data length of the middle application layer data information of the network data is more than the first preset value
The data length of the application layer data information included in the network data of analysis corresponds to the preset value, otherwise the net to be resolved
The data length of the application layer data information included in network data keeps constant.
Preferably, step S2 is specifically included：
According to the field value of two bytes in the network data packet header to be resolved, the packet length is obtained, and count
Total packet length of all network packets to be resolved collected；
When total packet length is more than the second preset value, stop data acquisition, and in the network data to be resolved
The packet header of bag sets the field of two bytes, and the field value represents total packet length of collected network packet to be resolved；
After dwell time is more than three preset values, proceed collection, and by the network packet to be resolved of collection,
Store into the data pool of network data to be resolved.
Preferably, step S2 specifically also includes：
Total packet length of the network packet to be resolved of all collections is counted, the network data to be resolved is obtained
The data volume of data pool, when the data volume of the data pool is more than the first pre-set threshold value, or the acquisition time is more than the
During two pre-set threshold values, stop collection.
The invention provides a kind of router, including：
Data package module, for the data link layer information in network data to be retained into terminal MAC address information, and will
The network data is re-packaged into network data to be resolved；
Data collection module, for according to a default data collecting mechanism, network number to be resolved described in continuous collecting
According to, store into the data pool of network data to be resolved, when the data volume of the data pool is more than the first pre-set threshold value, or
When acquisition time described in person is more than the second pre-set threshold value, stop collection.
Preferably, the data package module, is additionally operable to add one or two in the packet header of the network data to be resolved
The field of byte, the value of the field is used for the packet length value for setting the network data to be resolved.
Preferably, the data collection module is specifically included：
Statistic unit, for the field value of two bytes according to the network data packet header to be resolved, obtains described
Packet length, and count total packet length of all network packets to be resolved collected；
Judging unit, for when total packet length is more than the second preset value, stopping data acquisition, and wait to solve described
The packet header of the network packet of analysis sets the field of two bytes, and the field value represents collected network packet to be resolved
Total packet length；
The judging unit, is additionally operable to after dwell time is more than three preset values, proceed collection, and by collection
Network packet to be resolved, is stored into the data pool of network data to be resolved.
Preferably, the statistic unit, is additionally operable to count total packet length of the network packet to be resolved of all collections,
Obtain the data volume of the data pool of the network data to be resolved；
The judging unit, is additionally operable to when the data volume of the data pool is more than the first pre-set threshold value, or described adopt
When the collection time is more than the second pre-set threshold value, stop collection.
Present invention also offers a kind of network data resolver, including：
Router, for the data link layer information in network data to be retained into terminal MAC address information, and by the net
Network data are re-packaged into network data to be resolved, according to a default data collecting mechanism, to be resolved described in continuous collecting
Network data, store into the data pool of network data to be resolved, when the data pool data volume be more than first preset
During threshold values, or the acquisition time be more than the second pre-set threshold value when, stop collection；
Service end, the bag for a field value of the network data to be resolved to be designated to network data to be resolved
Length, and according to the packet length, the numerical value of a predeterminated position field of the network data to be resolved is obtained, if the number
Value meets the field value of the same position in standard network data protocol architecture, then judges the network data field to be resolved
It is worth for the packet length, and the network data to be resolved is parsed layer by layer.
Compared with prior art, the analysis method and device and router of a kind of network data of the invention, have with following
Beneficial effect：
1), by re-starting encapsulation to the network data, remove unwanted data link layer information, reduce and wait to solve
The packet length of the network data of analysis, increases the capacity for collecting network data to be resolved, reduces data parsing to router
The influence of performance, improves the resolution speed of network data.
2), by using the mechanism of data acquisition dead time, that is to say, that after collection network data is more than a preset value
Stop collecting, then continue to gather network data after a preset time, the network data of collection used reaches certain data
Afterwards, stop data acquisition, so at regular intervals or accumulation reaches a number of packet, form data APMB package and send
Parsed to server end, reduce influence of the Data Collection to router performance, improve the resolution speed of network data.
3), according to the original position of the positioning network data to be resolved, and the bag of network data to be resolved is obtained
Length, and then the network data to be resolved is parsed layer by layer, obtain the data link layer of network data to be resolved
Message, transport layer message and network layer information, and then the mac address information in network data, IP address information can be obtained
And URL information etc., and analyzed according to these information, user is used according to these information, brings very big to user
Convenience.
Brief description of the drawings
Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, to a kind of parsing of network data
Method and device and above-mentioned characteristic, technical characteristic, advantage and its implementation of router are further described.
Fig. 1 is a kind of schematic flow sheet of the analytic method of network data of the invention；
Fig. 2 is the structure chart of the network data of a specific embodiment of the invention；
Fig. 3 is a kind of composition structural representation of router of the invention；
Fig. 4 is a kind of composition structural representation of network data resolver of the invention.
Embodiment
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, control is illustrated below
The embodiment of the present invention.It should be evident that drawings in the following description are only some embodiments of the present invention, for
For those of ordinary skill in the art, on the premise of not paying creative work, other can also be obtained according to these accompanying drawings
Accompanying drawing, and obtain other embodiments.
To make only to schematically show part related to the present invention in simplified form, each figure, they are not represented
Its as product practical structures.In addition, so that simplified form is readily appreciated, there is identical structure or function in some figures
Part, only symbolically depicts one of those, or has only marked one of those.Herein, " one " is not only represented
" only this ", can also represent the situation of " more than one ".
As shown in figure 1, according to one embodiment of present invention, a kind of analytic method of network data comprises the following steps：
S1, the data link layer information in network data retained into terminal MAC address information, and by the network data weight
Newly it is encapsulated as network data to be resolved；
S2, according to a default data collecting mechanism, network data to be resolved, is stored to be resolved described in continuous collecting
Network data data pool in, when the data volume of the data pool is more than the first pre-set threshold value, or the acquisition time
During more than the second pre-set threshold value, stop collection；
S3, the packet length that a field value of the network data to be resolved is designated to network data to be resolved, and
According to the packet length, the numerical value of a predeterminated position field of the network data to be resolved is obtained, if the numerical value meets
The field value of same position in standard network data protocol architecture, then judge the network data field value to be resolved for institute
Packet length is stated, and the network data to be resolved is parsed layer by layer；
S4, otherwise, repeats step S3.
In specific implementation process, router is got after uplink network data, it is necessary to carry out data link to network data
The encapsulation of layer, Internet and transport layer.In the inventive solutions, there are two data transmission paths, one in the router
Data transmission path is to after the completion of network data progress complete package, according to the Path First of router, by the network number
According to forwarding, a data transmission path is network data to be acquired and parsed the network data in addition, and then right
The network data is analyzed.Specifically, when carrying out data link layer information encapsulation to network data, to data link layer
For information, it would be desirable to retain the mac address information of terminal, and the network packet re-started be encapsulated as it is to be resolved
Network data.Because during real network data behavioural analysis, other data messages of data link layer are substantially
It is that our institutes are unwanted, so without all data link layer informations are encapsulated into network packet.In the network
In the specific encapsulation process of data, router carries out the encapsulation of data link layer and transport layer in the network packet to reception
Before, in order to complete the forwarding to network data, it is necessary to which the source MAC address information in data link layer information has been replaced by into road
By the mac address information of device, therefore when being encapsulated as network data to be resolved to the network data, deposited according in router
The mac address table of storage, obtains the corresponding terminal MAC address information of the network packet, and by the network data
Data link layer information is replaced by terminal MAC address information, is encapsulated as network data to be resolved.In the network to be resolved
The field of one or two of byte is added in the packet header of data, and the value of the field is used for the packet length for setting the network data to be resolved
Value, the packet length for representing the network data to be resolved.
By the technical scheme, the data link layer information in network packet, analysis and user's row to network data
For analysis, it is most that the terminal MAC address information of data link layer information is used, and other data of data link layer are believed
Breath is that our institutes are unwanted, therefore during network data is parsed, encapsulation is re-started to the network packet, is gone
Fall unwanted data link layer information, reduce the packet length of network data to be resolved, increase and collect network to be resolved
The capacity of data, reduces influence of the data parsing to router performance, improves the resolution speed of network data.
The specific embodiment of the present invention, when the network data is encapsulated as network data to be resolved, if the net
It is when the data length of the middle application layer data information of network data is more than the first preset value, then described to be resolved in encapsulation process
Network data in the data length of application layer data information that includes correspond to the preset value, the otherwise network to be resolved
The data length of the application layer data information included in data keeps constant.Such as, first preset value is set to 512 bytes,
When the network packet is encapsulated as network data to be resolved, if in the network data application layer data information data
Length is more than 512 bytes, then in data encapsulation process, and the data length of the application layer data information retains 512 bytes, such as
When the data length of application layer data information is less than or equal to 512 byte in really described network data, retain original application
The data length of layer data information.According to the technical scheme, if the data length of the application layer data information in network data
Than larger, and effect of the data message of application layer to analyzing the behavior of network data is little, it is therefore desirable to which intercept part should
Entered with layer data Information encapsulation in network data to be resolved, reduce the size of network data to be resolved, improve router
Performance, is conducive to router to collect the network data to be resolved.
The network data is re-packaged into after network data to be resolved by router, according to a default data acquisition
Mechanism, network data to be resolved, is stored into the data pool of network data to be resolved, when the data described in continuous collecting
The data volume in pond be more than the first pre-set threshold value when, or the acquisition time be more than the second pre-set threshold value when, stop collection.Specifically
Ground, the field of two bytes is provided with the packet header of each network data to be resolved, and the field is used to represent that this is to be resolved
The packet length of network data.Router gathers network data to be resolved, often collects a network data to be resolved, according to
The field value of two bytes in the network data packet header to be resolved, obtains the packet length of the network data to be resolved, statistics
Total packet length of all network packets to be resolved collected, when counting on the network packet to be resolved that is collected
Total packet length be more than the second preset value when, stop data acquisition, and the network packet to be resolved packet header set two
The field of individual byte, the field value represents the packet length of collected network packet to be resolved.When dwell time is more than the
After three preset values, proceed network data acquisition to be resolved, repeat the collection that above-mentioned steps carry out data, will collect
Network packet to be resolved, store into the data pool of network data to be resolved.Router is according to collected to be resolved
Network packet packet header two bytes field value, obtain the packet length of the network packet to be resolved, and count
The packet length of the network packet to be resolved of all collections, obtains the data of the data pool of the network data to be resolved
Amount.When the data pool data volume be more than the first pre-set threshold value when, or the acquisition time be more than the second pre-set threshold value when,
Stop collection.
According to the technical scheme, using the mechanism of data acquisition dead time, that is to say, that collection network data is more than one
Stop collecting after preset value, then continue to gather network data after a preset time, the network data of collection used reaches
After certain data, stop data acquisition, so at regular intervals or accumulation reaches a number of packet, form packet
File is sent to server end to be parsed, and reduces influence of the Data Collection to router performance, improves network data
Resolution speed.
Service end obtains the network data to be resolved that router is sent, and network data to be resolved is solved
Analysis.Due in the gatherer process of network data, the data link layer information included in the network data to be resolved only
Remain each field location in terminal MAC address information, therefore the network data to be resolved and data packet length, and mark
The network data pack arrangement of quasi- agreement is inconsistent, that is to say, that the expression data link layer letter in network data to be resolved
The position of each field of breath, transport layer information and network layer information, the position with each field of standard agreement is inconsistent, it is impossible to
The network data to be resolved is parsed according to the structure of standard agreement, it is therefore desirable to reposition network to be resolved
The packet length of data, obtains the position of each field, the network data is parsed.
Specifically, service end obtains the network data to be resolved, by a field of the network data to be resolved
Value, is designated the packet length of network data to be resolved, and the packet length of packet is obtained according to the field value, is grown according to the bag
Degree, obtains the numerical value of the predeterminated position field in the network data to be resolved, if acquired numerical value meets network number
According to standard agreement structure in same field value, then judge that the banner word segment value of network data to be resolved is to be resolved for this
The packet length of network data, and then the packet length of the network data to be resolved is obtained, according to the packet length, according to network
The standard agreement of data, such as data link layer protocol, transport layer protocol and network layer protocol, and then obtain net to be resolved
Each field value in network data, is parsed layer by layer to the network data, so obtain network data in mac address information,
IP address information and URL information etc., and analyzed according to these information.If acquired numerical value does not meet network data
Same field value in standard agreement structure, then judges the original position of network data to be resolved again, i.e., again described
The packet length of network data to be resolved, repeats the above steps, such as, judge the next of the network data to be resolved
Byte is the original position, and the original position until finding the network data to be resolved obtains network number to be resolved
According to packet length, and the packet length of network data to be resolved is parsed layer by layer.
The specific embodiment of the present invention, in data-gathering process, the data link of the network data to be resolved
The reservation terminal MAC address information of layer information, but the network information in network packet does not change, therefore utilize Internet
The special field value of the standard data structure of agreement, such as version (version) fields and protocol (agreement) field,
Version refers to the version of IP agreement, and the hexadecimal value of the field value is 4, is expressed as IPv4, protocol fields refer to data
What agreement is the data that report is carried be, the corresponding hexadecimal of the field value is 6, and it is Transmission Control Protocol to represent agreement.The present embodiment is utilized
The two field values determine the packet length of the network data to be resolved.Assuming that by before the network data to be resolved
Two bytes obtain the field value of described two bytes as the original position of network data to be resolved, and the field value is used to
Represent the packet length of the network data to be resolved.According to the data structure of the network data to be resolved, obtain described
Version field values and protocol field values in network data to be resolved, if the numerical value and network number of the two fields
According to standard agreement structure in the two field values it is consistent, show two bytes of the network data starting to be resolved,
For the value for the actual length for demarcating network data to be resolved, according to the packet length of the data, according to the mark of network data
Quasi- protocol architecture, and then each field value of the network data to be resolved is obtained, and then to the network data to be resolved
Parsed layer by layer, such as, obtain data-link layer message, transport layer message and network layer information, and then net can be obtained
Mac address information, IP address information and URL information in network data etc., and analyzed according to these information.If described
The two field values in the standard agreement structure of the numerical value and network data of version field values and protocol fields differ
Cause, then show two bytes of the network data starting to be resolved, it is impossible to for demarcating the true of network data to be resolved
The value of true length degree, will also continue to position the original position of network data to be resolved, obtain network data to be resolved
Packet length, obtains the field value of two bytes behind network data to be resolved, repeats the above steps successively, until obtaining institute
The packet length of network data to be resolved is stated, then network data to be resolved is parsed layer by layer.
According to the technical scheme, the original position of the network data to be resolved is positioned, and obtains network to be resolved
The packet length of data, and then the network data to be resolved is parsed layer by layer, obtain the number of network data to be resolved
According to link layer message, transport layer message and network layer information, and then mac address information, IP in network data can be obtained
Address information and URL information etc., and analyzed according to these information, user is used according to these information, to user
Bring great convenience.
The specific embodiment of the present invention, the implementation that the packet header agreement to the network data to be resolved is parsed
Example.The network data to be resolved is stored with document form, by the network packet by decompressing decryption conversion
Into packet data collection, the packet data collection is stored in binary form.Network to be resolved as shown in Figure 2
The data structure diagram of data, preceding 4 bytes of network data to be resolved, the data of the network data to represent the parsing
Length, as shown in the PACKET contents in table 1.The data DATA of the network data to be resolved is message content, wherein
The protocol header of message content is to include various agreement headers, such as, Ethernet header information, IP header informations and TCP header
Portion's information, these header informations are all stored according to certain coded format.By the file by decompressing decryption conversion
Into real packet data collection cathpkt files, then handling to packet data collection cathpkt files, reads
Preceding 4 bytes of packet data collection, binary data are converted to time data as the length information L of packet, from
The data that L-4 byte is read and write in cathpkt files are the data of PACKET contents.Then PACKET data are obtained first
The data of preceding 14 bytes, by upack () function, the byte stream string numbers of this 14 bytes are parsed according to given format
According to, and return to the byte arrays parsed.Preceding 6 bytes and middle 6 byte are obtained, using ord () function pair byte fluxion
According to being handled, corresponding ASCII value is converted into, corresponding user's MAC address information and MAC address of server is then combined into
Information.Specific code implementation process is as follows：
def eth_addr(a):
B=" %.2x:%.2x:%.2x:%.2x:%.2x:%.2x " % (ord (a [0]), ord (a [1]), ord (a
[2]),ord(a[3]),ord(a[4]),ord(a[5]))
return b；
By using above-mentioned same design, the IP agreement header information of network data to be resolved is solved
Analysis, obtains the User IP information and server ip information in network data to be resolved.
Obtain remaining data package informatin in data set cathpkt files to be parsed, repeat the above steps, circular treatment number
According to collection cathpkt files, until all network datas to be resolved are parsed.
The another specific embodiment of the present invention, the implementation parsed to the data content of the network data to be resolved
Example.The head length of the packet header agreement of network data has its corresponding rule, obtains the length HL of data packet head agreement, whole net
The length information of network data exists in the packet header protocol section of network data, and the total length of network data is PL, then network number
Data DATA length in is DL=PL-HL.The data DATA of each network data is read from cathpkt data files
Binary data message, is parsed to the binary data messages of data DATA.By in BaseHTTPServer
BaseHTTPRequestHandler functions expand, parsing data DATA binary data information, and are stored in an example
In attribute set.It is different according to the type of network data, take different data DATA information processing manner.Such as, from
The data DATA information of a network data is obtained in cathpkt files, then data are parsed, instance properties are arrived in storage
In set.The upper items of strength attribute set are extracted, the type of network data classified according to the values of upper, class
Type has GET, POST, HEAD, PUT, DELETE, OPTIONS, TRACE and CONNECT.By taking POST data as an example, i.e.,
Request.command.upper=' post '；
Then network data uses the data processing data information modes of POST data bag.The instance properties knot of POST data bag
In conjunction remove upper outside, other guide stored in the form of dictionary structure URL, user-agent, cookie, refere and
The effective informations such as date.Retrieved by the key assignments to the dictionary structure information, you can obtain the content of all POST datas.Retrieval
Go out URL, user-agent, cookie, refere and date of all POST data bags in attribute instance set, obtain data letter
The user behavior data included in breath.
As shown in figure 3, according to one embodiment of present invention, a kind of router, including：
Data package module 30, for the data link layer information in network data to be retained into terminal MAC address information, and
The network data is re-packaged into network data to be resolved；
Data collection module 31, for according to a default data collecting mechanism, network to be resolved described in continuous collecting
Data, are stored into the data pool of network data to be resolved, when the data volume of the data pool is more than the first pre-set threshold value,
Or the acquisition time be more than the second pre-set threshold value when, stop collection.
Preferably, the data package module 30, is additionally operable to add one or two in the packet header of the network data to be resolved
The field of individual byte, the value of the field is used for the packet length value for setting the network data to be resolved.
Preferably, the data collection module is specifically included：
Statistic unit, for the field value of two bytes according to the network data packet header to be resolved, obtains described
Packet length, and count total packet length of all network packets to be resolved collected；
Judging unit, for when total packet length is more than the second preset value, stopping data acquisition, and wait to solve described
The packet header of the network packet of analysis sets the field of two bytes, and the field value represents collected network packet to be resolved
Total packet length；
The judging unit, is additionally operable to after dwell time is more than three preset values, proceed collection, and by collection
Network packet to be resolved, is stored into the data pool of network data to be resolved.
Preferably, the statistic unit, is additionally operable to count total packet length of the network packet to be resolved of all collections,
Obtain the data volume of the data pool of the network data to be resolved；
The judging unit, is additionally operable to when the data volume of the data pool is more than the first pre-set threshold value, or described adopt
When the collection time is more than the second pre-set threshold value, stop collection.
Router is got after uplink network data, it is necessary to carry out data link layer, Internet and transmission to network data
The encapsulation of layer.In the inventive solutions, there are two data transmission paths in the router, a data transmission path is
Network data is carried out after the completion of complete package, according to the Path First of router, the network data forwarded, in addition one
Data transmission path is network data to be acquired and parsed the network data, and then the network data is divided
Analysis.
Specifically, in data package module 30, when carrying out data link layer information encapsulation to network data, to data
For link layer information, it would be desirable to retain the mac address information of terminal, and the network data is re-started it is encapsulated as treating
The network data of parsing.Since during real network data behavioural analysis, other data message bases of data link layer
It is that our institutes are unwanted on this, so without all data link layer informations are encapsulated into network packet.Router
Before the encapsulation of data link layer and transport layer is carried out to the network packet of reception, in order to complete the forwarding to network data,
Need the source MAC address information in data link layer information being replaced by the mac address information of router, therefore in data
In package module 30, when being encapsulated as network data to be resolved to the network data, according to the MAC Address stored in router
Table, obtains the corresponding terminal MAC address information of the network packet, and the data link layer in the network data is believed
Breath is replaced by terminal MAC address information, is encapsulated as network data to be resolved.And in the packet header of the network data to be resolved
The field of one or two of byte is added, the value of the field is used for the packet length value for setting the network data to be resolved, for table
Show the packet length of the network data to be resolved.
By the technical scheme, the data link layer information in network packet, analysis and user's row to network data
For analysis, it is most that the terminal MAC address information of data link layer information is used, and other data of data link layer are believed
Breath is that our institutes are unwanted, therefore during network data is parsed, encapsulation is re-started to the network packet, is gone
Fall unwanted data link layer information, reduce the packet length of network data to be resolved, increase and collect network to be resolved
The capacity of data, reduces influence of the data parsing to router performance, improves the resolution speed of network data.
The network data is re-packaged into after network data to be resolved, data collection module 31 is default according to one
Data collecting mechanism, network data to be resolved, is stored into the data pool of network data to be resolved described in continuous collecting, when
The data volume of the data pool be more than the first pre-set threshold value when, or the acquisition time be more than the second pre-set threshold value when, stop
Collection.
Specifically, in the statistic unit of data collection module 31, a network data to be resolved is often collected, according to
The field value of two bytes in the network data packet header to be resolved, obtains the packet length of the network data to be resolved, statistics
Total packet length of all network packets to be resolved collected.By the judging unit of data collection module 31, work as statistics
When being more than the second preset value to total packet length of the network packet to be resolved collected, stop data acquisition, and at this
The packet header of network packet to be resolved sets the field of two bytes, and the field value represents collected network number to be resolved
According to the packet length of bag.In judging unit, after dwell time is more than three preset values, proceed network data to be resolved
Collection, repeats the collection that above-mentioned steps carry out data, the network packet to be resolved of collection is stored to be resolved
In the data pool of network data.Then by statistic unit, according to two words in the packet header for collecting network packet to be resolved
The field value of section, obtains the packet length of the network packet to be resolved, and counts the network data to be resolved of all collections
The packet length of bag, obtains the data volume of the data pool of the network data to be resolved.And by judging unit, when the data
The data volume in pond be more than the first pre-set threshold value when, or the acquisition time be more than the second pre-set threshold value when, stop collection.
According to the technical scheme, using the mechanism of data acquisition dead time, that is to say, that collection network data is more than one
Stop collecting after preset value, then continue to gather network data after a preset time, the network data of collection used reaches
After certain data, stop data acquisition, so at regular intervals or accumulation reaches a number of packet, form packet
File is sent to server end to be parsed, and reduces influence of the Data Collection to router performance, improves network data
Resolution speed.
As shown in figure 4, according to one embodiment of present invention, a kind of network data resolver, including：
Router 40, for the data link layer information in network data to be retained into terminal MAC address information, and will be described
Network data is re-packaged into network data to be resolved, according to a default data collecting mechanism, waits to solve described in continuous collecting
The network data of analysis, is stored into the data pool of network data to be resolved, when the data volume of the data pool is pre- more than first
If during threshold values, or the acquisition time be more than the second pre-set threshold value when, stop collection；
Service end 41, for a field value of the network data to be resolved to be designated into network data to be resolved
Packet length, and according to the packet length, the numerical value of a predeterminated position field of the network data to be resolved is obtained, if described
Numerical value meets the field value of the same position in standard network data protocol architecture, then judges the network data word to be resolved
Segment value is the packet length, and the network data to be resolved is parsed layer by layer.
The analysis method and device and router of a kind of network data of the present invention, reduce Data Collection to router performance
Influence, improves the resolution speed of network data, obtains the information in network data, and is analyzed according to these information, uses
Family is used according to these information, is brought great convenience to user.
It should be noted that above-described embodiment can independent assortment as needed.Described above is only the preferred of the present invention
Embodiment, it is noted that for those skilled in the art, is not departing from the premise of the principle of the invention
Under, some improvements and modifications can also be made, these improvements and modifications also should be regarded as protection scope of the present invention.
Claims (10)
1. a kind of analytic method of network data, it is characterised in that comprise the following steps：
S1, the data link layer information in network data retained into terminal MAC address information, and the network data is sealed again
Fill as network data to be resolved；
S2, according to a default data collecting mechanism, network data to be resolved, is stored to net to be resolved described in continuous collecting
In the data pool of network data, when the data volume of the data pool is more than the first pre-set threshold value, or the acquisition time is more than
During the second pre-set threshold value, stop collection；
S3, the packet length that a field value of the network data to be resolved is designated to network data to be resolved, and according to
The packet length, obtains the numerical value of a predeterminated position field of the network data to be resolved, if the numerical value meets standard
The field value of same position in network data protocol architecture, then judge the network data field value to be resolved for the bag
Length, and the network data to be resolved is parsed layer by layer；
S4, otherwise, repeats step S3.
2. the analytic method of network data as claimed in claim 1, it is characterised in that step S1 also includes：
The field of one or two of byte is added in the packet header of the network data to be resolved, the value of the field is used to set described treat
The packet length value of the network data of parsing.
3. the analytic method of network data as claimed in claim 1, it is characterised in that step S1 also includes：
It is described to be resolved if the data length of the middle application layer data information of the network data is more than the first preset value
The data length of the application layer data information included in network data corresponds to the preset value, otherwise the network number to be resolved
The data length of the application layer data information included in keeps constant.
4. the analytic method of network data as claimed in claim 1, it is characterised in that step S2 is specifically included：
According to the field value of two bytes in the network data packet header to be resolved, the packet length is obtained, and count all
Total packet length of the network packet to be resolved collected；
When total packet length is more than the second preset value, stop data acquisition, and in the network packet to be resolved
Packet header sets the field of two bytes, and the field value represents total packet length of collected network packet to be resolved；
After dwell time is more than three preset values, proceed collection, and by the network packet to be resolved of collection, store
Into the data pool of network data to be resolved.
5. the analytic method of network data as claimed in claim 4, it is characterised in that step S2 specifically also includes：
Total packet length of the network packet to be resolved of all collections is counted, the data of the network data to be resolved are obtained
The data volume in pond, when the data volume of the data pool is more than the first pre-set threshold value, or the acquisition time is pre- more than second
If during threshold values, stopping collection.
6. a kind of router, it is characterised in that including：
Data package module, for the data link layer information in network data to be retained into terminal MAC address information, and will be described
Network data is re-packaged into network data to be resolved；
Data collection module, for according to a default data collecting mechanism, network data to be resolved, to be deposited described in continuous collecting
Storage is into the data pool of network data to be resolved, when the data volume of the data pool is more than the first pre-set threshold value, Huo Zhesuo
When stating acquisition time more than the second pre-set threshold value, stop collection.
7. router as claimed in claim 6, it is characterised in that
The data package module, is additionally operable to add the field of one or two of byte in the packet header of the network data to be resolved,
The value of the field is used for the packet length value for setting the network data to be resolved.
8. router as claimed in claim 7, it is characterised in that the data collection module is specifically included：
Statistic unit, for the field value of two bytes according to the network data packet header to be resolved, obtains the bag length
Degree, and count total packet length of all network packets to be resolved collected；
Judging unit, for when total packet length is more than the second preset value, stopping data acquisition, and described to be resolved
The packet header of network packet sets the field of two bytes, and the field value represents the total of collected network packet to be resolved
Packet length；
The judging unit, be additionally operable to when dwell time be more than three preset values after, proceed collection, and by collection wait solve
The network packet of analysis, is stored into the data pool of network data to be resolved.
9. router as claimed in claim 8, it is characterised in that
The statistic unit, is additionally operable to count total packet length of the network packet to be resolved of all collections, obtain described in treat
The data volume of the data pool of the network data of parsing；
The judging unit, is additionally operable to when the data volume of the data pool is more than the first pre-set threshold value, or during the collection
Between be more than the second pre-set threshold value when, stop collection.
10. a kind of network data resolver, it is characterised in that including：
Router, for the data link layer information in network data to be retained into terminal MAC address information, and by the network number
According to being re-packaged into network data to be resolved, according to a default data collecting mechanism, net to be resolved described in continuous collecting
Network data, are stored into the data pool of network data to be resolved, when the data volume of the data pool is more than the first pre-set threshold value
When, or the acquisition time be more than the second pre-set threshold value when, stop collection；
Service end, the bag for a field value of the network data to be resolved to be designated to network data to be resolved is long
Degree, and according to the packet length, the numerical value of a predeterminated position field of the network data to be resolved is obtained, if the numerical value
Meet the field value of the same position in standard network data protocol architecture, then judge the network data field value to be resolved
For the packet length, and the network data to be resolved is parsed layer by layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710482223.XA CN107147674A (en) | 2017-06-22 | 2017-06-22 | The analytic method and router and device of a kind of network data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710482223.XA CN107147674A (en) | 2017-06-22 | 2017-06-22 | The analytic method and router and device of a kind of network data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107147674A true CN107147674A (en) | 2017-09-08 |
Family
ID=59782956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710482223.XA Pending CN107147674A (en) | 2017-06-22 | 2017-06-22 | The analytic method and router and device of a kind of network data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107147674A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113608504A (en) * | 2021-04-21 | 2021-11-05 | 北京智慧空间科技有限责任公司 | Self-adaptive wind field information acquisition method and device, medium and wind power system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7539750B1 (en) * | 2004-03-30 | 2009-05-26 | Extreme Networks, Inc. | System and method for packet processor status monitoring |
| CN101527654A (en) * | 2009-04-20 | 2009-09-09 | 中兴通讯股份有限公司 | Data transmission method and system in network management system |
| CN101741656A (en) * | 2008-11-12 | 2010-06-16 | 上海摩波彼克半导体有限公司 | Method for realizing optimization of point-to-point protocol data transmission of mobile communication wireless Modem terminal |
| CN101998508A (en) * | 2009-08-14 | 2011-03-30 | 华为技术有限公司 | Data encapsulation method and device |
| CN104640130A (en) * | 2015-03-12 | 2015-05-20 | 成都金本华科技股份有限公司 | Rapid transfer method for message |
-
2017
- 2017-06-22 CN CN201710482223.XA patent/CN107147674A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7539750B1 (en) * | 2004-03-30 | 2009-05-26 | Extreme Networks, Inc. | System and method for packet processor status monitoring |
| CN101741656A (en) * | 2008-11-12 | 2010-06-16 | 上海摩波彼克半导体有限公司 | Method for realizing optimization of point-to-point protocol data transmission of mobile communication wireless Modem terminal |
| CN101527654A (en) * | 2009-04-20 | 2009-09-09 | 中兴通讯股份有限公司 | Data transmission method and system in network management system |
| CN101998508A (en) * | 2009-08-14 | 2011-03-30 | 华为技术有限公司 | Data encapsulation method and device |
| CN104640130A (en) * | 2015-03-12 | 2015-05-20 | 成都金本华科技股份有限公司 | Rapid transfer method for message |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113608504A (en) * | 2021-04-21 | 2021-11-05 | 北京智慧空间科技有限责任公司 | Self-adaptive wind field information acquisition method and device, medium and wind power system |
| CN113608504B (en) * | 2021-04-21 | 2022-07-19 | 北京智慧空间科技有限责任公司 | Self-adaptive wind field information acquisition method and device, medium and wind power system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112468370B (en) | High-speed network message monitoring and analyzing method and system supporting custom rules | |
| JP5961354B2 (en) | Method and apparatus for efficient netflow data analysis | |
| CN106815112A (en) | A kind of mass data monitoring system and method based on deep-packet detection | |
| CN103281213B (en) | A kind of network traffic content extracts and analyzes search method | |
| CN103268350B (en) | Internet public opinion information monitoring system and monitoring method | |
| CN109089029B (en) | FPGA-based Gige Vision interface image transmission system and method | |
| US20040083299A1 (en) | Method and apparatus for monitoring traffic in a network | |
| CN102801714B (en) | Method for analyzing and reducing SQL (Structured Query Language) command in TNS (Transparent Network Substrate) protocol in by-pass manner | |
| CN108833299A (en) | A kind of large scale network data processing method based on restructural exchange chip framework | |
| CN106878409A (en) | A kind of game data processing system and processing method | |
| CN1897541A (en) | Method for realizing network sampling | |
| CN107862074A (en) | Big data quantity parameter rapid read-write method | |
| CN108023767A (en) | Internet behavior method for tracing, device and server | |
| CN106921665A (en) | A kind of message processing method and the network equipment | |
| Fusco et al. | pcapIndex: an index for network packet traces with legacy compatibility | |
| CN114327833A (en) | Efficient flow processing method based on software-defined complex rule | |
| CN107147674A (en) | The analytic method and router and device of a kind of network data | |
| CN112231320B (en) | Web data acquisition method, system and storage medium based on MapReduce algorithm | |
| CN107277109A (en) | Multi-string matching method for compressing flow | |
| CN112069305B (en) | Data screening method and device and electronic equipment | |
| CN103354546A (en) | Message filtering method and message filtering apparatus | |
| JP4491577B2 (en) | Log summarization device, log summarization program, and recording medium | |
| CN103532779B (en) | A kind of method and system of quick positioning shunting device packet loss | |
| CN106648912A (en) | Modular method and apparatus for data processing in data acquisition platform | |
| CN106326280A (en) | Data processing method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170908 |
|
| WD01 | Invention patent application deemed withdrawn after publication |