CN105938562B - A kind of automated network employing fingerprint extracting method and system - Google Patents
A kind of automated network employing fingerprint extracting method and system Download PDFInfo
- Publication number
- CN105938562B CN105938562B CN201610230313.5A CN201610230313A CN105938562B CN 105938562 B CN105938562 B CN 105938562B CN 201610230313 A CN201610230313 A CN 201610230313A CN 105938562 B CN105938562 B CN 105938562B
- Authority
- CN
- China
- Prior art keywords
- byte
- vector
- network
- employing fingerprint
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 239000013598 vector Substances 0.000 claims abstract description 61
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 239000000284 extract Substances 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000013528 artificial neural network Methods 0.000 claims description 16
- 230000000306 recurrent effect Effects 0.000 claims description 16
- 230000000750 progressive effect Effects 0.000 claims description 13
- 238000002887 multiple sequence alignment Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 11
- 108091035707 Consensus sequence Proteins 0.000 claims description 7
- 230000006798 recombination Effects 0.000 claims description 7
- 238000005215 recombination Methods 0.000 claims description 7
- 238000002864 sequence alignment Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 238000003064 k means clustering Methods 0.000 claims description 4
- 238000003780 insertion Methods 0.000 claims description 3
- 230000037431 insertion Effects 0.000 claims description 3
- 210000004218 nerve net Anatomy 0.000 claims 1
- 230000006870 function Effects 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000013467 fragmentation Methods 0.000 description 2
- 238000006062 fragmentation reaction Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 101001094649 Homo sapiens Popeye domain-containing protein 3 Proteins 0.000 description 1
- 101000608234 Homo sapiens Pyrin domain-containing protein 5 Proteins 0.000 description 1
- 101000578693 Homo sapiens Target of rapamycin complex subunit LST8 Proteins 0.000 description 1
- 235000006508 Nelumbo nucifera Nutrition 0.000 description 1
- 240000002853 Nelumbo nucifera Species 0.000 description 1
- 235000006510 Nelumbo pentapetala Nutrition 0.000 description 1
- 102100027802 Target of rapamycin complex subunit LST8 Human genes 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008521 reorganization Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Machine Translation (AREA)
Abstract
The present invention provides a kind of automated network employing fingerprint extracting method, and step includes: 1) to receive network application data packet, passes through language model and learns output byte vector；2) byte vector corresponding to the byte-by-byte value to above-mentioned network application data payload package is attached operation, obtains data pack load vector；3) according to above-mentioned data pack load vector, above-mentioned network application data payload package is divided into multiple and different cluster；4) to each cluster in the byte sequence strings of frequent co-occurrences a series of extract, obtain employing fingerprint.The present invention also provides a kind of automated network employing fingerprint extraction systems, comprising: data preprocessing module, byte vector study module, data pack load vectorization representation module, data pack load cluster module and employing fingerprint extraction module.
Description
Technical field
This patent belongs to technical field of network security, is related to network application fingerprint extraction technology, in particular to a kind of automatic
Change network application fingerprint extraction method and system.
Background technique
DPI technology is widely used in industry, is of great significance, and the key technology difficulty of DPI is that network application refers to
Line extracts.Existing network application fingerprint extraction method mainly includes artificial extraction and automation extracting method, wherein artificial side
Method manpower is at high cost, low efficiency；And there is also following deficiencies for automatic method: first is that extracted fingerprint quality is unable to satisfy reality
The requirements for high precision of border application identification, second is that it is higher to required training set data quality requirement, it is more difficult in application practice to obtain
It takes.Therefore, this patent discloses a kind of automated network employing fingerprint extracting method and system based on byte vector.
Summary of the invention
In order to overcome above-mentioned deficiency, the present invention provides a kind of automated network employing fingerprint extracting method and system, base
Learn in byte vector, the degree of dependence of required training set quality is low, and the employing fingerprint quality of extraction is high.
In order to solve the above-mentioned technical problem, the technology used in the present invention method is:
A kind of automated network employing fingerprint extracting method, step include:
1) network application data packet is received, output byte vector is learnt by language model；
2) byte vector corresponding to the byte-by-byte value to above-mentioned network application data payload package is attached operation, obtains
To data pack load vector；
3) according to above-mentioned data pack load vector, above-mentioned network application data payload package is divided into multiple and different gather
Cluster；
4) to each cluster in the byte sequence strings of frequent co-occurrences a series of extract, obtain employing fingerprint.
Further, the network application data packet is the sequence of data packet as unit of one-way flow, passes through primitive network
Data packet carries out IP recombination and TCP flow restoring operation obtains.
Further, the language model is Recognition with Recurrent Neural Network language model, passes through the network application data packet
Sample database building, the probabilistic model for being used to calculate given byte sequence probability of occurrence for one.
Further, the Recognition with Recurrent Neural Network language model is made of input layer, hidden layer and output layer.
Further, the byte vector is a regular length for portraying the real-valued vectors of byte value, and initial value is adopted
With 1-of-n coded representation.
Further, according to K-Means clustering algorithm, the network application data payload package is divided into multiple and different
It clusters.
Further, the Clustering Effect to cluster is assessed by silhouette coefficient.
Further, according to progressive Multiple Sequence Alignment algorithm, to each cluster in a series of frequent co-occurrences syllable sequence
Column string extracts, which is based on pairs of sequence alignment algorithms.
Further, it is described to each cluster in the byte sequence strings of frequent co-occurrences a series of extract and refer to, from
Consensus sequence is extracted in progressive Multiple Sequence Alignment result, the consensus sequence is by the highest byte of the frequency of occurrences on each position
Value sequentially forms, and is split further according to the gap of insertion to the consensus sequence, retains the byte sequence that length is greater than 2.
A kind of automated network employing fingerprint extraction system, including following module:
One data preprocessing module, receive raw network data packet, carry out IP recombination and TCP flow restoring operation, output with
One-way flow is the sequence of data packet of unit；
One byte vector study module is received and is extracted to the sequence of data packet as unit of one-way flow to construct instruction
Practice collection, exports the byte vector learnt by Recognition with Recurrent Neural Network language model；
One data pack load vectorization representation module carries out network application data payload package by byte vector byte-by-byte
The attended operation of the corresponding byte vector of value, output data packet load vectors；
One data pack load cluster module, according to data pack load vector, by executing K-Means clustering algorithm, by net
Network application data pack load is divided into multiple and different cluster；
One employing fingerprint extraction module, to each cluster in network application data payload package carry out progressive multisequencing ratio
To algorithm, a series of byte sequence string of frequent co-occurrences is exported as employing fingerprint.
The invention has the advantages that compared with prior art, the present invention is capable of providing the automation of network application fingerprint
It extracts, solves the problems, such as artificial extraction cost height, low efficiency；Extracted employing fingerprint quality is higher, that is, using should
The DPI method of fingerprint has higher network application detection accuracy, and lower to the degree of dependence of required training set quality.
Detailed description of the invention
Fig. 1 is a kind of frame diagram of automated network employing fingerprint extracting method.
Fig. 2 is the language model schematic diagram of the Recognition with Recurrent Neural Network at 3 moment of expansion.
Fig. 3 is a kind of frame diagram of automated network employing fingerprint extraction system.
Fig. 4 is the Precision contrast and experiment of ProDigger of the present invention and prior art ProWord.
Fig. 5 is the Recall contrast and experiment of ProDigger of the present invention and prior art ProWord.
Specific embodiment
To enable features described above and advantage of the invention to be clearer and more comprehensible, special embodiment below, and institute's attached drawing is cooperated to make
Detailed description are as follows.
The present embodiment provides a kind of automated network employing fingerprint extracting methods, and there are four core procedures: based on circulation mind
Byte vector study through netspeak model, load vectors expression, load cluster division and employing fingerprint extracts, such as Fig. 1
It is shown.
(1) the byte vector study based on Recognition with Recurrent Neural Network language model: the language of network-oriented application data pack load
Speech model refers to the probabilistic model for calculating given byte sequence probability of occurrence in network application data payload package, needs base
It is constructed in by pretreated network application data payload package sample database.The present invention utilizes Recognition with Recurrent Neural Network (Recurrent
Neural Network, RNN) language model, i.e. Recognition with Recurrent Neural Network language model are described.In Recognition with Recurrent Neural Network language mould
In type, input layer is a series of byte vector, during constructing language model, these byte vectors from given initial value not
Disconnected iteration update finally tends towards stability, i.e., byte vector learns from Recognition with Recurrent Neural Network language model and obtained.Formally, one
Sequence { the ω of section T byte0..., ωt-1, ωt,ωt+1... ωT-1Probability of occurrence be defined as follows:
As can be seen from the above formula that key is P (ωt+1|ω0..., ωt-1, ωt) probability distribution calculating, this point
Cloth indicates to give a byte sequenceCalculate next byte ωT+1The item of all possibility values
Part probability distribution.Calculate P (ωt+1|ω0..., ωt-1, ωt) formula are as follows:
Here,Indicating that t+1 byte (calculates) value since the 0th byte is that index ID is k's in table of bytes
Value.
As shown in Fig. 2, Recognition with Recurrent Neural Network language model is made of input layer, hidden layer and output layer.Input layer by
Two parts are constituted, and a part is the input byte value ω of current time ttCorresponding byte vector xt, initial value is 1-of-n coding
It indicates (also with regard to One hot representation), i.e. the vector that a dimension is byte table size, the position table that value is 1
Show index ID of the current input bytes section value in table of bytes；Another part is the historical context letter that last moment t-1 is stored
Cease vector ht-1.Hidden layer is then with xtWith ht-1For input, sigmoid is activation primitive, generates history context information vector ht。
Output layer is also the vector that a dimension is byte table sizeEach dimension indicates that next byte value is index ID institute's generation
The probability of the byte value of table.
Fig. 2 indicates the Recognition with Recurrent Neural Network language model schematic diagram of 3 moment (3times-step) of an expansion, the net
The formal definitions of network structure are as follows:
ht=sigmoid (Whxxt+Whhht-1)
Wherein, Whx、Whh、WhyIt is weight matrix between each layer, softmax is activation primitive.In Recognition with Recurrent Neural Network language
The cross entropy loss function based on desired output and reality output is used when the training of model, in moment t, for output layer
K-th of neuron, desired outputReality outputThen its cross entropy loss function is defined as:
In turn, for reality output vectorWith desired output vector ytBetween cross entropy loss function be defined as follows:
Further, since desired output vector is a 1-of-n vector, wherein the position for 1 is current training input word
The index ID of next byte value of section value, target loss function (i.e. above-mentioned cross entropy loss function) become:
The present invention carries out the training of RNN using BPTT algorithm, and BPTT algorithm is BP algorithm one extension.Therefore pass through expansion
When the τ moment, in moment t, maximized target loss function to become:
Due to only working as ωt+1=kzWhen,1 is taken, otherwise takes 0, therefore objective function simplifies are as follows:
Here,Next byte ωt-z+1Value is that index ID is kzByte value probability.
Wherein α is learning rate, and the calculation formula of above-mentioned partial derivative is as follows:
(2) load vectorsization indicate: the step is based on the byte vector that step (1) learns in advance and is obtained by attended operation
The vectorization for obtaining network application data payload package indicates, that is, obtains the data pack load vector of network application data packet.Given one
A data pack load p, vectorization xp(i.e. data pack load vector) formal definitions are as follows:
Wherein,It is attended operation symbol, x1:tIndicate according to payload package p the 1st byte value to t byte value correspond to byte to
The connection of amount.Since byte vector captures the contextual information of data pack load, it is formed by connecting based on byte vector
Data pack load vector also has same information, and then can be used in the otherness between metric data payload package and similar
Property.
(3) load clusters division: after the vectorization expression for obtaining network application data payload package by step (2), leading to
It crosses and disposes unsupervised K-Means clustering algorithm network application data payload package is assembled and divided, to obtain multiple and different
It clusters, the middle data pack load that each clusters has high similarity in syntactic structure and semantic information, and clusters it in difference
Between data pack load then otherness is larger, this excludes noise for next step (4) progressive Multiple Sequence Alignment algorithm, is conducive to
To better comparison result.In order to determine the number to cluster, it is used as and is commented using silhouette coefficient (SilhouetteCoefficient)
Valence index, the index can be used in the assessment of no label data Clustering Effect, mainly for each data pack load sample i's
The quality of cluster result is measured, and calculation formula is as follows:
Wherein, a (i) indicates the mean difference degree of data pack load sample i with other samples in same cluster, b
(i) indicate the minimum value of the mean difference degree of data pack load sample i and other middle samples that cluster, s (i) value -1~1 it
Between, value is bigger, indicates that Clustering Effect is better.By calculating the silhouette coefficient of all data pack load samples, and it is averaged
The overall profile coefficient of as current cluster result.
(4) employing fingerprint extract: the step be based on previous step (3) obtain purity is high it is multiple cluster after, further
Progressive Multiple Sequence Alignment algorithm is disposed in each cluster extracts a series of frequent co-occurrence byte sequence strings as employing fingerprint.
The basic thought of progressive Multiple Sequence Alignment algorithm is to carry out sequence alignment in pairs to all sequences first and calculate its phase
Like property score value, a guidance tree, and then the relationship between sequences parent provided according to the guidance tree then are constructed according to similarity score values
Density is gradually introducing neighbouring sequence and constantly rebuilds comparison, Zhi Daosuo in order since most close two sequences
There is sequence to be all added into position.The key foundation algorithm of progressive Multiple Sequence Alignment algorithm is pairs of sequence alignment algorithms, the step
The similarity scoring function calculating byte in pairs of sequence alignment algorithms is carried out using the byte vector of step (1) output
It improves, the basic thought of pairs of sequence alignment algorithms is the most ratio of greater inequality calculated between two sequences using Dynamic Programming Idea
Right, i.e., between two sequences of the calculating of iteration similarity score values, and be stored in a score matrix, then obtained according to this
Sub-matrix recalls optimal comparison out.Formally, two byte sequence { a are given1, a2..., aiAnd { b1, b2..., bj, in pairs
The purpose of alignment algorithm gradually fills score matrix M, each unit M of the matrixI, jCalculation formula it is as follows:
Wherein, SI, jIt is byte aiWith byte bjSimilarity scoring function, d be for be inserted into one ' gap ' penalty value.
Similarity score values of the present invention between calculating byte x and byte y pass through following formula:
Here e () indicates that the byte vector of corresponding byte value, m indicate that bytes match rewards score value, and u is that byte mismatches
Penalty score.After obtaining Multiple Sequence Alignment result, consensus sequence is extracted from comparison result, consensus sequence is by each
The highest byte value of the frequency of occurrences sequentially forms on position, and then the gap according to insertion is split consensus sequence, and protects
Stay byte sequence of the length greater than 2 as employing fingerprint.Above-mentioned progressive Multiple Sequence Alignment algorithm is to calculate in biological field extensively
Using and mature algorithm, this step the algorithm extend and improve to be suitable for problem handled by the present invention.
In conjunction with above-mentioned automated network employing fingerprint extracting method, the present embodiment provides a kind of automated network employing fingerprints
Extraction system is contained by data preprocessing module, byte vector study module, data pack load vectorization representation module, data
Lotus cluster module and five part of employing fingerprint extraction module are constituted.
1) data preprocessing module: this module receives raw network data packet, and pre-processes to it, i.e. progress IP weight
Group and TCP flow restoring operation export the sequence of data packet as unit of one-way flow.In the network system based on ICP/IP protocol cluster
In structure, since data link layer can be passed using MTU (Maximum TransmissionUnit, maximum transmission unit) limitation
The size of transmission of data packet, therefore when the size of network host transmitting terminal data packet is greater than MTU, it is necessary to Fragmentation is carried out,
Reorganization operation correspondingly is carried out in network host receiving end, IP fragmentation recombination depends on 16 bit identifications of IP data packet header.TCP
Agreement is connection-oriented reliable transport protocol, and the IP agreement of TCP lower layer is the unreliable protocol of Message Oriented, this will lead
Cause IP cannot be guaranteed TCP message reliably, be sequentially transmitted.In order to solve this problem, TCP takes sliding window mechanism, byte
Number mechanism and Fast retransmission algorithm mechanism etc. are flowed to guarantee the reliable transmission of data.Therefore, it is needed in network host receiver section
TCP session reduction is carried out according to TCP stem sequence number recombination IP data packet.
2) byte vector study module: this module receives the network uni-directional stream by IP recombination and TCP flow reduction, then right
Former a data pack loads of one-way flow are extracted to construct training set, and then are carried out based on Recognition with Recurrent Neural Network language model
Iterative learning output byte vector.
3) data pack load vectorization representation module: this module is right based on the byte vector that previous module learns
Network application data payload package carries out the attended operation that byte-by-byte value corresponds to byte vector, output data packet load vectors.
4) data pack load cluster module: this module received data packet load vectors are calculated by executing K-Means cluster
Method is assembled and is divided to network application data payload package, exports the multiple and different of high-purity and clusters.
5) employing fingerprint extraction module: this module it is resulting to a upper module each cluster in network application data
Payload package carries out progressive Multiple Sequence Alignment algorithm and calculates, by analysing and comparing as a result, exporting a series of syllable sequence of frequent co-occurrences
Column string is exported as employing fingerprint.
Fig. 4, Fig. 5 are (i.e. ProDigger) of the invention and prior art ProWord (Zhuo Zhang, Zhibin
Zhang,P.P.C.Lee,Yunjie Liu and Gaogang Xie,"ProWord:An unsupervised approach
to protocol feature word extraction,"INFOCOM,2014Proceedings IEEE,Toronto,ON,
2014, pp.1393-1401.doi:10.1109/INFOCOM.2014.6848073) experimental result compares.The experiment is to be based on
Five kinds of widely used network application-level protocols of HTTP, POP3, SSL, DNS, PPStream are primarily based on every kind of agreement
The automation that ProDigger and ProWord carries out network application fingerprint respectively is extracted, and then extracted employing fingerprint is disposed
It carries out to L7-filter using identification.The building of data set and the assessment content of experimental result are as follows:
1) for each the network application agreement assessed, training set and test set the building of data set: are constructed
Two set.Training set is used for the extraction of employing fingerprint, is all made of, does not include the data pack load positive sample of certain agreement
The data pack load of any other agreement；Test set is used for application identification, the mixing comprising target protocol positive sample and negative sample
Data set.
2) assessment of experimental result: evaluation index used in this experiment is four differences exported from two-value classifier
Prediction result set be derived, four prediction result set are as follows:
True Positives (TP): the stream in the set is by the correctly predicted stream for positive sample of protocol identification system.
False Positives (FP): the stream in the set is the stream by protocol identification system error prediction for positive sample.
True Negatives (TN): the stream in the set is by the correctly predicted stream for negative sample of protocol identification system.
False Negatives (FN): the stream in the set is the stream by protocol identification system error prediction for negative sample.
Based on above-mentioned four kinds of prediction result set, using accuracy rate widely used in numerous research fields
(Precision) and two kinds of evaluation indexes of recall rate (Recall) carry out verifying assessment.Accuracy rate, which refers to, to be correctly predicted
The ratio of the sample number of positive sample and the sample number for being predicted to be positive sample in total；Recall rate refers to the positive sample being correctly predicted
The ratio of this sample number and the sample number of true positive sample；The formal definitions of evaluation index are as follows:
It can be seen from the figure that method provided by the present invention is not reducing compared with ProWord method or even is having promotion to recall
Accuracy rate can be dramatically increased in the case where rate, demonstrate effectiveness of the invention and advantage.
Compared with prior art, the automation that the present invention is capable of providing network application fingerprint is extracted, and is efficiently solved artificial
The problem of extraction cost height, low efficiency；Extracted employing fingerprint quality is higher, i.e., is had more using the DPI method of the fingerprint
High network application detection accuracy, and former a data packets of network application data stream are only relied upon, without the entire stream of caching
Entire packet, i.e., it is lower to the degree of dependence of required training set quality.
Claims (10)
1. a kind of automated network employing fingerprint extracting method, step include:
1) network application data packet is received, output byte vector is learnt by language model；
2) byte vector corresponding to the byte-by-byte value to above-mentioned network application data payload package is attached operation, is counted
According to payload package vector；
3) according to above-mentioned data pack load vector, above-mentioned network application data payload package is divided into multiple and different cluster；
4) according to progressive Multiple Sequence Alignment algorithm, to each cluster in the byte sequence strings of frequent co-occurrences a series of mention
It takes, obtains employing fingerprint.
2. automated network employing fingerprint extracting method according to claim 1, which is characterized in that the network application number
According to the sequence of data packet that packet is as unit of one-way flow, IP recombination and TCP flow restoring operation are carried out by raw network data packet
It obtains.
3. automated network employing fingerprint extracting method according to claim 1, which is characterized in that the language model is
Recognition with Recurrent Neural Network language model is constructed by the sample database of the network application data packet, is used to calculate given byte for one
The probabilistic model of sequence probability of occurrence.
4. automated network employing fingerprint extracting method according to claim 3, which is characterized in that the circulation nerve net
Network language model is made of input layer, hidden layer and output layer.
5. automated network employing fingerprint extracting method according to claim 1, which is characterized in that the byte vector is
The real-valued vectors for being used to portray byte value of one regular length.
6. automated network employing fingerprint extracting method according to claim 1, which is characterized in that poly- according to K-Means
The network application data payload package is divided into multiple and different cluster by class algorithm.
7. automated network employing fingerprint extracting method according to claim 1 or 6, which is characterized in that described to cluster
Clustering Effect is assessed by silhouette coefficient.
8. automated network employing fingerprint extracting method according to claim 1, which is characterized in that the progressive multisequencing
Alignment algorithm is based on pairs of sequence alignment algorithms.
9. automated network employing fingerprint extracting method according to claim 1 or 8, which is characterized in that described to each
It is a cluster in the byte sequence strings of frequent co-occurrences a series of extract and refer to, extracted from progressive Multiple Sequence Alignment result consistent
Property sequence, which is sequentially made of the highest byte value of the frequency of occurrences on each position, further according to gap pairs of insertion
The consensus sequence is split, and retains the byte sequence that length is greater than 2.
10. a kind of automated network employing fingerprint extraction system, including following module:
One data preprocessing module receives raw network data packet, carries out IP recombination and TCP flow restoring operation, exports with unidirectional
Stream is the sequence of data packet of unit；
One byte vector study module receives and extracts the sequence of data packet as unit of one-way flow to construct training
Collection exports the byte vector learnt by Recognition with Recurrent Neural Network language model；
One data pack load vectorization representation module carries out byte-by-byte value institute to network application data payload package by byte vector
The attended operation of corresponding byte vector, output data packet load vectors；
One data pack load cluster module is answered network by executing K-Means clustering algorithm according to data pack load vector
Multiple and different cluster is divided into data pack load；
One employing fingerprint extraction module, to each cluster in network application data payload package carry out progressive Multiple Sequence Alignment calculation
Method exports a series of byte sequence string of frequent co-occurrences as employing fingerprint.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610230313.5A CN105938562B (en) | 2016-04-13 | 2016-04-13 | A kind of automated network employing fingerprint extracting method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610230313.5A CN105938562B (en) | 2016-04-13 | 2016-04-13 | A kind of automated network employing fingerprint extracting method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105938562A CN105938562A (en) | 2016-09-14 |
| CN105938562B true CN105938562B (en) | 2019-06-11 |
Family
ID=57151315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610230313.5A Expired - Fee Related CN105938562B (en) | 2016-04-13 | 2016-04-13 | A kind of automated network employing fingerprint extracting method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105938562B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107550500B (en) * | 2017-08-30 | 2020-06-12 | 西南交通大学 | Method and system for testing response inhibition capability of high-speed rail dispatcher |
| CN115766204B (en) * | 2022-11-14 | 2024-04-26 | 电子科技大学 | Dynamic IP equipment identification system and method for encrypted traffic |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7492928B2 (en) * | 2003-02-25 | 2009-02-17 | Activcard Ireland Limited | Method and apparatus for biometric verification with data packet transmission prioritization |
| CN102164049A (en) * | 2011-04-28 | 2011-08-24 | 中国人民解放军信息工程大学 | Universal identification method for encrypted flow |
| CN102420830A (en) * | 2010-12-16 | 2012-04-18 | 北京大学 | Peer-to-peer (P2P) protocol type identification method |
| CN102480677A (en) * | 2010-11-24 | 2012-05-30 | 中国移动通信集团公司 | Determination method of fingerprint positioning error and device thereof |
| CN104104659A (en) * | 2013-04-07 | 2014-10-15 | 中国科学技术大学 | Communication fingerprint extraction method and device |
| CN104463188A (en) * | 2014-10-24 | 2015-03-25 | 上海交通大学 | Cluster universality analysis method based on optimum transmission model |
| CN105303449A (en) * | 2015-11-09 | 2016-02-03 | 北京交通大学 | Social network user identification method based on camera fingerprint features and system thereof |
-
2016
- 2016-04-13 CN CN201610230313.5A patent/CN105938562B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7492928B2 (en) * | 2003-02-25 | 2009-02-17 | Activcard Ireland Limited | Method and apparatus for biometric verification with data packet transmission prioritization |
| CN102480677A (en) * | 2010-11-24 | 2012-05-30 | 中国移动通信集团公司 | Determination method of fingerprint positioning error and device thereof |
| CN102420830A (en) * | 2010-12-16 | 2012-04-18 | 北京大学 | Peer-to-peer (P2P) protocol type identification method |
| CN102164049A (en) * | 2011-04-28 | 2011-08-24 | 中国人民解放军信息工程大学 | Universal identification method for encrypted flow |
| CN104104659A (en) * | 2013-04-07 | 2014-10-15 | 中国科学技术大学 | Communication fingerprint extraction method and device |
| CN104463188A (en) * | 2014-10-24 | 2015-03-25 | 上海交通大学 | Cluster universality analysis method based on optimum transmission model |
| CN105303449A (en) * | 2015-11-09 | 2016-02-03 | 北京交通大学 | Social network user identification method based on camera fingerprint features and system thereof |
Non-Patent Citations (1)
| Title |
|---|
| 基于字节熵矢量加权指纹的二进制协议识别;黄笑言等;《计算机应用研究 》;20150228;第32卷(第2期);第493-497页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105938562A (en) | 2016-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110298037B (en) | Convolutional neural network matching text recognition method based on enhanced attention mechanism | |
| WO2019144521A1 (en) | Deep learning-based malicious attack detection method in traffic cyber physical system | |
| CN112084790A (en) | Relation extraction method and system based on pre-training convolutional neural network | |
| CN109831422B (en) | Encrypted flow classification method based on end-to-end sequence network | |
| CN111985581B (en) | Sample-level attention network-based few-sample learning method | |
| CN112669916A (en) | Molecular diagram representation learning method based on comparison learning | |
| CN109614614A (en) | A kind of BILSTM-CRF name of product recognition methods based on from attention | |
| CN111368920A (en) | Quantum twin neural network-based binary classification method and face recognition method thereof | |
| CN102289522A (en) | Method of intelligently classifying texts | |
| CN111078895B (en) | Remote supervision entity relation extraction method based on denoising convolutional neural network | |
| CN105938562B (en) | A kind of automated network employing fingerprint extracting method and system | |
| Zhao et al. | Towards unknown traffic identification via embeddings and deep autoencoders | |
| CN109376797B (en) | Network traffic classification method based on binary encoder and multi-hash table | |
| CN114896434B (en) | Hash code generation method and device based on center similarity learning | |
| CN110363001A (en) | A kind of application layer malicious requests detection method based on Transformer model | |
| CN114841151B (en) | Medical text entity relation joint extraction method based on decomposition-recombination strategy | |
| CN110196995B (en) | Complex network feature extraction method based on biased random walk | |
| CN116244484B (en) | Federal cross-modal retrieval method and system for unbalanced data | |
| EP4285281A1 (en) | Annotation-efficient image anomaly detection | |
| CN117318980A (en) | Small sample scene-oriented self-supervision learning malicious traffic detection method | |
| CN115659986B (en) | Entity relation extraction method for diabetes text | |
| CN115545005A (en) | Remote supervision relation extraction method fusing knowledge and constraint graph | |
| WO2022227957A1 (en) | Graph autoencoder-based fusion subspace clustering method and system | |
| CN109670057A (en) | A kind of gradual end-to-end depth characteristic quantization system and method | |
| CN107766928A (en) | A kind of terminal identification method based on artificial nerve network model and UA information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190611 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |