CN104601583A - Online real-time anonymization system and method for IP stream data - Google Patents
Online real-time anonymization system and method for IP stream data Download PDFInfo
- Publication number
- CN104601583A CN104601583A CN201510029241.3A CN201510029241A CN104601583A CN 104601583 A CN104601583 A CN 104601583A CN 201510029241 A CN201510029241 A CN 201510029241A CN 104601583 A CN104601583 A CN 104601583A
- Authority
- CN
- China
- Prior art keywords
- packet
- playback
- time
- anonymization
- cache blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0407—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the identity of one or more communicating identities is hidden
- H04L63/0421—Anonymous communication, i.e. the party's identifiers are hidden from the other party or parties, e.g. using an anonymizer
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides online real-time anonymization system and method for IP stream data. The system comprises an IP stream data capturing module, an IP stream data anonymization module, an IP stream data storing module and an anonymization IP stream replaying module; the IP stream data capturing module is used for extracting data packet IP address information and data packet head information from received network traffic; the IP stream anonymization module is used for processing the IP address online and on real time by anonymization through anonymization algorithm during capturing the IP stream data; the IP stream data storing module is used for storing the IP stream data subjected to anonymization to a storing device; the anonymization IP stream replaying module is used for specifying source and target MAC addresses of a data packet to be replayed and re-calculating the check bit of an IP stream data packet head during replaying so as to replay the IP stream data subjected to anonymization online and on real time.
Description
Technical field
The present invention relates to traffic capture playback technology field, be specifically related to the online real-time anonymous system and method for a kind of IP flow data.
Background technology
Network equipment detection is the key link of Logistics networks equipment dependability and stability; existing measuring technology is the test that the flow producing analog simulation by the Traffic simulation tester such as Smartbits, TestCenter carries out Network Security Device, and this measuring technology can not meet the demand of the function of the implementation safeguard protection that Network Security Device can be correct in more at a high speed and more complex environment.Real traffic back method is a kind ofly stored by live network traffic capture and carry out reverting back the network test method put.The method can reproduce actual network scenarios, reaches and investigates the function of system under test (SUT) in real network environment and the object of performance.It is a kind of important method that real traffic back method has become what carry out objective, comprehensive, system testing to disparate networks/safety means (as IDS, Firewall, IPS etc.).
Reality live network flow supports every network security and the very important resource of the every test of information security, is also the basis of live network flow playback.Because real traffic derives from existing network environment, show the behavior of user, carry a large amount of user behavior information and individual privacy information, the privacy information such as IP address, Content of Communication of the network user is had in IP flow data, just IP flow data is outwardly announced if do not add process or deal with improperly, will certainly invade the right of privacy or the business secret of the network user, this just significantly limit scope and service efficiency that real traffic allows use.How the privacy characteristic of real traffic is carried out anonymization, and the analogue simulation flow generated on this basis close to real traffic solves the effective ways that real traffic uses problem, is also the important channel promoting special equipment test environment structure.
First network traffics are mainly caught storage by existing network equipment real traffic reduction playback method of testing, then carrying out some anonymization process to network traffics, finally testing reverting back to be put in the network equipment under network traffics off-line case.The disadvantage of this method can not be carried out real-time reduction playback to network traffics and reproduce current network scene in traffic capture storing process.Be mainly manifested in for network equipment detection, first, generally the network equipment detection cycle is long, needs very large disk space to store network traffics after network traffics being caught storage; Secondly, network traffics can not use when not processing, and mainly process the IP address in flow packet, in the face of so large network traffics need the processing time grown to process very much; Finally, offline restore playback network traffics need long time equally, and reduction playback can not reproduce current network scene in real time.For network equipment detection, waste the valuable time of testing, extend test period greatly, real-time network scene can not be used to carry out security test to the network equipment.Therefore for network equipment detection technology, to catch in real time online in the urgent need to integrating network traffics, anonymization and the reduction network apparatus test system of playback and method.
Summary of the invention
The present invention is directed to the deficiencies in the prior art, provide IP flow data in real network is caught online in real time, system framework that anonymization and reduction playback are integrated.The deal with data that system uses anonymization algorithm real-time online in IP flow data acquisition procedure, processed rear in real time the network traffics after anonymization are carried out online playback.
According to an aspect of the present invention, there is provided a kind of IP flow data online real-time anonymous system, comprise IP flow data trapping module, IP flow data anonymization module, IP flow data memory module and anonymization IP and flow back to amplification module, wherein: IP flow data trapping module, for extracting packet IP address information and packet header information from the network traffics received; IP flows anonymization module, uses anonymization algorithm to carry out anonymization process to IP address in real time online in the process of catching at IP flow data; IP flow data memory module, for being stored in memory device by the IP flow data after anonymization; Anonymization IP flows back to amplification module, for the IP flow data of anonymization being carried out online real-time playback process by specifying source, the target MAC (Media Access Control) address of playback of data bag when playback and recalculating the check digit in IP flow data packet header.
According to a further aspect in the invention, provide a kind of IP flow data online real-time anonymous method, comprise the following steps:
Step 1, extracts packet IP address information and packet header information from the network traffics received;
Step 2, uses anonymization algorithm to carry out anonymization process to IP address in real time online in the process that IP flow data is caught;
Step 3, covers the source after anonymization process, object IP address the relevant position being written to source in the packet header of extraction, order IP address, is then written to by the IP packet after anonymization in the annular circular buffer applied in internal memory; Assuming that annular circular buffer has the cache blocks that N block size is M, according to the flag bit of the cache blocks P that current cache write operation is pointed to, buffer memory write operation comprises:
(1) if the cache blocks P flag bit of current sensing is empty, then directly write the data packet in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N, if (L+N) > M, it is full for then marking current cache block P, returns step 3;
(2) if the cache blocks P flag bit of current sensing is full, then next block cache block is pointed in current cache write operation, P=(P+1) %N; If the cache blocks P flag bit of current sensing is empty, then directly packet information to be written in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N; If P flag bit is full, then cache blocks is all write full, and system wait, until P flag bit is empty, returns step 3;
Step 4, same annular circular buffer is pointed in memory device write operation, and according to the flag bit of the cache blocks that current storage devices write operation is pointed to, memory device write operation comprises:
(1) if the cache blocks Q flag bit of current sensing is empty, then wait for until Q flag bit is full, return step 4;
(2) if the cache blocks Q flag bit of current sensing is full, then the content in this cache blocks is all written in memory device, has write rearmounted this cache blocks Q flag bit for empty, memory device write operation has been pointed to next block cache block, Q=(Q+1) %N, returns step 4;
Step 5, use binary chop algorithm by checking the timestamp information of the packet middle wrapping head stored according to the initial time that playback is specified, first timestamp is found to be more than or equal to the packet of specifying playback initial time, playback operation points to this packet, and arranging next second playback of data bag time adds 1 second for this data packet head time, and arranging next second system time is that the system time got adds 1 second;
Step 6, obtains present system time, reads the packet header time that current playback operation is pointed to, if the packet that current playback operation is pointed to is empty, turns to step 10; If the packet time that current playback operation is pointed to is more than or equal to the playback end time of setting, turn to step 11; Judge that playback operation comprises according to next second playback of data bag time, current data packet playback duration, next second system time and present system time:
(1) if present system time is less than next second system time and current playback packet time is less than next second playback of data bag time, step 7 is turned to;
(2) if present system time is more than or equal to next second system time and current playback packet time is more than or equal to next second playback of data bag time, step 8 is turned to;
(3) if present system time is more than or equal to next second system time and current playback packet time is less than or equal to next second playback of data bag time, step 8 is turned to;
(4) if present system time is less than next second system time and current playback packet time is more than or equal to next second playback of data bag time, step 9 is turned to;
Step 7, obtain the packet header information that playback operation points to, add source data packet, target MAC (Media Access Control) address, the load of packet is added at random according to the data packet length information in packet header, corresponding checking algorithm is used to recalculate check digit according to the protocol information of packet, usage data bag playback interface function is by packet real-time playback in real network environment, and playback operation points to next packet, and playback completes rear steering step 6;
Step 8, next second playback of data bag time added 1 second, and within next second, system time adds 1 second, turns to step 6;
Step 9, playback operation hangs up the time of setting, turns to step 6;
Step 10, playback operation hangs up the time of setting, turns to step 6;
Step 11, playback operation normally completes.
Technical scheme of the present invention has following advantage and effect:
(1) compared with prior art, being caught in real time online by IP stream of the invention, anonymization, storage and reduction playback have been combined into integrally.Compensate for long defect and deficiency with current network scene can not be reproduced in real time of existing reduction playback method of testing test period.
(2) the present invention is storing the partial information only storing 56 fixing byte data handbag heads in data procedures, at utmost remains the primitive character information of IP flow data, remains the use value of data, substantially increase the utilance of memory device;
(3) the present invention is when carrying out playback according to appointment playback duration, and the complexity of locator data bag algorithm is log
2n (), location efficiency is more efficient.
(4) the present invention stabs information according to the packet time stored in replayed section, service time comparison algorithm, packet playback rate has been accurate to level second compared to packet capture speed, at utmost reproduced current network scenarios, the present invention also can reproduce the network scenarios of arbitrary period in real time according to the playback duration of specifying in addition.
Accompanying drawing explanation
Fig. 1 is the block diagram according to the online real-time anonymous system of the IP flow data of the embodiment of the present invention;
Fig. 2 schematically shows the result of according to embodiment of the present invention use Crypto-PAn anonymization algorithm, IP address being carried out to anonymization;
Fig. 3 schematically shows the operation of flowing memory module according to the IP of the embodiment of the present invention, wherein Fig. 3 (a) is the state of current annular circular buffer, and Fig. 3 (b) is buffer memory write operation and memory device write operation writes the state of annular circular buffer after two packets;
Fig. 4 is the playback flow chart of the anonymization IP flow module according to the embodiment of the present invention;
Fig. 5 schematically shows the playback operation of the anonymization IP flow module according to the embodiment of the present invention.
Embodiment
Elaborate below in conjunction with accompanying drawing 1 ~ 5 pair of embodiments of the invention.Although should be appreciated that and set forth below is detailed execution mode and concrete operating process, protection scope of the present invention is not limited to following embodiment.
Seek to catch online the network traffics in network according to embodiments of the invention, use Crypto-PAn algorithm to carry out anonymization process to the IP address in IP stream packets simultaneously, and the network traffics after processing is complete after online real-time playback anonymization, thus reproduce current network scenarios.
Fig. 1 is the block diagram according to the online real-time anonymous system of the IP flow data of the embodiment of the present invention.This system comprises 4 modules: IP flow data trapping module, IP flow anonymization module, IP flow data memory module and anonymization IP and flow back to amplification module.
IP flow data trapping module is used for extracting packet IP address information and packet header information from the network traffics that (such as passing through network interface card) receives.In one example, the extraction of packet IP address information and packet header information can come by usage data Packet capturing interface function.
IP flows anonymization module and is used for using anonymization algorithm to carry out anonymization process to IP address in real time online in the process of catching at IP flow data.In one example, carrying out in anonymization process to IP address, when ensureing anonymization safety, using IP address prefix reservation anonymization algorithm Crypto-PAn at utmost to remain the primary characteristic of IP traffic.
IP flow data memory module is used for the IP flow data after anonymization to be stored in memory device.Particularly, the IP flow data after anonymization is written in annular circular buffer by IP flow data memory module, then the data in annular circular buffer is written in memory device (such as, high performance disk).
Anonymization IP flows back to amplification module for the IP flow data of anonymization being carried out online real-time playback process by specifying source, the target MAC (Media Access Control) address of playback of data bag when playback and recalculating the check digit in IP flow data packet header.At utmost ensure playback rate in replayed section and to catch speed identical, thus IP stream packets is played back in the network environment of reality.In one example, this playback process can have been come by usage data bag playback interface function.
Fig. 2 schematically shows the result of according to embodiment of the present invention use Crypto-PAn anonymization algorithm, IP address being carried out to anonymization.In the present embodiment, the Crypto-PAn algorithm that IP stream anonymization module uses IP address prefix to retain carries out anonymization process to the partial data bag IP address in the packet extracted by IP flow data trapping module.Packet 1 tuple Pktn (packet IP address) represents, wherein n is the precedence that in network reception parts (such as network interface card), packet arrives, and source, object IP address only represent with an IP address.As can be seen from the figure, if IP address prefix is identical, after so using Crypto-PAn anonymization, still there is identical prefix.
Fig. 3 schematically shows the operation of flowing memory module according to the IP of the embodiment of the present invention.In the present embodiment, IP flow data memory module is arranged to and performs following operation:
Step a1, covers the source after anonymization process, object IP address the relevant position being written to source in the packet header of extraction, order IP address, is then written to by the IP packet after anonymization in the annular circular buffer applied in internal memory.Assuming that annular circular buffer has the cache blocks that N block size is M, according to the flag bit of the cache blocks P that current cache write operation is pointed to, buffer memory write operation comprises:
(1) if the cache blocks P flag bit of current sensing is empty, then directly write the data packet in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N, if (L+N) > M, it is full for then marking current cache block P, returns step a1;
(2) if the cache blocks P flag bit of current sensing is full, then next block cache block is pointed in current cache write operation, P=(P+1) %N; If the cache blocks P flag bit of current sensing is empty, then directly packet information to be written in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N; If P flag bit is full, then cache blocks is all write full, and system wait, until P flag bit is empty, returns step a1.
In one example, in this step, fixing write data packet length is 56 bytes, comprise: (1) packet pcap header (16 byte), include packet time stamp (8 byte), the length (4 byte) of packet current group and the length (4 byte) of packet; (2) packet header information, includes packet networks layer IP header (20 byte), data packet transmission layer information (fixing acquisition 20 byte).
Step a2, same annular circular buffer is pointed in memory device write operation, and according to the flag bit of the cache blocks Q that current storage devices write operation is pointed to, memory device write operation comprises:
(1) if the cache blocks Q flag bit of current sensing is empty, then wait for until Q flag bit is full, return step a2;
(2) if the cache blocks Q flag bit of current sensing is full, then the content in this cache blocks is all written in memory device, write rearmounted this cache blocks Q flag bit for empty, memory device write operation is pointed to next block cache block, Q=(Q+1) %N, returns step a2.
In one example, step a1 and step a2 belongs to that two threads are asynchronous to carry out, two threads share an annular circular buffer, the thread that step a1 uses writes data in annular circular buffer, and the thread data read in annular circular buffer that step a2 uses are written in memory device (such as high-performance disk).
Fig. 3 (a) is the state of current annular circular buffer.Suppose that 8 cache blocks that annular circular buffer is 112 bytes by size form, each cache blocks can only store 2 packets, according to above-mentioned steps, for buffer memory write operation thread, when packet Pkt6 arrives, Pkt6 writes in cache blocks 2 by write operation, and it is full for putting cache blocks 2; When packet Pkt7 arrives, cache blocks 3 is pointed in write operation, is written in cache blocks 3 by packet Pkt7.For memory device write operation thread, Pkt1, Pkt2 are written in memory device by write operation successively, and put cache blocks 0 for empty, cache blocks 1 is pointed in write storage device operation, continues write storage device operation.Buffer memory write operation and memory device write operation write the state of annular circular buffer after two packets as shown in Fig. 3 (b).
Fig. 4 is the playback flow chart of the anonymization IP flow module according to the embodiment of the present invention.In the present embodiment, anonymization IP flows back to amplification module and is arranged to the following operation of execution:
Step b1, use binary chop algorithm by checking the timestamp information of the packet middle wrapping head stored according to the playback initial time of specifying, find first timestamp to be more than or equal to the packet of specifying playback initial time, playback operation points to this packet; Arrange next second playback of data bag time TS and add 1 second for this data packet head time, arranging next second system time T is that the system time got adds 1 second.
Step b2, obtains present system time T
new, read the packet header time TS that current playback operation is pointed to
newif the packet that current playback operation is pointed to is empty, turns to step b6, if the packet time that current playback operation is pointed to is more than or equal to the playback end time of setting, turn to step b7.According to T, T
new, TS and TS
newjudge that playback operation comprises:
(1) if T
new< T and TS
new< TS, turns to step b3;
(2) if T
new>=T and TS
new>=TS, turns to step b4;
(3) if T
new>=T and TS
new<=TS, turns to step b4;
(4) if T
new< T and TS
new>=TS, turns to step b5;
Step b3, IP flow data playback module obtains the packet header information that playback operation points to, add source data packet, target MAC (Media Access Control) address, the load of packet is added at random according to the data packet length information in packet header, corresponding checking algorithm is used to recalculate check digit according to the protocol information of packet, usage data bag playback interface function is by packet real-time playback in real network environment, and playback operation points to next packet, and playback completes rear steering step b2;
Step b4, within next second, playback of data bag time TS adds 1 second, TS=TS+1, and within next second, system time T adds 1 second, and T=T+1, turns to step b2;
Step b5, playback operation hangs up the time (such as, setting 100 milliseconds) of setting, turns to step b2;
Step b6, playback operation hangs up time second (such as, setting 1 second) of setting, turns to step b2;
Step b7, playback operation normally completes.
Fig. 5 schematically shows the playback operation of the anonymization IP flow module according to the embodiment of the present invention.For convenience of setting forth, packet 1 tuple Pktn (packet time stamp) represents, wherein n is the precedence that packet stores in memory device.Flow back to the concrete implementation step of putting according to above-mentioned IP, first system uses binary chop algorithm to navigate to initial data bag for Pkt1 according to the playback initial time of specifying, and playback operation points to Pkt1; Its subsystem acquisition number is according to the timestamp of bag, and arranging TS is that packet time stamp adds 1 second, TS=1.6+1=2.6, and obtain system time, setup times T is that the system time got adds 1 second, supposes T=10+1=11; Again obtain present system time T
new, T
new=10, read data packet packet header time stamp T S
new, TS
new=10, then according to T, T
new, TS and TS
newpkt1 is judged to be played back to network interface card; Source in last playback module interpolation packet, target MAC (Media Access Control) address information, stochastic generation load information, after recalculating check digit, usage data bag playback interface function playback of data bag is in real network, and playback operation is pointed to next packet.Playback operation flows back to the concrete implementation step playback of putting until playback terminates according to above-mentioned anonymization IP.
As mentioned above, the invention provides the system framework that IP stream is caught online in real time, anonymization and reduction playback are combined as a whole.Especially, adapt with this system framework, the processing method corresponding to the operation of modules can also be provided.In one example, IP flows back to amplification module independent of IP flow data trapping module, IP flow data anonymization module and IP flow data memory module.Correspondingly, IP flows back to operation that amplification module performs and the step that performs of all the other modules and belongs to that two processes are asynchronous carries out.IP flows back to amplification module can reproduce random time section in real time network scenarios according to the playback duration of setting, substantially reduces the cycle of existing reduction playback method of testing thus.
Be to be understood that; above content is in conjunction with concrete preferred implementation further description made for the present invention; can not assert that the specific embodiment of the present invention is only limitted to this; for general technical staff of the technical field of the invention; without departing from the inventive concept of the premise; some simple deduction or replace can also be made, all should be considered as belonging to the present invention by submitted to claims determination scope of patent protection.
Claims (9)
1. the online real-time anonymous system of IP flow data, comprises IP flow data trapping module, IP flow data anonymization module, IP flow data memory module and anonymization IP and flows back to amplification module, wherein:
IP flow data trapping module, for extracting packet IP address information and packet header information from the network traffics received;
IP flows anonymization module, uses anonymization algorithm to carry out anonymization process to IP address in real time online in the process of catching at IP flow data;
IP flow data memory module, for being stored in memory device by the IP flow data after anonymization;
Anonymization IP flows back to amplification module, for the IP flow data of anonymization being carried out online real-time playback process by specifying source, the target MAC (Media Access Control) address of playback of data bag when playback and recalculating the check digit in IP flow data packet header.
2. system according to claim 1, preferably, wherein said anonymization process uses IP address prefix to retain algorithm Crypto-Pan.
3. system according to claim 1, wherein said IP flow data memory module is arranged to and performs following operation:
Step a1, covers the source after anonymization process, object IP address the relevant position being written to source in the packet header of extraction, order IP address, is then written to by the IP packet after anonymization in the annular circular buffer applied in internal memory; Assuming that annular circular buffer has the cache blocks that N block size is M, according to the flag bit of the cache blocks P that current cache write operation is pointed to, buffer memory write operation comprises:
(1) if the cache blocks P flag bit of current sensing is empty, then directly write the data packet in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N, if (L+N) > M, it is full for then marking current cache block P, returns step a1;
(2) if the cache blocks P flag bit of current sensing is full, then next block cache block is pointed in current cache write operation, P=(P+1) %N; If the cache blocks P flag bit of current sensing is empty, then directly packet information to be written in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N; If P flag bit is full, then cache blocks is all write full, and system wait, until P flag bit is empty, returns step a1;
Step a2, same annular circular buffer is pointed in memory device write operation, and according to the flag bit of the cache blocks Q that current storage devices write operation is pointed to, memory device write operation comprises:
(1) if the cache blocks Q flag bit of current sensing is empty, then wait for until Q flag bit is full, return step a2;
(2) if the cache blocks Q flag bit of current sensing is full, then the content in this cache blocks is all written in memory device, write rearmounted this cache blocks Q flag bit for empty, memory device write operation is pointed to next block cache block, Q=(Q+1) %N, returns step a2.
4. system according to claim 1, wherein anonymization IP flows back to amplification module and is arranged to the following operation of execution:
Step b1, use binary chop algorithm by checking the timestamp information of the packet middle wrapping head stored according to the playback initial time of specifying, find first timestamp to be more than or equal to the packet of specifying playback initial time, playback operation points to this packet; Arrange next second playback of data bag time TS and add 1 second for this data packet head time, arranging next second system time T is that the system time got adds 1 second;
Step b2, obtains present system time T
new, read the packet header time TS that current playback operation is pointed to
newif the packet that current playback operation is pointed to is empty, turns to step b6, if the packet time that current playback operation is pointed to is more than or equal to the playback end time of setting, turn to step b7; According to T, T
new, TS and TS
newjudge that playback operation comprises:
(1) if T
new< T and TS
new< TS, turns to step b3;
(2) if T
ncw>=T and TS
ncw>=TS, turns to step b4;
(3) if T
new>=T and TS
new<=TS, turns to step b4;
(4) if T
new< T and TS
new>=TS, turns to step b5;
Step b3, obtain the packet header information that playback operation points to, add source data packet, target MAC (Media Access Control) address, the load of packet is added at random according to the data packet length information in packet header, corresponding checking algorithm is used to recalculate check digit according to the protocol information of packet, usage data bag playback interface function is by packet real-time playback in real network environment, and playback operation points to next packet, and playback completes rear steering step b2;
Step b4, within next second, playback of data bag time TS adds 1 second, TS=TS+1, and within next second, system time T adds 1 second, and T=T+1, turns to step b2;
Step b5, playback operation hangs up the time of setting, turns to step b2;
Step b6, playback operation hangs up time second (such as, setting 1 second) of setting, turns to step b2;
Step b7, playback operation normally completes.
5. the online real-time anonymous method of IP flow data, comprises the following steps:
Step 1, extracts packet IP address information and packet header information from the network traffics received;
Step 2, uses anonymization algorithm to carry out anonymization process to IP address in real time online in the process that IP flow data is caught;
Step 3, covers the source after anonymization process, object IP address the relevant position being written to source in the packet header of extraction, order IP address, is then written to by the IP packet after anonymization in the annular circular buffer applied in internal memory; Assuming that annular circular buffer has the cache blocks that N block size is M, according to the flag bit of the cache blocks P that current cache write operation is pointed to, buffer memory write operation comprises:
(1) if the cache blocks P flag bit of current sensing is empty, then directly write the data packet in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N, if (L+N) > M, it is full for then marking current cache block P, returns step 3;
(2) if the cache blocks P flag bit of current sensing is full, then next block cache block is pointed in current cache write operation, P=(P+1) %N; If the cache blocks P flag bit of current sensing is empty, then directly packet information to be written in this cache blocks, cache blocks write length L is fixed write data packet length N, L=L+N; If P flag bit is full, then cache blocks is all write full, and system wait, until P flag bit is empty, returns step 3;
Step 4, same annular circular buffer is pointed in memory device write operation, and according to the flag bit of the cache blocks that current storage devices write operation is pointed to, memory device write operation comprises:
(1) if the cache blocks Q flag bit of current sensing is empty, then wait for until Q flag bit is full, return step 4;
(2) if the cache blocks Q flag bit of current sensing is full, then the content in this cache blocks is all written in memory device, has write rearmounted this cache blocks Q flag bit for empty, memory device write operation has been pointed to next block cache block, Q=(Q+1) %N, returns step 4;
Step 5, use binary chop algorithm by checking the timestamp information of the packet middle wrapping head stored according to the initial time that playback is specified, first timestamp is found to be more than or equal to the packet of specifying playback initial time, playback operation points to this packet, and arranging next second playback of data bag time adds 1 second for this data packet head time, and arranging next second system time is that the system time got adds 1 second;
Step 6, obtains present system time, reads the packet header time that current playback operation is pointed to, if the packet that current playback operation is pointed to is empty, turns to step 10; If the packet time that current playback operation is pointed to is more than or equal to the playback end time of setting, turn to step 11; Judge that playback operation comprises according to next second playback of data bag time, current data packet playback duration, next second system time and present system time:
(1) if present system time is less than next second system time and current playback packet time is less than next second playback of data bag time, step 7 is turned to;
(2) if present system time is more than or equal to next second system time and current playback packet time is more than or equal to next second playback of data bag time, step 8 is turned to;
(3) if present system time is more than or equal to next second system time and current playback packet time is less than or equal to next second playback of data bag time, step 8 is turned to;
(4) if present system time is less than next second system time and current playback packet time is more than or equal to next second playback of data bag time, step 9 is turned to;
Step 7, obtain the packet header information that playback operation points to, add source data packet, target MAC (Media Access Control) address, the load of packet is added at random according to the data packet length information in packet header, corresponding checking algorithm is used to recalculate check digit according to the protocol information of packet, usage data bag playback interface function is by packet real-time playback in real network environment, and playback operation points to next packet, and playback completes rear steering step 6;
Step 8, next second playback of data bag time added 1 second, and within next second, system time adds 1 second, turns to step 6;
Step 9, playback operation hangs up the time of setting, turns to step 6;
Step 10, playback operation hangs up the time of setting, turns to step 6;
Step 11, playback operation normally completes.
6. method according to claim 5, wherein, in step 3, fixing write data packet length is 56 bytes, comprise: the packet pcap header of (1) 16 byte, include the packet time stamp of 8 bytes, the length of packet current group of 4 bytes and the length of the packet of 4 bytes; (2) packet header information, includes the packet networks layer IP header of 20 bytes and the data packet transmission layer information of 20 bytes.
7. method according to claim 5, wherein, step 3 and step 4 belong to that two processes are asynchronous carries out, two processes share an annular circular buffer, the process of step 3 writes data in annular circular buffer, and the data that the process of step 4 reads in annular circular buffer are written in memory device.
8. method according to claim 5, wherein, step 1-4, step 5-11 belong to that two processes are asynchronous carries out.
9. method according to claim 5, wherein, described anonymization process uses IP address prefix to retain algorithm Crypto-Pan.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510029241.3A CN104601583B (en) | 2015-01-21 | 2015-01-21 | A kind of online real-time anonymous system and method for IP flow datas |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510029241.3A CN104601583B (en) | 2015-01-21 | 2015-01-21 | A kind of online real-time anonymous system and method for IP flow datas |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104601583A true CN104601583A (en) | 2015-05-06 |
| CN104601583B CN104601583B (en) | 2017-11-10 |
Family
ID=53127089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510029241.3A Expired - Fee Related CN104601583B (en) | 2015-01-21 | 2015-01-21 | A kind of online real-time anonymous system and method for IP flow datas |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104601583B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049277A (en) * | 2015-06-08 | 2015-11-11 | 国家计算机网络与信息安全管理中心 | Network flow generation method based on data flow features |
| CN105721627A (en) * | 2016-02-25 | 2016-06-29 | 中国科学院信息工程研究所 | Method for online anonymization of IP network streaming data |
| CN108989142A (en) * | 2018-05-25 | 2018-12-11 | 中国科学院计算机网络信息中心 | Network test method, device and storage medium |
| CN109104426A (en) * | 2018-08-21 | 2018-12-28 | 西安交通大学 | A kind of encryption flow analysis defence method based on packet sending speed |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101795230B (en) * | 2010-02-23 | 2012-05-23 | 西安交通大学 | Network flow recovery method |
| CN102761517A (en) * | 2011-04-25 | 2012-10-31 | 工业和信息化部电信传输研究所 | Content reduction method for high-speed network |
| CN102185705B (en) * | 2011-01-24 | 2014-04-30 | 中国人民解放军国防科学技术大学 | Intranet video file monitoring method based on information reduction |
-
2015
- 2015-01-21 CN CN201510029241.3A patent/CN104601583B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101795230B (en) * | 2010-02-23 | 2012-05-23 | 西安交通大学 | Network flow recovery method |
| CN102185705B (en) * | 2011-01-24 | 2014-04-30 | 中国人民解放军国防科学技术大学 | Intranet video file monitoring method based on information reduction |
| CN102761517A (en) * | 2011-04-25 | 2012-10-31 | 工业和信息化部电信传输研究所 | Content reduction method for high-speed network |
Non-Patent Citations (3)
| Title |
|---|
| 史冰等: ""IP地址前缀保留匿名化算法的改进"", 《微电子学与计算机》 * |
| 张敏: "基于应用层和传输层的网络测量分析研究", 《中国 知网》 * |
| 杨明等: ""基于用户可用带宽测算的匿名路由算法性能评价"", 《南京航空航天大学学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049277A (en) * | 2015-06-08 | 2015-11-11 | 国家计算机网络与信息安全管理中心 | Network flow generation method based on data flow features |
| CN105049277B (en) * | 2015-06-08 | 2018-11-13 | 国家计算机网络与信息安全管理中心 | A kind of network flow generation method based on data flow characteristics |
| CN105721627A (en) * | 2016-02-25 | 2016-06-29 | 中国科学院信息工程研究所 | Method for online anonymization of IP network streaming data |
| CN105721627B (en) * | 2016-02-25 | 2018-12-11 | 中国科学院信息工程研究所 | A kind of online de-identification method of IP network flow data |
| CN108989142A (en) * | 2018-05-25 | 2018-12-11 | 中国科学院计算机网络信息中心 | Network test method, device and storage medium |
| CN109104426A (en) * | 2018-08-21 | 2018-12-28 | 西安交通大学 | A kind of encryption flow analysis defence method based on packet sending speed |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104601583B (en) | 2017-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8457128B2 (en) | Capturing packets with parallel capture engines | |
| US8310942B2 (en) | Flow statistics aggregation | |
| US7953092B2 (en) | Traffic receiver using parallel capture engines | |
| CN101924757B (en) | Method and system for reviewing Botnet | |
| JP4995310B2 (en) | Destination packet detection apparatus and detection method | |
| CN102420701B (en) | Method for extracting internet service flow characteristics | |
| CN110401624A (en) | The detection method and system of source net G system mutual message exception | |
| CN104601583A (en) | Online real-time anonymization system and method for IP stream data | |
| RU2608874C2 (en) | Method and device for modifying and forwarding messages in data network | |
| US20120197847A1 (en) | Method and System for Monitoring and Tracing Multimedia Resource Transmission | |
| CN103259737B (en) | A kind of method for rapidly positioning of flow of parallel storage high-speed network | |
| US20130329572A1 (en) | Misdirected packet statistics collection and analysis | |
| CN108287905A (en) | A kind of extraction of network flow feature and storage method | |
| CN109451486A (en) | WiFi acquisition system and WiFi terminal detection method based on probe request | |
| US8654643B2 (en) | Wide field indexing for packet tracking | |
| CN105847147A (en) | Data transmission method and device | |
| CN109802992B (en) | Method and device for reading and writing resource change log | |
| CN115174676A (en) | Convergence and shunt method and related equipment thereof | |
| CN101986611A (en) | Quick flow grouping method based on two-level cache | |
| CN103067690A (en) | Method and device of equalized audio-video data transmission based on intelligent monitoring platform | |
| CN109802990B (en) | Resource log reading and writing method and device for reducing data redundancy | |
| CN109462283A (en) | Intelligent substation individual equipment flux monitoring method and system | |
| CN108965093A (en) | A kind of VLAN allocation method and device | |
| CN102821047A (en) | Method, network apparatus and system for establishing corresponding relation in networks | |
| CN110768854B (en) | Data statistics method and device based on video network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20171110 Termination date: 20210121 |