CN107995061A - Collection and playback system and method when more specification 10Gbps network signals are long - Google Patents
Collection and playback system and method when more specification 10Gbps network signals are long Download PDFInfo
- Publication number
- CN107995061A CN107995061A CN201711242627.8A CN201711242627A CN107995061A CN 107995061 A CN107995061 A CN 107995061A CN 201711242627 A CN201711242627 A CN 201711242627A CN 107995061 A CN107995061 A CN 107995061A
- Authority
- CN
- China
- Prior art keywords
- data
- collection
- thread
- cache blocks
- playback
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000004458 analytical method Methods 0.000 claims abstract description 8
- 238000001152 differential interference contrast microscopy Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 15
- 230000015654 memory Effects 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 6
- 239000013307 optical fiber Substances 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000007405 data analysis Methods 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 4
- 238000011084 recovery Methods 0.000 claims description 3
- 230000011664 signaling Effects 0.000 claims description 3
- 229910017435 S2 In Inorganic materials 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 7
- 125000004122 cyclic group Chemical group 0.000 abstract description 3
- 230000007246 mechanism Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000003111 delayed effect Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- RGNPBRKPHBKNKX-UHFFFAOYSA-N hexaflumuron Chemical compound C1=C(Cl)C(OC(F)(F)C(F)F)=C(Cl)C=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F RGNPBRKPHBKNKX-UHFFFAOYSA-N 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of more specification 10Gbps network signals it is long when collection and playback system and method, be related to express network signal acquisition and playback technology field.The system and method can realize the collection and playback of the express network signal to the plurality of specifications such as 10Gbps Ethernets and 10Gbps SDH at the same time, solve the signal acquisition and playback mode of conventional single specification.Meanwhile with high-speed high capacity high-performance Sasobit and SBR D storage dishes and multithreading so that the system can reach the collection of long-time linear speed and playback of 10Gbps network signals, and can carry out real-time continuous storage to the signal of collection.In addition, the system is provided presses linear speed, timestamp or constant speed playback function to signal, while data readback number can be specified, Infinite Cyclic playback mode is defaulted as, so as to provide multiple means for data signal analysis.
Description
Technical field
The present invention relates to express network signal acquisition and playback technology field, more particularly to a kind of more specification 10Gbps networks
Collection and playback system and method during Chief Signal Boatswain.
Background technology
Possess the express network signal of more specifications it is long when collection and playback system, analysis to network signal, screening and
Extraction is vital.
At present, existing express network signal acquisition and playback system, on the one hand, its adaptable signal specification of institute is opposite
It is relatively simple, a certain network signal of playback can only be gathered；On the other hand, for express network signal, first signal can only be adopted
Collect in memory, then slowly deposited, and the tester memory of standard is small, it is impossible to prolonged continuous acquisition, storage
With the ability of playback, the message of abundant in content super large file can not be sent.The collection to signal will certainly be thus caused to be
It is interrupted, is incomplete, the analysis meeting to follow-up signal produces certain influence.
The content of the invention
It is an object of the invention to provide a kind of more specification 10Gbps network signals it is long when collection and playback system and method,
So as to solve foregoing problems existing in the prior art.
To achieve these goals, the technical solution adopted by the present invention is as follows：
Collection and playback system when a kind of more specification 10Gbps network signals are long, including：Industrial personal computer, 10GE/10G SDH high
Fast network interface card and SSD high-performance storage card, the industrial personal computer pass through PCI-E interface and the 10GE/10G SDH High_speed NICs respectively
Connected with the SSD high-performance storage card；The 10GE/10G SDH High_speed NICs are connected by optical fiber with data source；
The SSD high-performance storage card includes multiple FPGA, each controller respectively with multiple NAND FLASH cores
Piece is connected with DDR.
Collection and back method, utilize above-mentioned more specification 10Gbps networks when a kind of more specification 10Gbps network signals are long
Collection and playback system, include the following steps during Chief Signal Boatswain：
S1,10G Ethernet or 10G SDH signals are by the way that in intelligent acess High_speed NIC, High_speed NIC is to different size
High speed signal be acquired, converge and distribute according to corresponding strategy；
S2, after the high speed signal of distribution reaches the memory of industrial personal computer by DMA, operating system is delayed using multithreading and annular
Area is rushed, high speed signal is received respectively, is handled, and is stored in SSD high-performance storage cards；
S3, by the way of multithreading reads file, and in the deblocking caching industrial personal computer memory of reading；
S4, reading cache data, High_speed NIC is transmitted to by DMA；
S5, High_speed NIC transfer out the data convert of reading by optical fiber.
Preferably, in S1, the High_speed NIC is acquired the high speed signal of different size according to corresponding strategy, converges
Poly- and distribution, specifically includes to the collection of STM64 frames and the extraction of IP bags, wherein, it is described to be specially to the collection of STM64 frames：To adopting
After the high speed signal collected carries out clock recovery, serioparallel exchange, frame synchronization and descrambling, original frame data is obtained, and this is original
Frame data are transmitted in the queue of host computer side；The extraction of the IP bags is specially：To the SDH protocol analysis of IP signaling bearers also
Original, and load balancing is into multiple receiving queues of host computer side.
Preferably, the SDH agreements include PoS frames, gfp frame and the WAN frames of carrying.
Preferably, in S2, the multithreading includes：Data receiver thread, data processing threads and data storage thread, institute
State data receiver thread to be used to receive data from High_speed NIC, the data processing threads are used for data analysis, and the data are deposited
Disk is used for data and writes the SSD high-performance storage card, and the buffer circle is used for the work for coordinating multithreading.
Preferably, in S3, the multithreading includes reading thread and playback thread, and the reading thread, which is used to read, to be stored
Data simultaneously do the processing of some bags, and playback thread is used to send data packet；The reading thread is arranged to multiple, each reading
Thread corresponds to one piece of cache blocks of fixed size, and every piece of cache blocks are both provided with read-write mark, all cache blocks composition
Buffer circle.
Preferably, S3 is specially：When the reading thread judges that its corresponding cache blocks is writeable, will be read from hard disk
File be put into cache blocks, and by the traffic sign placement of cache blocks to be readable；When playback thread travels through the annular in sequence
During buffering area, whether the mark for first determining whether cache blocks is readable, if it is, the data of cache blocks are transmitted, then
It is writeable by the traffic sign placement of cache blocks, repeats the above process, untill file reads and is sent completely.
Preferably, a certain size space is reserved in the front end that each reading thread corresponds to cache blocks, when each thread handle
After corresponding data write cache blocks, according to the long instruction of each bag, whether detection caching data packet in the block is complete, if detection
Data packet to the end is imperfect, then moves the incomplete content to follow-up adjacent read line from current cache blocks
In the front end headspace of the corresponding cache blocks of journey, spelled with the follow-up adjacent another part content for reading the data packet in thread
It is connected together, forms a complete data packet.
The beneficial effects of the invention are as follows：Collection and returned when more specification 10Gbps network signals provided in an embodiment of the present invention are long
Place system and method, can realize adopting to the express network signal of the plurality of specifications such as 10Gbps Ethernets and 10Gbps SDH at the same time
Collection and playback, solve the signal acquisition and playback mode of conventional single specification.Meanwhile with high speed NAND FLASH storage dishes
And multithreading so that the system can reach the collection of long-time linear speed and playback of 10Gbps network signals, and to collection
Signal can carry out real-time continuous storage.In addition, the system is provided plays back work(to signal by linear speed, timestamp or constant speed
Can, while data readback number can be specified, Infinite Cyclic playback mode is defaulted as, so as to be provided for data signal analysis a variety of
Means.
Brief description of the drawings
Fig. 1 is collection and the logic knot of playback system when more specification 10Gbps network signals provided in an embodiment of the present invention are long
Structure schematic diagram；
Fig. 2 is the structure diagram of SSD high-performance storage cards；
Fig. 3 is that the built-in function of High_speed NIC realizes flow diagram；
Fig. 4 is that multithreading loop buffer mechanism realizes process schematic.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, below in conjunction with attached drawing, to the present invention into
Row is further described.It should be appreciated that the specific embodiments described herein are not used to only to explain the present invention
Limit the present invention.
Embodiment one
As illustrated in fig. 1 and 2, an embodiment of the present invention provides a kind of more specification 10Gbps network signals it is long when collection and playback
System, including：Industrial personal computer, 10GE/10G SDH High_speed NICs and SSD high-performance storage cards, the industrial personal computer pass through PCI- respectively
E interface is connected with the 10GE/10G SDH High_speed NICs and the SSD high-performance storage card；The 10GE/10G SDH are at a high speed
Network interface card is connected by optical fiber with data source；
The SSD high-performance storage card includes multiple FPGA, each controller respectively with multiple NAND FLASH cores
Piece is connected with DDR.
In the present invention, to realize system integraty, hardware configuration takes the additional function expansion card of integrated portable industrial personal computer
Form.Main platform is a industrial personal computer for carrying display and keyboard, additionally by PCI-E interface extension 10GE/10G SDH
High_speed NIC and SSD high-performance storage cards.Wherein, High_speed NIC can adopt 10GE and 10G SDH express network signals
Collect, and the real-time storage of data is acquired with SSD high-performance storage cards.Meanwhile, it is capable to the data of collection or user are given birth to
Into data pass through network interface card carry out linear speed playback.Its logical construction is as shown in Figure 1.
So the system possesses the express network signal acquisition and playback capability of more specifications, more specifications are mainly reflected in energy
Support the signals such as 10G Ethernets and the 10G SDH of current mainstream.
In addition, in the embodiment of the present invention, storage employs high performance SSD storage cards.This is one and is based on NAND
The high-performance solid-state memory card of FLASH chip, has the characteristics that high integration, high-performance and low delay.Controller and bridge logic
It is combined into one, improves integrated level.FLASH can support ONFI2.0 interfaces, and FLASH particle velocities are up to 200M/T.Every piece of board
3 controllers are could support up, each controller can support 16+1 passages FLASH.Meanwhile support power down protection, host is abnormal
Data are not lost in the case of lower electricity, the security of data are effectively ensured, its hardware structure diagram is as shown in Figure 2.
Above-mentioned high performance SSD storage cards have following technical characterstic：
1) RAID-0 technologies are used, reach high bandwidth by the multiple FLASH particles of concurrent reading and writing.
2) asynchronous command queue technology is used, command queue's depth can handle 256 read write commands at the same time, carry up to 256
The concurrency of high FLASH operations, there is provided high IOPS.
3) small data block random write performance is improved by increasing redundancy.The more big visible capacity of redundancy is smaller, user
Visible capacity can be set to meet different application scenarios.
4) realize that DMA address dissipates poly- technology, the data volume of single read-write requests is improved, so as to obtain high I/O bandwidth.
5) dynamic dispatching algorithm, parallel processing garbage reclamation and the write-in work of new block are used, while ensureing to write bandwidth,
Reduce performance inconsistency.
6) without mechanical parts such as magnetic heads, when read-write operation positions without magnetic head to be delayed.
7) Native PCIE interfaces are used, there is provided the bandwidth of bigger, reads to be not required to, by caching, be effectively reduced during data
System delay.
8) quick start is supported.The list item in NAND FLASH is directly read in initialization procedure, is substantially reduced initial
Change the time.
Embodiment two
An embodiment of the present invention provides a kind of more specification 10Gbps network signals it is long when collection and back method, utilize implementation
Collection and playback system, include the following steps when more specification 10Gbps network signals described in example one are long：
S1,10G Ethernet or 10G SDH signals are by the way that in intelligent acess High_speed NIC, High_speed NIC is to different size
High speed signal be acquired, converge and distribute according to corresponding strategy；
S2, after the high speed signal of distribution reaches the memory of industrial personal computer by DMA, operating system is delayed using multithreading and annular
Area is rushed, high speed signal is received respectively, is handled, and is stored in SSD high-performance storage cards；
S3, by the way of multithreading reads file, and in the deblocking caching industrial personal computer memory of reading；
S4, reading cache data, High_speed NIC is transmitted to by DMA；
S5, High_speed NIC transfer out the data convert of reading by optical fiber.
The above method, in gatherer process, 10G Ethernets or 10G SDH signals first passes through intelligent acess high speed network
In card, network interface card is acquired, converges and distributes according to different strategies to the high speed signal of different size.When the data of distribution are led to
After crossing DMA arrival calculator memories, operating system utilizes multi-thread mechanism, and gathered data is respectively processed and is deposited.Together
Sample, when being played back to data, since the speed of disk-read is relatively slow, in order to reach linear speed playback, it is also necessary to using multithreading
Read the mode of file.The data of reading are transmitted to network interface card by DMA mechanism and are played back by each thread, and final network interface card passes through light
Fibre transfers out signals revivification.
So using the above method, realize to the express network signal acquisition of more specifications and playback, more main bodies of specification
It can support the signals such as 10G Ethernets and the 10G SDH of current mainstream now.Realize the transmitting-receiving to ethernet ip initial data frame
Bag and rule-based filtering function, possess collection, transmission, convergence, distribution and rule-based filtering.Relative to High_speed NIC, in application program
Under conditions of not blocking, it is ensured that zero packet loss.Simultaneously as using technologies such as DMA and its zero-copy so that high-speed link
Collection and playback are not take up any cpu resource.Equally, system also possess for SDH/SONET initial data frames collection,
The data of a variety of SDH/SONET links can be carried out frame synchronization, collection, convergence and distribution by playback function.At the same time can also be to it
In IP data packets extracted.
Wherein, in S1, the High_speed NIC is acquired the high speed signal of different size according to corresponding strategy, converges
And distribution, as shown in figure 3, specifically include to the collection of STM64 frames and the extraction of IP bags, wherein, it is described that STM64 frames are gathered specifically
For：After carrying out clock recovery, serioparallel exchange, frame synchronization and descrambling to the high speed signal collected, original frame data is obtained, and will
The original frame data is transmitted in the queue of host computer side；The extraction of the IP bags is specially：To the SDH agreement solutions of IP signaling bearers
Analysis reduction, and load balancing is into multiple receiving queues of host computer side.
Wherein, the SDH agreements include PoS frames, gfp frame and the WAN frames of carrying.
Since the speed of network signal is very high, general system cannot store the data of collection in real time.Can only
First collection one piece of data is placed in memory, then stops collection, the data in memory are slowly stored in hard disk, are then continued to
Collection, then deposit.The acquisition mode of this interruption, causes the discontinuous of gathered data, the application to needing continuous big data quantity
It is clearly unsuitable.
In the present embodiment, long-time continuous acquisition and real-time storage function are mainly realized by the improvement of two aspects.
First is that storage employs high performance SSD storage cards.Second, using the reception to signal, processing and deposit operation using more
Thread adds loop buffer mechanism.
In the present embodiment, in S2, the multithreading includes：Data receiver thread, data processing threads and data storage line
Journey, the data receiver thread are used to receive data from High_speed NIC, and the data processing threads are used for data analysis, the number
It is used for data according to deposit and writes the SSD high-performance storage card, the buffer circle is used for the work for coordinating multithreading.
In the prior art, the reception and deposit of data are only carried out with single thread mode, since signal rate is too high, often
There is the situation for losing data.In the present embodiment, in order to avoid it is such the occurrence of, data receiver line has been respectively created in system
Journey, data processing threads and data storage thread, Each performs its own functions for three classes thread, it is each to the greatest extent its then, it is ensured that data acquisition it is real-time
Property and continuity.Data receiver thread is responsible for receiving data from network interface card, and data processing threads are responsible for data analysis, and data storage is born
Data write-in high speed storing card is blamed, and the buffer circle without lock enables multiple threads more efficiently to cooperate, will not
It is blocked.This aspect has given full play to the advantage of polycaryon processor, improves the performance and efficiency of data processing；The opposing party
Face, introducing for buffering area can be to avoid causing the loss of data because of shake.
In the present embodiment, in S3, the multithreading includes reading thread and playback thread, and the reading thread is used to read
Storage data simultaneously do the processing of some bags, and playback thread is used to send data packet；The reading thread is arranged to multiple, each described
One piece of cache blocks that thread corresponds to fixed size are read, every piece of cache blocks are both provided with read-write mark, all cache blocks
Form buffer circle.
S3 is specially：When the reading thread judges that its corresponding cache blocks is writeable, the file that will be read from hard disk
It is put into cache blocks, and is readable by the traffic sign placement of cache blocks；When playback thread travels through the buffer circle in sequence
When, whether the mark for first determining whether cache blocks is readable, if it is, the data of cache blocks are transmitted, then will caching
The traffic sign placement of block is writeable, is repeated the above process, untill file reads and is sent completely.
When the single big file to deposit is continuously played back, due to the linear speed playback for needing to reach 10Gbps, bottleneck
It is the speed that file is read from hard disk, general hard-disk interface does not reach so high speed.In such a system, employ
The technical solution of high-performance Sasobit and SBR D disks, this solves the problems, such as interface rate.However, if only go to read text using single thread
Part, due to the limitation of single processor performance, reading rate does not still reach linear speed, and this requires must use multithreading place to go
Reason.But gone to read single file with multiple threads, and orderly transmission is needed, this management and control to multithreading proposes
The requirement of higher is, it is necessary to there is a good mechanism to ensure that the collaboration between multiple threads works in order.The embodiment of the present invention,
Using the synergistic mechanism of multithreading, the continuous linear speed playback of big file is realized.
The multi-thread mechanism of Ethernet data playback is specifically described below, its operation principle is as shown in Figure 4.System
In devise two class threads, one kind is that another kind of is to be used for for reading storage data and doing reading threads of some bag processing
Send the playback thread of data packet.Because if only creating independent reading thread, file is read out from the beginning to the end, it is real
It is now simple, but reading speed does not reach 10Gbps, can not meet the requirement of system linear speed, so multiple threads must be created at the same time
File is read out.One piece of cache blocks of fixed size are distributed for each thread that reads in the present system, and are set read-write
Mark, these cache blocks form a buffer circle.When thread judges that its cache blocks is writeable, will each be read from hard disk
File be put into cache blocks, then put upper readable mark to cache blocks.When playback thread travels through buffer circle in sequence
When, first determine whether the readable mark of cache blocks, if readable, the data of cache blocks are transmitted, then writeable mark is set
Will.Reading thread, which is constantly in, waits cache blocks writeable, when it is judged that cache blocks are writeable, then continuation is read data from hard disk and is put into
In cache blocks, circulated with this, until file is sent completely.
In a preferred embodiment of the invention, a certain size is reserved in the front end that each reading thread corresponds to cache blocks
Space, after corresponding data are write cache blocks by each thread, according to the long instruction of each bag, detection caches data in the block
Whether bag is complete, if the data packet of detection to the end is imperfect, which is removed from current cache blocks
Move on in the follow-up adjacent front end headspace for reading the corresponding cache blocks of thread, with the follow-up adjacent number read in thread
It is stitched together according to another part content of bag, forms a complete data packet.
During data readback, because data type is numerous on Ethernet, and each type of bag length is not of uniform size, so
The long data of a variety of bags must be included in the file collected.When per thread alternately reads file with fixed block size,
Occur that last being surrounded by may not be a complete data packet situation in cache blocks.This is influenced whether at follow-up bag
Reason, causes playback file imperfect.In order to solve the problems, such as this, in the present embodiment, the mechanism of reserved front buffer is employed.Exist
A certain size space is reserved in the front end that per thread corresponds to cache blocks, when corresponding data are write cache blocks by each thread
Afterwards, according to the long instruction of each bag, detect whether data packet in the block is complete, if the data packet of detection to the end is not complete
Bag, that just moves the incomplete data packet in this part from current block the front end headspace to subsequent thread cache blocks
In, it is stitched together with the part data packet in thread below, so as to form a complete data packet.So ensure each
It is all complete to cache data packet in the block, and necessary condition is created for subsequent packet processing and playback.
By using above-mentioned technical proposal disclosed by the invention, following beneficial effect has been obtained：The embodiment of the present invention carries
Collection and playback system and method when the more specification 10Gbps network signals supplied are long, can realize at the same time to 10Gbps Ethernets and
The collection and playback of the express network signal of the plurality of specifications such as 10Gbps SDH, solve conventional single specification signal acquisition and
Playback mode.Meanwhile with high speed NAND FLASH storage dishes and multithreading so that the system can reach 10Gbps networks
The collection of long-time linear speed and playback of signal, and real-time continuous storage can be carried out to the signal of collection.In addition, the system carries
For pressing linear speed, timestamp or constant speed playback function to signal, while data readback number can be specified, be defaulted as Infinite Cyclic
Playback mode, so as to provide multiple means for data signal analysis.
The above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications also should
Depending on protection scope of the present invention.
Claims (8)
- Collection and playback system when 1. a kind of more specification 10Gbps network signals are long, it is characterised in that including：Industrial personal computer, 10GE/ 10GSDH High_speed NICs and SSD high-performance storage cards, the industrial personal computer pass through PCI-E interface and the 10GE/10GSDH respectively High_speed NIC is connected with the SSD high-performance storage card；The 10GE/10GSDH High_speed NICs are connected by optical fiber and data source Connect；The SSD high-performance storage card includes multiple FPGA, each controller respectively with multiple NANDFLASH chips and DDR connections.
- Collection and back method when 2. a kind of more specification 10Gbps network signals are long, it is characterised in that using described in claim 1 More specification 10Gbps network signals it is long when collection and playback system, include the following steps：S1,10G Ethernet or 10GSDH signals are by intelligent acess High_speed NIC, high speed of the High_speed NIC to different size Signal is acquired, converges and distributes according to corresponding strategy；S2, after the high speed signal of distribution reaches the memory of industrial personal computer by DMA, operating system uses multithreading and loop buffer Area, receives high speed signal, is handled respectively, and is stored in SSD high-performance storage cards；S3, by the way of multithreading reads file, and in the deblocking caching industrial personal computer memory of reading；S4, reads data, High_speed NIC is transmitted to by DMA；S5, High_speed NIC transfer out the data convert of reading by optical fiber.
- Collection and back method when 3. more specification 10Gbps network signals according to claim 2 are long, it is characterised in that S1 In, the High_speed NIC is acquired, converges and distributes according to corresponding strategy to the high speed signal of different size, specifically includes To the collection of STM64 frames and the extraction of IP bags, wherein, it is described to be specially to the collection of STM64 frames：The high speed signal collected is carried out After clock recovery, serioparallel exchange, frame synchronization and descrambling, original frame data is obtained, and the original frame data is transmitted to host computer side Queue in；The extraction of the IP bags is specially：The SDH protocol analysis of IP signaling bearers is reduced, and load balancing is to host In multiple receiving queues of side.
- Collection and back method when 4. more specification 10Gbps network signals according to claim 3 are long, it is characterised in that institute Stating SDH agreements includes PoS frames, gfp frame and the WAN frames of carrying.
- Collection and back method when 5. more specification 10Gbps network signals according to claim 2 are long, it is characterised in that S2 In, the multithreading includes：Data receiver thread, data processing threads and data storage thread, the data receiver thread are used In receiving data from High_speed NIC, the data processing threads are used for data analysis, and the data storage is used for data write-in institute SSD high-performance storage cards are stated, the buffer circle is used for the work for coordinating multithreading.
- Collection and back method when 6. more specification 10Gbps network signals according to claim 5 are long, it is characterised in that S3 In, the multithreading includes reading thread and playback thread, and the reading thread is used to read storage data and does at some bags Reason, playback thread are used to send data packet；The reading thread is arranged to multiple, and each reading thread is corresponding fixed big One piece of small cache blocks, every piece of cache blocks are both provided with read-write mark, all cache blocks composition buffer circle.
- Collection and back method when 7. more specification 10Gbps network signals according to claim 6 are long, it is characterised in that S3 Specially：When the reading thread judges that its corresponding cache blocks is writeable, the file read from hard disk is put into cache blocks In, and be readable by the traffic sign placement of cache blocks；When playback thread travels through the buffer circle in sequence, first determine whether Whether the mark of cache blocks is readable, if it is, the data of cache blocks are transmitted, then by the traffic sign placement of cache blocks To be writeable, repeat the above process, untill file reads and is sent completely.
- Collection and back method when 8. more specification 10Gbps network signals according to claim 7 are long, it is characterised in that A certain size space is reserved in the front end that each reading thread corresponds to cache blocks, is cached when each thread writes corresponding data After block, according to the long instruction of each bag, detection caches whether data packet in the block is complete, if the data packet of detection to the end is endless It is whole, then the incomplete content is moved from current cache blocks to before the follow-up adjacent corresponding cache blocks of reading thread Hold in headspace, be stitched together with another part content of the data packet in follow-up adjacent reading thread, form one Complete data packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711242627.8A CN107995061A (en) | 2017-11-30 | 2017-11-30 | Collection and playback system and method when more specification 10Gbps network signals are long |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711242627.8A CN107995061A (en) | 2017-11-30 | 2017-11-30 | Collection and playback system and method when more specification 10Gbps network signals are long |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107995061A true CN107995061A (en) | 2018-05-04 |
Family
ID=62034905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711242627.8A Pending CN107995061A (en) | 2017-11-30 | 2017-11-30 | Collection and playback system and method when more specification 10Gbps network signals are long |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107995061A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109408434A (en) * | 2018-09-03 | 2019-03-01 | 上海威固信息技术股份有限公司 | A kind of multipath high-speed AD data acquisition and storage system based on FPGA |
| CN109861782A (en) * | 2018-12-07 | 2019-06-07 | 天津光电通信技术有限公司 | A kind of SDH protocol signal analysis platform based on PCIE analog input card |
| CN110209358A (en) * | 2019-06-05 | 2019-09-06 | 哈尔滨工业大学 | A kind of NVMe equipment storage speed method for improving based on FPGA |
| CN110647477A (en) * | 2018-06-27 | 2020-01-03 | 广州神马移动信息科技有限公司 | Data caching method, device, terminal and computer readable storage medium |
| CN111274175A (en) * | 2020-01-15 | 2020-06-12 | 杭州华冲科技有限公司 | DMA working method based on data ping-pong filling |
| CN113890887A (en) * | 2021-09-26 | 2022-01-04 | 苏州众联运动科技有限公司 | Data forwarding method |
| WO2022001430A1 (en) * | 2020-06-29 | 2022-01-06 | 中兴通讯股份有限公司 | High-throughput flow processing method and apparatus, and computer-readable storage medium |
| CN113630442B (en) * | 2021-07-14 | 2023-09-12 | 远景智能国际私人投资有限公司 | Data transmission method, device and system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6587470B1 (en) * | 1999-03-22 | 2003-07-01 | Cisco Technology, Inc. | Flexible cross-connect with data plane |
| CN101847429A (en) * | 2010-04-13 | 2010-09-29 | 苏州达通泰科信息技术有限公司 | Multi-flash memory parallel storage device with network repeater function |
| CN102447628A (en) * | 2011-12-22 | 2012-05-09 | 苏州阔地网络科技有限公司 | Data packet transmission processing method and system |
| CN102710424A (en) * | 2012-05-30 | 2012-10-03 | 曙光信息产业(北京)有限公司 | Gigabit/10-gigabit multifunctional network card and implementation method for same |
| CN103885909A (en) * | 2014-03-26 | 2014-06-25 | 国电南瑞科技股份有限公司 | SSD controller based on native PCIe interface and control method thereof |
| CN205003680U (en) * | 2015-08-05 | 2016-01-27 | 杭州世平信息科技有限公司 | A running gear that is used for high speed data to prevent leakage system |
| CN105608027A (en) * | 2015-12-18 | 2016-05-25 | 华为技术有限公司 | Non-volatile storage device and method for accessing non-volatile storage device |
| CN107124286A (en) * | 2016-02-24 | 2017-09-01 | 深圳市知穹科技有限公司 | A kind of mass data high speed processing, the system and method for interaction |
| CN207530841U (en) * | 2017-11-30 | 2018-06-22 | 北京卓讯科信技术有限公司 | Acquisition and playback system when more specification 10Gbps network signals are long |
-
2017
- 2017-11-30 CN CN201711242627.8A patent/CN107995061A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6587470B1 (en) * | 1999-03-22 | 2003-07-01 | Cisco Technology, Inc. | Flexible cross-connect with data plane |
| CN101847429A (en) * | 2010-04-13 | 2010-09-29 | 苏州达通泰科信息技术有限公司 | Multi-flash memory parallel storage device with network repeater function |
| CN102447628A (en) * | 2011-12-22 | 2012-05-09 | 苏州阔地网络科技有限公司 | Data packet transmission processing method and system |
| CN102710424A (en) * | 2012-05-30 | 2012-10-03 | 曙光信息产业(北京)有限公司 | Gigabit/10-gigabit multifunctional network card and implementation method for same |
| CN103885909A (en) * | 2014-03-26 | 2014-06-25 | 国电南瑞科技股份有限公司 | SSD controller based on native PCIe interface and control method thereof |
| CN205003680U (en) * | 2015-08-05 | 2016-01-27 | 杭州世平信息科技有限公司 | A running gear that is used for high speed data to prevent leakage system |
| CN105608027A (en) * | 2015-12-18 | 2016-05-25 | 华为技术有限公司 | Non-volatile storage device and method for accessing non-volatile storage device |
| CN107124286A (en) * | 2016-02-24 | 2017-09-01 | 深圳市知穹科技有限公司 | A kind of mass data high speed processing, the system and method for interaction |
| CN207530841U (en) * | 2017-11-30 | 2018-06-22 | 北京卓讯科信技术有限公司 | Acquisition and playback system when more specification 10Gbps network signals are long |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110647477A (en) * | 2018-06-27 | 2020-01-03 | 广州神马移动信息科技有限公司 | Data caching method, device, terminal and computer readable storage medium |
| CN110647477B (en) * | 2018-06-27 | 2022-02-11 | 阿里巴巴（中国）有限公司 | Data caching method, device, terminal and computer readable storage medium |
| CN109408434A (en) * | 2018-09-03 | 2019-03-01 | 上海威固信息技术股份有限公司 | A kind of multipath high-speed AD data acquisition and storage system based on FPGA |
| CN109408434B (en) * | 2018-09-03 | 2021-11-30 | 上海威固信息技术股份有限公司 | Multi-path high-speed AD data acquisition and storage system based on FPGA |
| CN109861782A (en) * | 2018-12-07 | 2019-06-07 | 天津光电通信技术有限公司 | A kind of SDH protocol signal analysis platform based on PCIE analog input card |
| CN110209358A (en) * | 2019-06-05 | 2019-09-06 | 哈尔滨工业大学 | A kind of NVMe equipment storage speed method for improving based on FPGA |
| CN110209358B (en) * | 2019-06-05 | 2022-07-15 | 哈尔滨工业大学 | NVMe equipment storage speed improving method based on FPGA |
| CN111274175A (en) * | 2020-01-15 | 2020-06-12 | 杭州华冲科技有限公司 | DMA working method based on data ping-pong filling |
| WO2022001430A1 (en) * | 2020-06-29 | 2022-01-06 | 中兴通讯股份有限公司 | High-throughput flow processing method and apparatus, and computer-readable storage medium |
| CN113630442B (en) * | 2021-07-14 | 2023-09-12 | 远景智能国际私人投资有限公司 | Data transmission method, device and system |
| CN113890887A (en) * | 2021-09-26 | 2022-01-04 | 苏州众联运动科技有限公司 | Data forwarding method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107995061A (en) | Collection and playback system and method when more specification 10Gbps network signals are long | |
| US8505013B2 (en) | Reducing data read latency in a network communications processor architecture | |
| US8935483B2 (en) | Concurrent, coherent cache access for multiple threads in a multi-core, multi-thread network processor | |
| US8514874B2 (en) | Thread synchronization in a multi-thread network communications processor architecture | |
| US8537832B2 (en) | Exception detection and thread rescheduling in a multi-core, multi-thread network processor | |
| Duato et al. | A new scalable and cost-effective congestion management strategy for lossless multistage interconnection networks | |
| US7227841B2 (en) | Packet input thresholding for resource distribution in a network switch | |
| US7283556B2 (en) | Method and system for managing time division multiplexing (TDM) timeslots in a network switch | |
| US8949838B2 (en) | Multi-threaded processing with hardware accelerators | |
| US9444737B2 (en) | Packet data processor in a communications processor architecture | |
| US7406041B2 (en) | System and method for late-dropping packets in a network switch | |
| US8910171B2 (en) | Thread synchronization in a multi-thread network communications processor architecture | |
| US20130042038A1 (en) | Non-blocking processor bus bridge for network processors or the like | |
| US8868889B2 (en) | Instruction breakpoints in a multi-core, multi-thread network communications processor architecture | |
| CN103986931A (en) | Method for transmitting video data on FPGA and DSP structure on basis of SRIO bus | |
| CN108366111A (en) | A kind of data packet low time delay buffer storage and method for switching equipment | |
| CN106027424A (en) | Ethernet exchange device based on RapidIO exchange technology | |
| CN103517085B (en) | Method for implementing remote server management based on video decoding design | |
| US8874878B2 (en) | Thread synchronization in a multi-thread, multi-flow network communications processor architecture | |
| CN106372029A (en) | Point-to-point on-chip communication module based on interruption | |
| CN103731364B (en) | X86 platform based method for achieving trillion traffic rapid packaging | |
| CN105653213A (en) | Double control disk array based on Freescale P3041 | |
| CN116136748B (en) | High-bandwidth NVMe SSD read-write system and method based on FPGA | |
| CN207530841U (en) | Acquisition and playback system when more specification 10Gbps network signals are long | |
| CN110297785A (en) | A kind of finance data flow control apparatus and flow control method based on FPGA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |