CN103678235A - Network data processing device and method based on parallel flow lines - Google Patents
Network data processing device and method based on parallel flow lines Download PDFInfo
- Publication number
- CN103678235A CN103678235A CN201310662519.1A CN201310662519A CN103678235A CN 103678235 A CN103678235 A CN 103678235A CN 201310662519 A CN201310662519 A CN 201310662519A CN 103678235 A CN103678235 A CN 103678235A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- network
- sending
- feedback
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
- Multi Processors (AREA)
Abstract
The invention relates to a network data processing device and method based on parallel flow lines. The device comprises a PCIEX8 bus interface which is connected with an arbitration module through a cache module. The arbitration module is connected with a sending-receiving device through a storage device. The sending-receiving device is connected with an internet access data transmit-receive device. The storage device comprises at least two storage modules. Sending-receiving modules with the number correspondingly consistent with the number of storage device memory modules are arranged in the sending-receiving device. Internet access transmit-receive data modules with the number correspondingly consistent with the number of the storage modules are arranged in the internet access data transmit-receive device. The storage modules in the storage device are correspondingly connected with the sending-receiving modules in the sending-receiving device in a one-to-one mode. The structure is compact, parallel flow line network data processing can be achieved, data processing speed is improved, the application range is wide, and safety and reliability are achieved.
Description
Technical field
The present invention relates to a kind of data processing equipment and method, especially a kind of based on parallel pipeline network data processing device and method, belong to the technical field of network data processing.
Background technology
Parallel transmission refers to that long numeric data transmits by parallel line simultaneously, can greatly improve the speed of data transmission.Streamline refers to that when program is carried out a kind of accurate parallel processing that many effects of overlappings operate realizes technology.How when network data, realizing parallel pipelining process line technology is a current difficult problem.
Summary of the invention
The object of the invention is to overcome the deficiencies in the prior art, provide a kind of based on parallel pipeline network data processing device and method, its compact conformation, can realize the network data processing of parallel pipeline, improve the speed of data processing, wide accommodation, safe and reliable.
According to technical scheme provided by the invention, described based on parallel pipeline network data processing device, comprise PCIE X8 bus interface, described PCIE X8 bus interface is connected with arbitration modules by cache module, described arbitration modules is connected with transceiver by memory storage, described transceiver is connected with network interface transceiving data device, described memory storage comprises at least two memory modules, in transceiver, comprise and in memory storage, store the corresponding consistent sending/receiving module of module number, in network interface transceiving data device, comprise and the corresponding consistent network interface transceiving data module of memory module quantity, memory module in memory storage and the sending/receiving module in transceiver connect one to one, and memory module connects one to one by sending/receiving module and network interface transceiving data module,
PCIE X8 bus interface deposits the transmission data that receive in cache module in, when detecting cache module, arbitration modules has while sending data, to sending the data head of data in cache module, resolve, after transmission data all deposit in cache module, arbitration modules selects corresponding network interface transceiving data module to carry out data transmission according to the data head that sends data; When network interface transceiving data module detects network feedback data, packet number and the byte number of sending/receiving module statistics network feedback data, and packet number and the byte number together with statistics all deposits in the memory module being connected with described sending/receiving module by network feedback data; Arbitration modules reads the data of storage in memory module and adds required data head and receives data to form network-feedback, and described network-feedback reception data are deposited in cache module, for PCIE X8 bus interface, reads.
Described arbitration modules is numbered the network interface transceiving data module in network interface transceiving data device, and the data head that arbitration modules is added comprises the numbering that is connected network interface transceiving data module with memory module, packet number and the byte number of network feedback data.
Described PCIE X8 bus interface is connected with host computer, and the network-feedback that host computer reads in cache module by PCIE X8 bus interface receives data, and can transmit required transmission data to PCIE X8 bus interface.
Based on a parallel pipeline network data processing method, described network data processing method comprises the steps:
A, by PCIE X8 bus interface, receive to send data, and described transmission data are deposited in cache module;
B, arbitration modules detect in real time to cache module, and while having data in cache module, arbitration modules is resolved sending the data head of data in cache module; Arbitration modules sends data according to the data head judgement that sends data and whether enters cache module completely, and after transmission data enter in cache module completely, arbitration modules deposits transmission data in corresponding memory module according to the data head that sends data;
C, the sending/receiving module being connected with memory module detect and send after data, and sending/receiving module will send data and by network interface transceiving data module, will send data and outwards send;
D, by network interface transceiving data module, receive network feedback data, sending/receiving module is added up packet number, the byte number of network feedback data, and network feedback data is all deposited in memory module together with packet number, the byte number of statistics;
E, arbitration modules read network feedback data, packet number, the byte number in memory module, and according to the numbering of the network interface transceiving data module being connected with memory module, network feedback data are added to data head, to form network-feedback, receive data;
F, arbitration modules receive data by network-feedback and deposit in cache module, read described network-feedback receive data for PCIE X8 bus interface.
While all having network feedback data in having a plurality of memory modules, arbitration modules reads successively according to the order of network interface transceiving data module numbering.
In described step b, arbitration modules is resolved the numbering of the network interface transceiving data module that obtains sending packet number, the byte number of data and outwards send to sending the data head of data.
Advantage of the present invention: receive transmission data by PCIE X8 bus interface, arbitration modules receives sending data, judgement and parsing, and send by corresponding network interface transceiving data module, the network feedback data that arbitration modules receives network interface transceiving data module is added data head and is formed network-feedback reception data, network-feedback is received to data to be deposited in cache module, after reading for PCIE X8 bus interface, be transferred to host computer, its compact conformation, can realize the network data processing of parallel pipeline, improve the speed of data processing, wide accommodation, safe and reliable.
Accompanying drawing explanation
Fig. 1 is structured flowchart of the present invention.
Fig. 2 is that arbitration modules of the present invention deposits transmission data in the process flow diagram of memory storage.
Fig. 3 is the process flow diagram that the present invention's the first network interface transceiving data module is carried out data transmission.
Fig. 4 is that arbitration modules of the present invention is to receiving the process flow diagram of data processing.
Description of reference numerals: 1-PCIE X8 bus interface, 2-cache module, 3-arbitration modules, 4-memory storage, 5-transceiver and 6-network interface transceiving data device.
Embodiment
Below in conjunction with concrete drawings and Examples, the invention will be further described.
As shown in Figure 1: in order to realize by a PCIE X8 bus interface 1 network data processing of parallel pipeline, the present invention includes PCIE X8 bus interface 1, described PCIE X8 bus interface 1 is connected with arbitration modules 3 by cache module 2, described arbitration modules 3 is connected with transceiver 5 by memory storage 4, described transceiver 5 is connected with network interface transceiving data device 6, described memory storage 4 comprises at least two memory modules, in transceiver 5, comprise and the corresponding consistent sending/receiving module of the interior storage module number of memory storage 4, in network interface transceiving data device 6, comprise and the corresponding consistent network interface transceiving data module of memory module quantity, sending/receiving module in memory module in memory storage 4 and transceiver 5 connects one to one, and memory module connects one to one by sending/receiving module and network interface transceiving data module,
PCIE X8 bus interface 1 deposits the transmission data that receive in cache module 2 in, when detecting cache module 2, arbitration modules 3 has while sending data, data head to the interior transmission data of cache module 2 is resolved, when send data all deposit in cache module 2 interior after, arbitration modules 3 selects corresponding network interface transceiving data module to carry out data transmission according to the data head that sends data; When network interface transceiving data module detects network feedback data, packet number and the byte number of sending/receiving module statistics network feedback data, and packet number and the byte number together with statistics all deposits in the memory module being connected with described sending/receiving module by network feedback data; Arbitration modules 3 reads the data of storage in memory module and adds required data head and receives data to form network-feedback, and described network-feedback reception data are deposited in cache module 2, for PCIE X8 bus interface 1, reads.
Particularly, the memory block that cache module 2 circulates between PCIE X8 bus interface 1 and arbitration modules 3 for data and needs, PCIE X8 bus interface 1 is existing conventional a kind of bus interface form.In the embodiment of the present invention, memory storage 4 at least comprises two memory modules, memory storage 4 has been shown in Fig. 1 and has comprised four memory modules, be i.e. the first memory module, the second memory module, the 3rd memory module and the 4th memory module.When memory storage 4 comprises four memory modules, in transceiver 5, comprise four sending/receiving modules, meanwhile, in network interface transceiving data device 6, comprise four network interface transceiving data modules (MAC); In order to realize one to one, connect, the first memory module is connected with the first sending/receiving module, the first sending/receiving module is connected with the first network interface transceiving data module, the second memory module is connected with the second sending/receiving module, the second sending/receiving module is connected with the second network interface transceiving data module, the 3rd memory module is connected with the 3rd sending/receiving module, the 3rd sending/receiving module is connected with the 3rd network interface transceiving data module, the 4th memory module is connected with the 4th sending/receiving module, and the 4th sending/receiving module is connected with the 4th network interface transceiving data module.In the specific implementation, the interior memory module of memory storage 4 can, more than four, send the quantity comprising in reception 5 and network interface transceiving data device 6 corresponding consistent with memory module, can meet the needs that connect one to one.
Further, network interface transceiving data module in 3 pairs of network interface transceiving data devices 6 of arbitration modules is numbered, when carrying out data transmission, can select to send by the network interface transceiving data module of reference numeral, when carrying out data receiver, can add data head according to the numbering of network interface transceiving data module.A plurality of network interface transceiving data modules that the present invention connects by arbitration modules 3 are carried out the operation of data transmit-receive simultaneously, realize the function that parallel data sends.
Network interface transceiving data module in 3 pairs of network interface transceiving data devices 6 of described arbitration modules is numbered, and the data head that arbitration modules 3 is added comprises the numbering that is connected network interface transceiving data module with memory module, packet number and the byte number of network feedback data.
Described PCIE X8 bus interface 1 is connected with host computer, and the network-feedback that host computer 1 reads in cache module 2 by PCIE X8 bus interface 1 receives data, and can be to the required transmission data of PCIE X8 bus interface 1 transmission.
As shown in Figure 2, Figure 3 and Figure 4: a kind of based on parallel pipeline network data processing method, described network data processing method comprises the steps:
A, by PCIE X8 bus interface 1, receive and send data, and described transmission data are deposited in cache module 2;
The transmission data of PCIE X8 bus interface 1 are that host computer sends, and the transmission data of host computer transmission have been specified and need to have been sent by the network interface transceiving data module of corresponding numbering.
B, 3 pairs of cache modules 2 of arbitration modules detect in real time, and while having data in cache module 2, the data head of 3 pairs of interior transmission data of cache module 2 of arbitration modules is resolved; Arbitration modules 3 sends data according to the data head judgement that sends data and whether enters cache module 2 completely, when send data enter completely cache module 2 interior after, arbitration modules 3 deposits transmission data in corresponding memory module according to the data head that sends data;
In described step b, the data head of 3 pairs of transmission data of arbitration modules is resolved the numbering of the network interface transceiving data module that obtains sending packet number, the byte number of data and outwards send.Arbitration modules 3 can judge according to the packet number of resolving and byte number whether send data enters in cache module 2 completely.In parsing, obtain sending after the numbering of the network interface transceiving data module of setting, arbitration modules 3 deposits transmission data with network interface transceiving data module in and numbers in corresponding memory module, while sending as passed through the first network interface transceiving data module, need to deposit in the first memory module sending data.
C, the sending/receiving module being connected with memory module detect and send after data, and sending/receiving module will send data and by network interface transceiving data module, will send data and outwards send;
Further, when the data head of 3 pairs of transmission data of arbitration modules is resolved, can also obtain the number of times that described transmission data need to send, when sending by network interface transceiving data module, also need to detect the number of times sending and whether reach the transmission times of setting in the data head that sends data, when not arriving transmission times, need to repeat to send, until meet the transmission times of setting, thereby realize the object that streamline sends.
D, by network interface transceiving data module, receive network feedback data, sending/receiving module is added up packet number, the byte number of network feedback data, and network feedback data is all deposited in memory module together with packet number, the byte number of statistics;
E, arbitration modules 3 read network feedback data, packet number, the byte number in memory module, and according to the numbering of the network interface transceiving data module being connected with memory module, network feedback data are added to data head, to form network-feedback, receive data;
While all having network feedback data in having a plurality of memory modules, arbitration modules 3 reads successively according to the order of network interface transceiving data module numbering.In the embodiment of the present invention, described order reads the order referring to according to network interface transceiving data module is numbered, as read successively according to the order of the first network interface transceiving data module, the second network interface transceiving data module, the 3rd network interface transceiving data module and the 4th network interface transceiving data module.
F, arbitration modules 3 receive data by network-feedback and deposit in cache module 2, read described network-feedback receive data for PCIE X8 bus interface 1.
The present invention receives transmission data by PCIE X8 bus interface 1, 3 pairs of arbitration modules send data and receive, judgement and parsing, and send by corresponding network interface transceiving data module, the network feedback data that 3 pairs of network interface transceiving data modules of arbitration modules receive is added data head and is formed network-feedback reception data, network-feedback is received to data to be deposited in cache module 2, after reading for PCIE X8 bus interface 1, be transferred to host computer, its compact conformation, can realize the network data processing of parallel pipeline, improve the speed of data processing, wide accommodation, safe and reliable.
Claims (6)
1. one kind based on parallel pipeline network data processing device, it is characterized in that: comprise PCIE X8 bus interface (1), described PCIE X8 bus interface (1) is connected with arbitration modules (3) by cache module (2), described arbitration modules (3) is connected with transceiver (5) by memory storage (4), described transceiver (5) is connected with network interface transceiving data device (6), described memory storage (4) comprises at least two memory modules, in transceiver (5), comprise and the corresponding consistent sending/receiving module of storage module number in memory storage (4), in network interface transceiving data device (6), comprise and the corresponding consistent network interface transceiving data module of memory module quantity, memory module in memory storage (4) and the sending/receiving module in transceiver (5) connect one to one, and memory module connects one to one by sending/receiving module and network interface transceiving data module,
PCIE X8 bus interface (1) deposits the transmission data that receive in cache module (2) in, when detecting cache module (2), arbitration modules (3) has while sending data, to sending the data head of data in cache module (2), resolve, after transmission data all deposit in cache module (2), arbitration modules (3) selects corresponding network interface transceiving data module to carry out data transmission according to the data head that sends data; When network interface transceiving data module detects network feedback data, packet number and the byte number of sending/receiving module statistics network feedback data, and packet number and the byte number together with statistics all deposits in the memory module being connected with described sending/receiving module by network feedback data; Arbitration modules (3) reads the data of storage in memory module and adds required data head and receives data to form network-feedback, and described network-feedback reception data are deposited in cache module (2), for PCIE X8 bus interface (1), reads.
2. according to claim 1 based on parallel pipeline network data processing device, it is characterized in that: described arbitration modules (3) is numbered the network interface transceiving data module in network interface transceiving data device (6), the data head that arbitration modules (3) is added comprises the numbering that is connected network interface transceiving data module with memory module, packet number and the byte number of network feedback data.
3. according to claim 1 based on parallel pipeline network data processing device, it is characterized in that: described PCIE X8 bus interface (1) is connected with host computer, the network-feedback that host computer (1) reads in cache module (2) by PCIE X8 bus interface (1) receives data, and can transmit required transmission data to PCIE X8 bus interface (1).
4. based on a parallel pipeline network data processing method, it is characterized in that, described network data processing method comprises the steps:
(a), by PCIE X8 bus interface (1), receive to send data, and described transmission data are deposited in cache module (2);
(b), arbitration modules (3) detects in real time to cache module (2), while having data in cache module (2), arbitration modules (3) is resolved sending the data head of data in cache module (2); Arbitration modules (3) sends data according to the data head judgement that sends data and whether enters cache module (2) completely, after transmission data enter in cache module (2) completely, arbitration modules (3) deposits transmission data in corresponding memory module according to the data head that sends data;
(c), the sending/receiving module that is connected with memory module detects and sends after data, sending/receiving module will send data and by network interface transceiving data module, will send data and outwards send;
(d), by network interface transceiving data module, receive network feedback data, sending/receiving module is added up packet number, the byte number of network feedback data, and network feedback data is all deposited in memory module together with packet number, the byte number added up;
(e), arbitration modules (3) reads network feedback data, packet number, the byte number in memory module, and according to the numbering of the network interface transceiving data module being connected with memory module, network feedback data is added to data head, to form network-feedback, receive data;
(f), arbitration modules (3) receives data by network-feedback and deposits in cache module (2), for PCIE X8 bus interface (1), read described network-feedback and receive data.
5. according to claim 4ly based on parallel pipeline network data processing method, it is characterized in that, while all having network feedback data in having a plurality of memory modules, arbitration modules (3) reads successively according to the order of network interface transceiving data module numbering.
6. according to claim 4 based on parallel pipeline network data processing method, it is characterized in that, in described step (b), arbitration modules (3) is resolved the numbering of the network interface transceiving data module that obtains sending packet number, the byte number of data and outwards send to sending the data head of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310662519.1A CN103678235B (en) | 2013-12-09 | 2013-12-09 | Based on parallel pipeline network data processing device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310662519.1A CN103678235B (en) | 2013-12-09 | 2013-12-09 | Based on parallel pipeline network data processing device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103678235A true CN103678235A (en) | 2014-03-26 |
| CN103678235B CN103678235B (en) | 2016-06-08 |
Family
ID=50315851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310662519.1A Active CN103678235B (en) | 2013-12-09 | 2013-12-09 | Based on parallel pipeline network data processing device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103678235B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9753876B1 (en) | 2016-09-22 | 2017-09-05 | International Business Machines Corporation | Processing of inbound back-to-back completions in a communication system |
| CN109687943A (en) * | 2018-10-11 | 2019-04-26 | 中国人民解放军海军陆战队训练基地 | A kind of dual-host backup redundancy control system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090063728A1 (en) * | 2007-08-27 | 2009-03-05 | Arimilli Lakshminarayana B | System and Method for Direct/Indirect Transmission of Information Using a Multi-Tiered Full-Graph Interconnect Architecture |
| CN102394732A (en) * | 2011-09-06 | 2012-03-28 | 中国人民解放军国防科学技术大学 | Multi-micropacket parallel processing structure |
| CN102650976A (en) * | 2012-04-01 | 2012-08-29 | 中国科学院计算技术研究所 | Control device and method supporting single IO (Input/Output) virtual user level interface |
| CN102855090A (en) * | 2012-07-23 | 2013-01-02 | 深圳市江波龙电子有限公司 | Storage equipment and running method thereof |
| CN103109260A (en) * | 2010-05-04 | 2013-05-15 | 谷歌公司 | Parallel processing of data |
-
2013
- 2013-12-09 CN CN201310662519.1A patent/CN103678235B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090063728A1 (en) * | 2007-08-27 | 2009-03-05 | Arimilli Lakshminarayana B | System and Method for Direct/Indirect Transmission of Information Using a Multi-Tiered Full-Graph Interconnect Architecture |
| CN103109260A (en) * | 2010-05-04 | 2013-05-15 | 谷歌公司 | Parallel processing of data |
| CN102394732A (en) * | 2011-09-06 | 2012-03-28 | 中国人民解放军国防科学技术大学 | Multi-micropacket parallel processing structure |
| CN102650976A (en) * | 2012-04-01 | 2012-08-29 | 中国科学院计算技术研究所 | Control device and method supporting single IO (Input/Output) virtual user level interface |
| CN102855090A (en) * | 2012-07-23 | 2013-01-02 | 深圳市江波龙电子有限公司 | Storage equipment and running method thereof |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9753876B1 (en) | 2016-09-22 | 2017-09-05 | International Business Machines Corporation | Processing of inbound back-to-back completions in a communication system |
| CN109687943A (en) * | 2018-10-11 | 2019-04-26 | 中国人民解放军海军陆战队训练基地 | A kind of dual-host backup redundancy control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103678235B (en) | 2016-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103490966B (en) | A kind of dual-redundant CAN bus method for receiving and processing data | |
| CN103490959B (en) | A kind of dual-redundant CAN bus fault detection method | |
| CN104750588B (en) | A kind of method for testing pressure based on serial communication | |
| CN104199795B (en) | Data transferring and receiving method of bus framework | |
| CN102420719B (en) | Apparatus for testing PCIe bus bandwidth and method thereof | |
| CN104281548A (en) | Method, device and system for data transmission based on AXI bus | |
| CN103178872B (en) | Method and the device of USB system transfers distance is extended by Ethernet | |
| CN105791439A (en) | Data processing method, data receiving apparatus and data processing system | |
| CN103617138A (en) | Multi-mainframe arbitration method and multi-mainframe communication system | |
| CN103440187A (en) | Hardware script-based automated CAN (Controller Area Network) bus test method | |
| CN103678235B (en) | Based on parallel pipeline network data processing device and method | |
| CN103428050A (en) | Multipath CAN (controller area network) simulation system based on CAN bus | |
| CN103701776A (en) | Network data card and data transmission method based on FPGA (Field Programmable Gate Array) | |
| CN104360973A (en) | Communicating system, communicating method and relevant communicating devices | |
| CN201341158Y (en) | Stateful inspection circuit for RS485 network | |
| CN204597988U (en) | The AFDX terminal test equipment of Based PC PCI interface | |
| GB2523865A (en) | Methods and systems for sending and receiving information data | |
| CN102946335A (en) | Network condition detection method and network condition detection system | |
| CN103441894A (en) | Method and system for L2TP concurrent connection performance test | |
| CN203982361U (en) | A kind of multiple serial communication proving installation | |
| CN108769134B (en) | Multi-protocol dual-host communication method | |
| CN103517307B (en) | A kind of remote debugging system based on TD-SCDMA | |
| CN105897494A (en) | Testing method of network transmission rate | |
| CN103531002B (en) | A kind of remote debugging method based on TD-SCDMA | |
| CN110096002A (en) | A kind of automatization test system and test method based on CANFD bus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |