The method of high speed veneer message transmission, field programmable gate array and high speed veneer
Technical field
The present invention relates to technical field of data transmission, be specifically related to a kind of method, the field programmable gate array on a kind of high speed veneer (FPGA, Field Programmable GateArray) and a kind of high speed veneer of high speed veneer message transmission.
Background technology
For Peripheral Component Interconnect (PCI, Peripheral Component Interconnect) Express promptly: PCIe bussing technique or traditional pci bus, the transmission of data is normally based on direct memory access (DMA, Direct Memory Access) mechanism.The major advantage of this operation is a workload of alleviating processor, usually use dma controller to initiate direct memory access, thereby under the situation of not using processor, finish moving of data, general chip system (SOC, System on Chip) processor all can provide a plurality of dma controllers for the user.
Fig. 1 has provided based on the transfer of data schematic diagram between each veneer on the router of PCIe bussing technique, as shown in Figure 1, each veneer comprises: the CPU (CPU on master control borad, the wiring board, Central Processing Unit) initialization dma controller at first, in the internal memory of each veneer, set up sending and receiving buffer memory descriptor (BD, Buffer Descriptor) chain, every the BD chain all is equivalent to a first-in first-out (FIFO, First Input First Output) formation, send out BD and be used for the buffer memory outgoing data, receive BD and be used for the data that buffer memory receives.Each BD chain comprises BD head and BD buffering area two parts, and the BD head is indicated the various information of this BD, as: there are the data that need transmission or received the addresses of BD state, length and buffering area etc., BD buffering area.
Most important in the DMA mechanism is exactly the management of sending and receiving BD chain, the CPU of each veneer will be with all potential source veneers of receipts BD address notification of this veneer, also the sky of receiving BD in time to be reported all potential source veneers at full situation, to prevent owing to receiving the full phenomenon that causes overflowing packet loss of BD.
As shown in Figure 1, when wiring board 1 will be to wiring board m message transmission, the CPU of wiring board 1 at first puts into message and sends out BD, judge subsequently whether wiring board m has enough receipts BD to receive message,, then wait for if do not have, if have, then start dma controller, finish message transmissions by dma controller, and pass through the mode announcement trunk plate m of interruption or message: message is end of transmission.
Main transmission two class messages between the veneer: a class is the control message, comprises the message packet of Flow Control message and notification data end of transmission; Another kind of is data message.
Based on processor performance, wiring board can be divided into two kinds: low speed line plate and high-speed line plate.The low speed line plate is mainly based on single core processor, and such processor contains the PCIe controller, can directly pass through the interconnected communication of exchange chip between the processor.The high-speed line plate is mainly based on high performance polycaryon processor or network processing unit, such processor is not supported the PCIe interface, general high bandwidth interface is SPI4.2, for realizing high speed business plate based on PCIe, need finish the conversion of PCIe, realize the conversion of PCIe by FPGA at present to SPI4.2 to SPI4.2.
Fig. 2 has provided the schematic diagram of the router that has low speed line plate and high-speed line plate simultaneously, and as shown in Figure 2, the intercommunication between the wiring board mainly contains three kinds of modes:
Mode one, low speed line communication between plates.The intercommunication mode is machine-processed consistent with the DMA among Fig. 1.
Mode two, high-speed line communication between plates.FPGA does not participate in any DMA and handles, and only realizes the conversion of PCIe packet and SPI4.2 data message, and BD initialization and management are still realized by the dma controller that carries by the high-performance CPU on the high-speed line plate.The message content that FPGA neither needs to participate in the DMA management and also do not need to understand transmission is control message or data message, directly load (payload) part of the SPI4.2 packet received is peeled off, and according to the address in the load data message is sent.
Mutual communication between mode three, high, low speed wiring board.This mode is the combination of mode one and mode two.
From technical scheme shown in Figure 2 as can be seen: the CPU of high-speed line plate need carry out the DMA management, can will increase the whole plate efficiency of work load, reduction of CPU like this.
Summary of the invention
The embodiment of the invention provides a kind of method, the FPGA on a kind of high speed veneer and a kind of high speed veneer of high speed veneer message transmission, with the work load of the CPU that reduces the high speed veneer.
The technical scheme of the embodiment of the invention is achieved in that
FPGA on a kind of high speed veneer comprises: dma controller, and this dma controller is safeguarded the BD that DMA is required, FPGA is according to this BD and other single plate interactive message.
Described dma controller comprises: DMA receiver module, BD administration module and DMA sending module, wherein:
The DMA receiver module takes out the message that other veneer is sent by the Peripheral Component Interconnect PCIe bus of FPGA from receive BD, if this message is the BD administrative message, then message is directly sent to the BD administration module; If this message is a data message, then message is sent to central processing unit CPU;
Sending and receiving BD is created and safeguarded to the BD administration module on this high speed veneer, receive the BD administrative message that the DMA receiver module is sent, and manages receipts BD on this high speed veneer according to this message;
The DMA sending module receives the message that CPU sends, and this message is put into a BD send to other veneer.
Described BD administration module further comprises: according to predefined receipts BD update condition, carry the message of this veneer for the empty full condition information of receipts BD chain of this other veneer maintenance to other veneer transmission.
Described FPGA further comprises: receive data cache module, SPI4.2 transmission control module and SPI4.2 Flow Control module,
And, after described dma controller receives the data message that other veneer sends, the metadata cache in the payload segment of message to receiving data cache module, and is sent control module to SPI4.2 and sends existing new outgoing data notice, wherein:
Receive data cache module, be used for the data that the buffer memory dma controller is sent;
SIP4.2 sends control module, receives described existing new outgoing data notice, whether inquires about Flow Control to SPI4.2 Flow Control module, if Flow Control is not taken out data in buffer from receiving data cache module, will send to CPU behind the SPI4.2 control head on the data encapsulation;
SPI4.2 Flow Control module is used for the SPI4.2 passage of this FPGA is carried out Flow Control.
Described FPGA further comprises: SPI4.2 receives control module, sends data cache module, wherein:
SPI4.2 receives control module, receives the data message that CPU sends by the SPI4.2 passage, and the payload segment of this data message is peeled off, and the metadata cache that obtains is arrived send data cache module, and send existing new outgoing data notice to dma controller;
Send data cache module, buffer memory SPI4.2 receives the data that control module is sent;
And described dma controller is received described existing new outgoing data notice, takes out data in buffer from sending data cache module.
A kind of high speed veneer comprises: FPGA and CPU comprise among the FPGA: dma controller, and this dma controller is safeguarded the BD that DMA is required, FPGA is according to this BD and other single plate interactive message.
Described dma controller comprises: DMA receiver module, BD administration module and DMA sending module, wherein:
The DMA receiver module takes out the message that other veneer is sent by the PCIe bus of FPGA from receive BD, if this message is the BD administrative message, then message is directly sent to the BD administration module; If this message is a data message, then message is sent to CPU;
Sending and receiving BD is created and safeguarded to the BD administration module on this high speed veneer, receive the BD administrative message that the DMA receiver module is sent, and manages BD on this high speed veneer according to this message;
The DMA sending module receives the message that CPU sends, and this message is put into a BD send to other veneer.
A kind of method of high speed veneer message transmission, this method comprises:
FPGA on the high speed veneer safeguards the BD that DMA is required, and FPGA is according to this BD and other single plate interactive message.
The FPGA of described high speed veneer safeguards that the required BD of DMA comprises:
FPGA creates respectively for each source veneer of this high speed veneer and safeguards that is received a BD chain, and when the initialization of this veneer, will send to this source veneer for the receipts BD chain information of each source veneer maintenance.
Described FPGA comprises according to BD and other single plate interactive message:
FPGA receives the indication that finishes of message transmissions that the source veneer of this veneer sends, from take out message from the receipts BD as this source veneer maintenance, if message is the BD administrative message, then according to this message management BD; If message is a data message, then message is sent to CPU.
Described FPGA sends to CPU with message and comprises:
Judge that SPI4.2 passage between FPGA and the CPU whether by Flow Control, if not, sends to CPU with message; Otherwise, wait for to SPI4.2 passage during not by Flow Control, message is sent to CPU.
Described FPGA is from from further comprising as taking out message the receipts BD of this source veneer maintenance: FPGA will from as the sky of the receipts BD chain of this source veneer maintenance completely condition information send to this source veneer.
Described FPGA creates respectively for each source veneer of this high speed veneer and safeguards that receiving the BD chain for one further comprises: FPGA creates and safeguards that is sent out a BD chain.
Described FPGA comprises according to BD and other single plate interactive message:
FPGA receives the data message that the CPU of this high speed veneer sends, this message is put into a BD, identify according to the target single board in the message, judge and in the receipts BD chain of safeguarding as this target single board, whether have the idle BD of receipts, if exist, free time is received the BD address put into a BD, start the DMA transport process.
FPGA preestablishes receipts BD chain update condition for each source veneer of this high speed veneer, and when satisfying this condition, FPGA expires condition information with this veneer for the sky of the receipts BD chain of this source veneer maintenance and sends to this source veneer.
Compared with prior art, the embodiment of the invention is by increasing dma controller in the FPGA of high speed veneer, this dma controller is safeguarded the BD that DMA is required, FPGA is according to this BD and other single plate interactive message, discharged the DMA management work of the CPU of high speed veneer, reduce the work load of CPU, improved the whole plate efficiency of high speed veneer.
Description of drawings
Fig. 1 is existing based on the transfer of data schematic diagram between each veneer of router of PCIe bussing technique;
Fig. 2 is the existing schematic diagram that has the router of low speed line plate and high-speed line plate simultaneously;
The schematic diagram of the router of low speed line plate and high-speed line plate is provided when providing for the embodiment of the invention Fig. 3;
The composition schematic diagram of FPGA on the high speed veneer that Fig. 4 provides for the embodiment of the invention;
The schematic diagram of sending out, receive BD that Fig. 5 provides for the embodiment of the invention;
The high speed veneer that Fig. 6 provides for the embodiment of the invention receives the process chart of message;
The high speed veneer that Fig. 7 provides for the embodiment of the invention sends the process chart of message.
Embodiment
The present invention is further described in more detail below in conjunction with drawings and the specific embodiments.
In the embodiment of the invention, high speed master control borad and high-speed line plate are referred to as the high speed veneer.
The schematic diagram of the router of low speed line plate and high-speed line plate is provided when providing for the embodiment of the invention Fig. 3, and as shown in Figure 3, it mainly comprises: master control borad, PCIe switch, low speed line plate and high-speed line plate.Wherein, the 26S Proteasome Structure and Function of PCIe switch, low speed line plate can be same as the prior art, and if master control borad is the low speed veneer, then the 26S Proteasome Structure and Function of master control borad can be same as the prior art.
As shown in Figure 3, comprise dma controller among the FPGA of high-speed line plate, then do not comprise dma controller among the CPU.Simultaneously, if master control borad is the high speed veneer, then the structure of the FPGA in the structure of the FPGA of master control borad and the high-speed line plate shown in Figure 3 is identical: comprise dma controller, and do not comprise dma controller among the CPU of master control borad.
Fig. 4 has provided the composition schematic diagram of the FPGA on the high speed veneer that the embodiment of the invention provides, as shown in Figure 4, FPGA mainly comprises: DMA receiver module 411, reception data cache module 412, SPI4.2 send control module 413, SPI4.2 Flow Control module 414, SPI4.2 reception control module 415, send data cache module 416, DMA sending module 417, BD administration module 418, and each module functions is as follows:
DMA receiver module 411: receive indication message that the source veneer of this veneer the sends interruption or the message of end of transmission, take out message from this veneer for the receipts BD chain of this source veneer maintenance, if this message upgrades message for receiving the BD chain, then message is directly sent to BD administration module 418; If this message is a data message, then the metadata cache in the payload segment of data message is arrived and receive data cache module 412, send to SPI4.2 transmission control module 413 simultaneously and have new outgoing data notice.
Receive data cache module 412: the data that buffer memory DMA receiver module 411 is sent.
SPI4.2 sends control module 413: receive the existing new outgoing data notice that DMA receiver module 411 is sent, send the Flow Control inquiry to SPI4.2 Flow Control module 414, if receive the not Flow Control indication that SPI4.2 Flow Control module 414 is returned, obtain data in buffer from receiving data cache module 412, will send to CPU by the SPI4.2 passage behind the SPI4.2 control head on this data encapsulation; If receive the Flow Control indication that SPI4.2 Flow Control module 414 is returned, then send the notice that the SPI4.2 passage of this veneer has blocked to the source of this veneer veneer by BD administration module 418, DMA sending module 417, and continue to send the Flow Control inquiry to SPI4.2 Flow Control module 414, return not Flow Control indication until SPI4.2 Flow Control module 414.
SPI4.2 Flow Control module 414:, then begin Flow Control if receive the Flow Control notice that CPU sends; If receive the cancellation Flow Control notice that CPU sends, then stop Flow Control; Receive SPI4.2 and send the Flow Control inquiry that control module 413 is sent,, send control module 413 to SPI4.2 and return Flow Control indication or not Flow Control indication according to current Flow Control situation.What receive the source of carrying veneer sign that BD administration module 418 sends stops to send data notification, this is stopped to send data notification send to CPU; What receive the source of carrying veneer sign that BD administration module 418 sends begins to send data notification, this is begun to send data notification send to CPU.
SPI4.2 receives control module 415: receive the data message that CPU sends by the SPI4.2 passage of this FPGA, metadata cache in the payload segment of data message to sending data cache module 416, is sent existing new outgoing data notice to DMA sending module 417 simultaneously.
Send data cache module 416: buffer memory SPI4.2 receives the data that control module 415 is sent.
DMA sending module 417: receive SPI4.2 and receive the existing new outgoing data notice that control module 415 is sent, obtain the data that carry the target single board sign from sending data cache module 416, be PCIe control head on the data encapsulation, the data message that obtains is put into a BD, and send the receipts BD inquiry of carrying the target single board sign to BD administration module 418, receive the BD address if receive the free time that BD administration module 418 returns, this free time is received the BD address put into the data message of sending out BD, start the DMA process of transmitting; If receive the full indication that BD administration module 418 returns, then repeat to send the receipts BD inquiry of carrying the target single board sign, until the free time receipts BD address of receiving that BD administration module 418 returns to BD administration module 418.
A BD administration module 418:, and create and safeguard a receipts BD chain respectively for each potential source veneer of this veneer for this veneer is created and safeguarded that is sent out a BD chain.When the initialization of this veneer, will be carried at for the address of all the receipts BD in the receipts BD chain of each source veneer maintenance in the receipts BD chain Initial message and send to this source veneer; And when satisfying predefined receipts BD chain update condition, will be carried at for the full condition information of sky of the receipts BD chain of each source veneer maintenance and receive the BD chain and upgrade in the message and notify this source veneer by DMA sending module 417.If it is full for the receipts BD chain of certain source veneer maintenance to detect this veneer, then to SPI4.2 Flow Control module 414 send carry this source veneer sign stop to send data notification, and be the receipts BD chain of this source veneer maintenance when idle detecting this veneer, to SPI4.2 Flow Control module 414 send carry this source veneer sign begin to send data notification.Receive the receipts BD chain Initial message that other veneer is sent, preserving the source veneer sign and the source veneer that carry in this message is the corresponding relation of the receipts BD address information in the receipts BD chain of this veneer maintenance; Receive the receipts BD chain renewal message that DMA receiver module 411 is sent, preserve source veneer sign and this source veneer of carrying in this message and be the empty full condition information of the receipts BD chain of this veneer maintenance.
Receiving BD chain Initial message, receiving BD chain renewal message all is the BD administrative message.
Fig. 5 has provided the schematic diagram of sending out, receive the BD chain on the high speed veneer that the embodiment of the invention provides, as shown in Figure 5, A_Tx is the BD chain that this high speed veneer is set up, content in the buffer memory is an outgoing data, B_Rx is the receipts BD chain that target single board duplicates in this veneer, after confirming that target single board is B, the free time receipts BD address among the B_Rx that this high speed veneer can be safeguarded for B is filled among the transmission BD of A_Tx, starts the DAM transport process then.
The high speed veneer that Fig. 6 provides for the embodiment of the invention receives the process chart of message, and as shown in Figure 6, its concrete steps are as follows:
Step 600:BD administrative unit 418 is created for this veneer and is safeguarded that is sent out a BD chain, and creates and safeguard a receipts BD chain respectively for each potential source veneer of this veneer; When the initialization of this veneer, will send to this source veneer by receiving BD chain Initial message for the address of all the receipts BD in the receipts BD chain of each source veneer maintenance; Receive the receipts BD chain Initial message that other veneer is sent, write down this message source veneer sign that carries and the corresponding relation of receiving the receipts BD address information in the BD chain.
Step 601:DMA receiver module 411 receives interruption or the message that indication message transmissions that the source veneer sends by the PCIe bus of FPGA finishes, and takes out message from the receipts BD of this veneer for the receipts BD chain correspondence of this source veneer maintenance.
In this step, DMA receiver module 411 is after taking out message from receive BD, source veneer sign in the message can be sent to BD administrative unit 418, after BD administrative unit 418 is received this source veneer sign, upgrade this veneer and be the receipts BD chain of this source veneer maintenance, and this veneer sign and this veneer are received the BD chain upgrade in the message and send to this source veneer for the full condition information of sky of the receipts BD chain of this source veneer maintenance is carried at.
In addition, in the embodiment of the invention, this veneer also can be received the BD chain update cycle for each source veneer preestablishes one, when each receipts BD chain update cycle of source veneer arrived, the BD administrative unit 418 on this veneer expired condition information with this veneer for the sky of the receipts BD chain of this source veneer maintenance and sends to this source veneer by receiving BD chain renewal message.
Step 602:DMA receiver module 411 judges that this message upgrades message or data message for receiving the BD chain, upgrades message, execution in step 603 if receive the BD chain; If data message, execution in step 605.
Step 603:DMA receiver module 411 will be received BD chain renewal message and directly send to BD administration module 418.
Step 604:BD administration module 418 receives that the BD chain upgrades message, preserves source veneer sign and this source veneer of carrying in this message and is the empty full condition information of the receipts BD chain of this veneer maintenance, and this flow process finishes.
For example: in this step, setting this veneer is high-speed line plate a, and receipts BD chain renewal message is for veneer b sends, and then carrying veneer b sign and veneer b in the message is the empty condition information of expiring of receipts BD chain that high-speed line plate a safeguards.Here, receiving the empty full situation of BD chain specifically can be: complete empty, full up, receive in all idle BD addresses in the BD chain, the receipts BD chain a kind of in all busy BD addresses.
Step 605:DMA receiver module 411 is peeled off out with the payload segment in the data message, and the metadata cache in the load to receiving data cache module 412, is sent control module 413 to SPI4.2 simultaneously and sends existing new outgoing data notice.
Step 606:SPI4.2 sends control module 413 and receives the existing new outgoing data notice that DMA receiver module 411 is sent.
Step 607:SPI4.2 sends control module 413 and whether inquires about Flow Control to SPI4.2 Flow Control module 414, if, execution in step 608; Otherwise, execution in step 609.
Handle load if the processing of CPU load has surpassed tolerable, then CPU can send the Flow Control notice to SPI4.2 Flow Control module 414, and SPI4.2 Flow Control module 414 begins to carry out Flow Control after receiving this Flow Control notice; When CPU is lower than tolerable processing load at the processing load, can notify to SPI4.2 Flow Control module 414 transmission cancellation Flow Controls, SPI4.2 Flow Control module 414 stops Flow Control after receiving this cancellation Flow Control notice.
Step 608:SPI4.2 sends control module 413 and sends the notice that the SPI4.2 passage of this veneer has blocked by BD administration module 418, DMA sending module 417 to the source veneer, goes to step 607.
In this step, if only comprise a SPI4.2 passage between FPGA and the CPU, then SPI4.2 sends control module 41 3 and need send the notice that the SPI4.2 passage of this veneer has blocked to all potential source veneers; If comprise more than a SPI4.2 passage between FPGA and the CPU, and the flow of all potential source veneers is distributed in respectively on each bar SPI4.2 passage, at this moment, SPI4.2 sends the notice that SPI4.2 passage that 413 of control modules need send from this veneer to the source veneer of the SPI4.2 passage correspondence of having blocked has blocked.
Step 609:SPI4.2 sends control module 413 and obtains data in buffer from receiving data cache module 412, will send to CPU by the SPI4.2 passage behind the SPI4.2 control head on this data encapsulation.
The flow chart that the high speed veneer that Fig. 7 provides for the embodiment of the invention sends datagram, as shown in Figure 7, its concrete steps are as follows:
Step 701:SPI4.2 receives control module 415 and receives the data message that carries the target single board sign that CPU sends by the SPI4.2 passage of this FPGA.
Step 702:SPI4.2 receives control module 415 payload segment of data message is peeled off out, and the metadata cache in the load to sending data cache module 416, is sent existing new outgoing data notice to DMA sending module 417 simultaneously.
Step 703:DMA sending module 417 receives SPI4.2 and receives the existing new outgoing data notice that control module 415 is sent, and obtains data from sending data cache module 416.
Step 704:DMA sending module 417 adds the PCIe control head to data, the data message that obtains is put into sent out BD.
Step 705:DMA sending module 417 obtains the sky full condition information of target single board for the receipts BD chain of this veneer maintenance from BD administrative unit 418.
Step 706:DMA sending module 417 judges to receive whether there is the idle BD of receipts in the BD chain, if, execution in step 707; Otherwise, return execution in step 705.
Step 707:DMA sending module 417 is received the BD address with the free time and is put into data message, starts the DMA transport process, and the PCIe bus by this FPGA sends to target single board with data message.
From Fig. 3~4,6~7 illustrated embodiments as can be seen, the embodiment of the invention realizes the DMA management by replace CPU with FPGA, not only alleviated the workload of CPU, simultaneously also because CPU does not need to construct the PCIe message, save the expense in PCIe packet header, thereby remedied the loss of SPI4.2 bus bandwidth.
The above only is process of the present invention and method embodiment, in order to restriction the present invention, all any modifications of being made within the spirit and principles in the present invention, is not equal to replacement, improvement etc., all should be included within protection scope of the present invention.