DSP-based multi-protocol concurrent transmission method, system, equipment and storage medium
Technical Field
The invention belongs to the technical field of data transmission, and particularly relates to a multi-protocol concurrent transmission method, a multi-protocol concurrent transmission system, multi-protocol concurrent transmission equipment and a storage medium based on a DSP.
Background
The DSP is used to identify, transform, filter, etc. specific signals, and the DSP has a very high processing efficiency for such specific mathematical algorithms, but is inefficient for some logic decisions due to the lack of hardware logic required by a general-purpose operating system such as Linux, as compared to a general-purpose processor. Currently, as the computing task at the upper layer becomes more and more complex and larger in scale, a single DSP device may not be able to meet the performance requirements, and thus a distributed system is commonly composed of multiple heterogeneous processors and DSPs to meet the increasingly complex computing requirements. Therefore, to ensure the strong real-time performance of the distributed system, the inter-core communication of the DSP and the communication of the DSP with other devices become the main problems currently facing.
The communication mode of the current DSP is mainly realized by directly using various physical buses through bottom layer hardware, including PCIe, SRIO, serial ports, and the like.
There are many deficiencies to conventional communication methods.
1. Because the bottom hardware, design principle, transmission protocol and the like of various heterogeneous devices are different, different transmission modes need to be realized aiming at different hardware when the DSP is transmitted with other devices, once the hardware model is changed, specific transmission codes need to be changed, the reusability of the codes is poor, and the workload of developers is increased.
2. The traditional communication mode is realized mainly from the perspective of hardware development, and is very complex, so that developers of upper-layer application need to pay attention to not only the application, but also the hardware, the protocol and the like of the bottom layer, and the development difficulty is increased.
3. When data exchange is carried out between each upper layer application and other applications, the bottom layer calling interfaces are different, so that when the upper layer application is transplanted, corresponding changes need to be made according to the driving change of bottom layer hardware, and the portability of the upper layer application is poor.
4. Because the way of realizing data transmission from the hardware development angle depends on bottom hardware and the like, most developers use a development way facing the process, the coupling degree among all functions is high, and the expandability is poor.
Disclosure of Invention
The invention aims to provide a multi-protocol concurrent transmission method and a multi-protocol concurrent transmission system based on a DSP (digital signal processor) so as to solve the problems.
In order to achieve the purpose, the invention adopts the following technical scheme:
the multiprotocol concurrent transmission method based on the DSP comprises the following steps:
step 1, acquiring function and performance requirements on a DSP (digital signal processor), and acquiring four non-functional requirements of the DSP on data transmission with other equipment and reusability, usability, expandability, portability and the like;
step 2, according to the obtained requirements, performing modular splitting on the functional requirements, namely starting and stopping of the client, publishing and subscribing of the theme, receiving and sending of data between upper-layer applications and matching of the theme;
and 3, integrating the four modules from a code level: firstly, a client object is created, publishing and subscribing of a theme, receiving and sending of data between upper-layer applications and packaging of a theme matching module code are carried out, and only 5 interfaces of publishing, subscribing, data sending, data receiving, matching and the like are provided for the client object to use.
Further, independently realize the start-stop of customer end: establishing a client and a server, wherein the starting of the client comprises the initialization of self parameters and the establishment of TCP connection with the server; the stopping of the client comprises disconnecting the TCP connection with the server and releasing all held resources.
Further, independently realizing the publish-subscribe of the topic: the sending end sends the theme to the server end through a CtosProtocol protocol, and the server end analyzes the data, takes topic as a key and info as a value to store the key into a hash table; the receiving end sends the theme to the server end in a CtosProtocol structure form through a fixed application layer protocol, and the server end stores the topic and the info of the receiver in the hash table in the same way.
Further, data receiving and sending are independently realized: and three different data transmission channels are realized by using a mode of coexistence of three transmission modes, namely a TCP data transmission channel, an SRIO data transmission channel and a shared memory data transmission channel.
Further, the TCP data transmission channel stores all other client descriptors connected to the client in the memory, the client continuously traverses all descriptors, and executes corresponding read-write operation when a read signal or a write signal is triggered; the SRIO data transmission channel is connected through an SRIO address, before the channel is used, two structural bodies of a Heap structure and a ShmEntry structure are initialized, and data reading and writing are carried out by using MessageQ; the shared memory data transmission channel is connected through the core ID, the operation is similar to that of the SRIO, two structures of the Heap and the ShmEntry need to be initialized first, and then the MessageQ is used for reading and writing data.
Further, client matching is independently realized: after receiving the theme sent by the sender or the receiver each time, the server needs to inquire whether the same theme exists in the hash table, if the matching is successful, the server needs to firstly judge whether the sending end and the receiving end are the same device and the type of the device where the sending end and the receiving end are located, if the sending end and the receiving end are the same device, a shared memory is used for establishing a data channel, if the sending end and the receiving end are ARM and the DSP, the TCP is used for establishing the data channel, and if the sending end and the receiving end are the DSP and the FPGA, the SRIO is used for establishing the data channel; finally, the information of the sending end and the receiving end is packaged by using a StoCProtocol structural body and sent to the receiving end and the sending end, so that a data channel is established between the two clients by using the information of the other client; if the matching is not found, storing the theme into a hash table, and finishing the matching; the server will only trigger a matching operation when a publish or subscribe request is received.
Further, a layered structure is established according to the functional requirements and the non-functional requirements in the step 1, and the layered structure comprises a matching layer, a message queue layer and a transmission layer; after the client object is created, the matching layer provides publishing, subscribing, sending data, receiving data, and matching interfaces.
Further, the matching layer is used for making clear the relation between the applications of the upper layers, when the application a needs to send data to the application B, the application a needs to publish a theme to the server of the data exchange service, the application B subscribes the theme to the server of the data exchange service, and if the themes used by the applications a and B are the same, the server of the data exchange service returns information, so that a data transmission channel can be established between the application a and the application B for data transmission.
Further, the data receiving and sending functions are completed by a message queue layer; the message queue layer is used for receiving and transmitting data between the application A and the application B, the data is not directly transmitted to the application, but is transmitted and received through the message queue, when the application A wants to transmit the data to the application B, the data needs to be stored in the message queue corresponding to the application A, the data in the queue is transmitted to the message queue of the application B through a certain transmission mode by the message queue, and when the application B wants to acquire the data, only the fact that whether the data exists in the message queue corresponding to the application B needs to be checked.
Further, a multi-protocol concurrent transmission system based on DSP includes:
the demand acquisition module is used for acquiring the function and performance demands on the DSP and acquiring the function demands of the DSP for data transmission with other equipment and four non-functional demands of reusability, usability, expandability, portability and the like;
the demand splitting module is used for modularly splitting the functional demands according to the obtained demands, and the demand splitting module is respectively used for starting and stopping the client, publishing and subscribing the theme, receiving and sending data between upper-layer applications and matching the theme with four modules;
the module integration module is used for integrating four modules from a code level: firstly, a client object is created, publishing and subscribing of a theme, receiving and sending of data between upper-layer applications and packaging of a theme matching module code are carried out, and only 5 interfaces for publishing, subscribing, sending data, receiving data and matching are provided for the client object for use.
Further, an apparatus comprising a memory and a processor, the memory having a computer program, the processor implementing the method when executing the computer program.
Further, a storage medium stores a computer program which, when executed by a processor, implements the method.
Compared with the prior art, the invention has the following technical effects:
the transmission function of the DSP uses the extensible real-time kernel provided by the DSP, and the difference of bottom hardware is shielded. The invention is based on the idea of object-oriented modular design, wherein the communication transport layer is an integral module, and if new requirements are needed in subsequent work, new functions can be realized by adding new modules or replacing original blocks without modifying original codes.
The invention provides a uniform interface for the upper application, so that the upper component developer can complete the data exchange between devices or in the devices without concerning the specific implementation of the bottom layer.
Furthermore, by formulating a transmission protocol at an application layer, the problems of packet sticking, half packet and the like during communication by using the TCP are solved, and a message confirmation mechanism is designed at the application layer, so that the reliability of TCP transmission is improved.
Furthermore, a current limiting measure is adopted in the data transmission process, so that the client is prevented from being crashed due to resource exhaustion when the data processing capacity of the upper application is not matched.
Furthermore, fusing processing is adopted in the processes of subscription publishing and data transmission, so that the problem that data which cannot be analyzed cannot be processed in the processes of subscription publishing and data transmission is avoided, and the reliability of data exchange service is improved.
Drawings
FIG. 1 is a hierarchical structure diagram of a data exchange service in the present invention
FIG. 2 is a diagram of a data exchange service transport layer structure in the present invention
FIG. 3 is a data transmission protocol
Detailed Description
The invention is further described below with reference to the accompanying drawings:
referring to fig. 1 to fig. 3, the present invention provides a DSP-based multi-protocol concurrent transmission method with more convenient use and better expandability.
The invention is realized by the following scheme:
step 1, obtaining the functional requirement and the non-functional requirement of the invention through the function and performance requirements of common radar, electric frame and other software on a DSP.
And 2, designing an integral framework of the system according to the requirements obtained in the first step, and modularly splitting functional requirements required to be completed by the system by using an object-oriented design idea, wherein the functional requirements are four modules of starting and stopping of a client, publishing and subscribing of a theme, transceiving of data between upper-layer applications and matching of the theme.
And the start-stop module of the client is independently realized. The invention is divided into two parts of client and server integrally, the starting of the client includes the initialization of the self parameter and the establishment of TCP connection with the server; the stopping of the client comprises disconnecting the TCP connection with the server and releasing all held resources.
And independently realizing a publish-subscribe module of the theme. A data sender in the system sends the theme to a server through a CtosProtocol protocol, and the server analyzes the data, takes topic as a key and info as a value to store the key into a hash table.
The data receiver sends the theme to the server side in a CtosProtocol structure form through a fixed application layer protocol, and the server side stores the topic and the info of the receiver in the hash table in the same way.
And the data receiving and transmitting module is independently realized. The data receiving and sending between the clients are realized through a special data channel, and the invention realizes three different data transmission channels by using a mode of coexistence of three transmission modes. The TCP data transmission channel stores all other client descriptors connected with the client in a memory, the client can continuously traverse all the descriptors, and when a read signal or a write signal is triggered, corresponding read-write operation can be executed; the SRIO data transmission channel is connected through an SRIO address, before the channel is used, two structural bodies of a Heap structure and a ShmEntry structure need to be initialized, and MessageQ is used for reading and writing data; the shared memory data transmission channel is connected through the core ID, the operation is similar to that of the SRIO, two structures of the Heap and the ShmEntry need to be initialized first, and then the MessageQ is used for reading and writing data.
And independently realizing a client matching module. After receiving the theme sent by the sender or the receiver each time, the server needs to query whether the same theme exists in the hash table. If the matching is successful, whether the sending end and the receiving end are the same device and the type of the device where the sending end and the receiving end are located need to be judged, if the sending end and the receiving end are the same device, a shared memory is used for establishing a data channel, if the sending end and the receiving end are the ARM and the DSP, a TCP is used for establishing the data channel, and if the sending end and the receiving end are the DSP and the FPGA, an SRIO is used for establishing the data channel. Finally, the information of the sending end and the receiving end is packaged by using a StoCProtocol structural body and sent to the receiving end and the sending end, so that a data channel is established between the two clients by using the information of the other client;
if the matching is not found, the theme is stored in the hash table, and the matching is finished. The server will only trigger a matching operation when a publish or subscribe request is received.
And 3, integrating the four modules from a code level. Firstly, a client object is created, publishing and subscribing of a theme, receiving and sending of data between upper-layer applications and packaging of a theme matching module code are carried out, and only 5 interfaces of publishing, subscribing, data sending, data receiving, matching and the like are provided for the client object to use.
Fig. 1 is a layered structure diagram, which is designed according to the functional and non-functional requirements in the scheme design step 1 in combination with the layered idea of the software design process, and consists of a matching layer, a message queue layer and a transmission layer. After the client object is created, the client object will use the publishing, subscribing, sending, receiving, matching, etc. interfaces provided by the present invention to accomplish specific tasks, and these interfaces are provided by the matching layers in the hierarchical structure. But the matching layer only actually implements the functions of publishing, subscribing and matching, and the functions of data receiving and sending are completed by the message queue layer of the lower layer. In the invention, the part depending on the bottom hardware and the protocol is completely encapsulated into the transmission layer, and the message queue layer only needs to be responsible for data receiving and transmitting without paying attention to the specific bottom difference. Each layer is described in detail below.
1. A matching layer.
In a distributed system composed of heterogeneous devices, a plurality of upper-layer applications exist on each device, the overall computing task needs to be completed through data exchange among the applications, and a matching layer is responsible for clarifying the relation among the upper-layer applications. When the application A needs to send data to the application B, the application A needs to publish a theme to a server of the data exchange service, the application B subscribes the theme to the server of the data exchange service, and if the themes used by the application A and the application B are the same, the server of the data exchange service returns information, so that a data transmission channel can be established between the application A and the application B for data transmission.
2. And a message queue layer.
Data are transmitted and received between the application A and the application B, and the data are not directly handed to the application per se but transmitted and received through a message queue. When the application A wants to send data to the application B, the data needs to be stored in a message queue corresponding to the application A, the message queue sends the data in the queue to a message queue of the application B through a certain transmission mode, and when the application B wants to obtain the data, the application B only needs to check whether the data exists in the message queue corresponding to the application B.
3. A transport layer.
Currently, the transport layer provides three different transport modes, TCP/IP, shared memory, and SRIO. When the DSP equipment communicates with the ARM, the TCP/IP can be automatically selected to receive and transmit data, when the DSP equipment communicates with the FPGA, the SRIO can be automatically selected to receive and transmit data, and when the DSP equipment performs inter-core communication, the shared memory can be automatically selected to receive and transmit data.
A transport layer structure diagram of a data exchange service is given in fig. 2.
The communication between the two cores inside the device 1 is realized through SharedRegion and MessageQ provided by an IPC toolkit. Firstly, each core acquires the core ID of the core and how many cores in the same DSP equipment are in a working state. And setting three queue names, namely a core queue name, a previous core queue name and a next core queue name, and establishing a queue through the core queue name. And creating a SharedRegion entity again, and respectively specifying the attributes of an entity number, a real address, a total length, a core ID which has the control right of the entity, whether the entity can be accessed by other cores and the like. After these steps are completed, SharedRegion and MessageQ may be registered in heap space and the heap ID is obtained. The control core uses the heap ID application space, and after the application is successful, the data can be accessed in the heap, and each core can use the shared memory for communication.
If device 1 and device 2 communicate, SRIO may be used for data transmission. The SRIO is implemented in a similar manner to the shared memory, but the QMSS and CPPI coprocessors on the DSP need to be initialized on the basis of the shared memory.
If the TCP is used for communication, the two communication parties are logically divided into a client side and a server side. The initialization of the server side uses a socket interface to specify an IP protocol and a TCP protocol and acquire a file descriptor. And after the file descriptor and the address of the file descriptor are bound, setting the maximum TCP connection number which can be processed by the file descriptor at the same time. And finally, monitoring the logic client, and when the client initiates connection, completing TCP three-way handshake to perform data transmission. The initialization of the client is much simpler, the connection with the server is directly initiated after the socket interface is used for designating an IP protocol and a TCP protocol, and data transmission can be carried out after the connection is successful.
When the shared memory, the SRIO and the TCP are used for data transmission, reliability design is respectively carried out.
1) An application layer message acknowledgement mechanism. When the TCP is used for network transmission, the TCP itself is a reliable protocol, but because the reliability is established on the premise that the network channel is available, if the reliability is not satisfied, the TCP cannot ensure the reliability of network transmission, so a message acknowledgement mechanism is used in an application layer to ensure the transmission reliability. The specific method is that after receiving data, the receiving end sends a confirmation message to the sending end, and when receiving the confirmation message, the sending end sends the next piece of data. If the sending end does not receive the confirmation information of the receiving end within a period of time, the sending end repeatedly sends the previous piece of data. Similarly, the reliability of the shared memory and the SRIO itself is not guaranteed, so that an application layer message acknowledgement mechanism is also required to ensure the reliability of data transmission.
2) A fusing mechanism. When a sending end sends information, data is packed and sent according to a data stream transmission protocol, but in the transmission process, the data may be partially lost, so that the receiving end cannot analyze the data sent by the sending end, and a large amount of data which cannot be analyzed can cause the resource exhaustion of the receiving end and the running of programs. The fusing mechanism can ensure that when the data which cannot be analyzed and sent by one data channel exceeds a certain amount, the client automatically closes the channel and does not receive the data any more, so that stable operation is achieved.
3) A current limiting mechanism. When two applications transmit data, the problem of inconsistent data processing speed may exist. When the sending speed of the sending end is too fast, data can be accumulated at the receiving end. The current limiting mechanism only receives data with a fixed size within a specified fixed time range, so that the condition that the sending speed of a sending end is too high to cause program crash is prevented.