CN113472964B - Image processing device and system - Google Patents
Image processing device and system Download PDFInfo
- Publication number
- CN113472964B CN113472964B CN202110627980.8A CN202110627980A CN113472964B CN 113472964 B CN113472964 B CN 113472964B CN 202110627980 A CN202110627980 A CN 202110627980A CN 113472964 B CN113472964 B CN 113472964B
- Authority
- CN
- China
- Prior art keywords
- data
- image processing
- interface
- transmission module
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 240
- 230000005540 biological transmission Effects 0.000 claims abstract description 118
- 238000004891 communication Methods 0.000 claims abstract description 49
- 238000006243 chemical reaction Methods 0.000 claims description 29
- 239000000872 buffer Substances 0.000 claims description 24
- 230000004044 response Effects 0.000 claims description 16
- 238000000034 method Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 239000004606 Fillers/Extenders Substances 0.000 claims description 3
- 238000011144 upstream manufacturing Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00095—Systems or arrangements for the transmission of the picture signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/40—Bus networks
- H04L12/40006—Architecture of a communication node
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/40—Bus networks
- H04L12/4013—Management of data rate on the bus
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Facsimiles In General (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an image processing device and system, comprising: the transmission module is connected to the central processing unit of the external equipment through a bus, and is configured to transmit data packets storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packets to be mutually converted between a data packet form and a data stream form; the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module; and the control module is connected with the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module. The invention can improve the expansibility and flexibility related to image processing, reduce communication delay and meet the requirement of image processing application on high-speed data transmission and processing.
Description
Technical Field
The present invention relates to the field of graphics processing, and more particularly, to an image processing apparatus and system.
Background
In the field of deep learning based on convolutional neural networks, the computing power and transmission power of image processing systems are required to be continuously improved. The traditional CPU-GPU (central processing unit-pass processor) architecture has the problems of high energy consumption, low cost performance and the like. The FPGA is used as a field programmable gate array, can realize hardware dynamic reconstruction according to application requirements, and has highly parallel pipeline processing capability. The CPU-FPGA heterogeneous processing architecture needs a high-speed data transmission interface to realize efficient cooperative processing between the CPU and the FPGA.
The PCIe (computer and peripheral equipment interconnection standard extension) bus supports the expansion of a PCIe Switch (PCIe interface expander) based chip interface, supports the dynamic configuration of the interface bit width and the interface quantity, and has excellent expandability, flexibility, universality and low delay characteristics. High-speed data communication between the CPU and the FPGA can be realized through PCIe DMA technology. However, because PCIe interfaces are complex, PCIe interfaces for an image processing system are not widely popularized, and a unified PCIe-based image processing frame is not formed, so that different applications need to design FPGA hardware architecture from scratch, application development efficiency is low, expansibility and flexibility are poor, and communication delay is high.
Aiming at the problems of low development efficiency, poor expansibility and flexibility and high communication delay of an image processing frame in the prior art, no effective solution exists at present.
Disclosure of Invention
In view of the above, an object of the embodiments of the present invention is to provide an image processing apparatus and system, which can improve expansibility and flexibility related to image processing, reduce communication delay, and meet requirements of image processing applications for high-speed data transmission and processing.
In view of the above object, a first aspect of an embodiment of the present invention provides an image processing apparatus including:
the transmission module is connected to the central processing unit of the external equipment through a bus, and is configured to transmit data packets storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packets to be mutually converted between a data packet form and a data stream form;
the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module;
and the control module is connected with the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module.
In some embodiments, the transmission module is connected to the central processor through a PCIE bus, and the transmission module includes:
the integrated hard core is connected to the central processing unit through the PCIE bus and is configured to send or receive data packets to or from the central processing unit on the PCIE bus;
a communication framework unit connected to the integrated hard core and configured to parse data from the data packet or package the data into the data packet;
an interface conversion unit, connected to the communication frame unit and the image processing module, configured to receive data from the communication frame unit and convert it into an interface form compatible with the image processing module to transmit to the image processing module, and to receive data from the image processing module and convert it into an interface form compatible with the communication frame unit to transmit to the communication frame unit.
In some embodiments, the interface conversion unit is configured to convert data between a Riffa interface and an Axis interface, wherein the image processing module uses the Axis interface and the communication frame unit uses the Riffa interface.
In some embodiments, the interface conversion unit is configured to buffer data in the form of a Riffa interface to a first interface connected to a first buffer of the interface conversion unit in response to the communication frame unit beginning to transmit the data in the form of the Riffa interface, and to read out the data from a second interface of the first buffer to transmit the data to the image processing module in the form of an Axis interface in response to the data transmitted by the communication frame unit being completely buffered.
In some embodiments, the interface conversion unit is configured to buffer data in the form of an Axis interface to a first interface connected to a second buffer of the interface conversion unit in response to the image processing module beginning to transmit data in the form of an Axis interface, and to read out data from the second buffer of the second interface to transmit data in the form of a Riffa interface to the communication frame unit in response to the data transmitted by the image processing module being completely buffered.
In some embodiments, the image processing module includes:
an input data stream connected to the transmission module and configured to receive the data stream sent by the transmission module;
an input parameter list connected to the input data stream, configured to acquire and extract image data from the data stream, and to fill input parameters corresponding to the image data into the input parameter list for the image processing module to process the image data based on the image data;
an output parameter list configured to be filled with output parameters corresponding to the image processing data based on the image processing data obtained by processing the image data by the image processing module;
an output data stream connected to the output parameter list and the transmission module, configured to receive the output parameters of the output parameter list and output the data stream to the transmission module based on the output parameters;
a control signal coupled to the input data stream, the input parameter list, the output parameter list, and the output data stream, configured to control the input data stream, the input parameter list, the output parameter list, and the output data stream.
In some implementations, the input parameters include a source image height and/or a source image width.
In some embodiments, the output parameters include a number of rows of image processing data and/or a number of columns of image processing data.
In some embodiments, the image processing apparatus is implemented using a field programmable gate array.
A second aspect of an embodiment of the present invention provides an image processing system, including:
a central processing unit;
a main line interface expander connected to the central processing unit through an uplink bus;
a plurality of image processing apparatuses connected to the main line interface extender through a downstream bus, wherein each of the image processing apparatuses includes:
the transmission module is connected to the central processing unit of the external equipment through a bus, and is configured to transmit data packets storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packets to be mutually converted between a data packet form and a data stream form;
the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data to write back to the data stream, and feed back the data stream to the transmission module;
and the control module is connected with the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module.
The invention has the following beneficial technical effects: the image processing device and the system provided by the embodiment of the invention are connected to a central processing unit of an external device through a bus by using a transmission module, and are configured to transmit a data packet storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packet to be mutually converted between a data packet form and a data stream form; the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module; the control module is connected to the transmission module and the image processing module and is configured to provide a technical scheme of driving control signals for the transmission module and the image processing module, so that expansibility and flexibility related to image processing can be improved, communication delay is reduced, and the requirement of image processing application on high-speed data transmission and processing is met.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an image processing apparatus according to the present invention;
fig. 2 is a schematic structural diagram of a transmission module of the image processing apparatus provided by the present invention;
fig. 3 is a schematic structural diagram of an image processing module of the image processing apparatus provided by the present invention;
fig. 4 is a schematic structural diagram of an image processing system according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
It should be noted that, in the embodiments of the present invention, all the expressions "first" and "second" are used to distinguish two entities with the same name but different entities or different parameters, and it is noted that the "first" and "second" are only used for convenience of expression, and should not be construed as limiting the embodiments of the present invention, and the following embodiments are not described one by one.
In view of the above-mentioned objects, a first aspect of the embodiments of the present invention proposes an embodiment of an image processing apparatus that improves expandability and flexibility involved in image processing, reduces communication delay, and satisfies the need of an image processing application for high-speed data transmission and processing. Fig. 1 is a schematic diagram showing the structure of an image processing apparatus provided by the present invention.
The image processing apparatus package as shown in fig. 1 includes:
the transmission module is connected to the central processing unit of the external equipment through a bus, and is configured to transmit data packets storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packets to be mutually converted between a data packet form and a data stream form;
the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module;
and the control module is connected with the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module.
The devices and apparatuses disclosed in the embodiments of the present invention may be various electronic terminal apparatuses, for example, mobile phones, personal Digital Assistants (PDAs), tablet computers (PADs), smart televisions, and the like, or may be large-sized terminal apparatuses, for example, servers, etc., so the protection scope disclosed in the embodiments of the present invention should not be limited to a specific type of devices and apparatuses. The client disclosed by the embodiment of the invention can be applied to any one of the electronic terminal devices in the form of electronic hardware, computer software or a combination of the electronic hardware and the computer software.
In some embodiments, the transmission module is connected to the central processor through a PCIE bus. The transmission module includes: the integrated hard core is connected to the central processing unit through the PCIE bus and is configured to send or receive data packets to or from the central processing unit on the PCIE bus; a communication framework unit connected to the integrated hard core and configured to parse data from the data packet or package the data into the data packet; an interface conversion unit, connected to the communication frame unit and the image processing module, configured to receive data from the communication frame unit and convert it into an interface form compatible with the image processing module to transmit to the image processing module, and to receive data from the image processing module and convert it into an interface form compatible with the communication frame unit to transmit to the communication frame unit.
In some embodiments, the interface conversion unit is configured to convert data between a Riffa interface and an Axis interface, wherein the image processing module uses the Axis interface and the communication frame unit uses the Riffa interface.
In some embodiments, the interface conversion unit is configured to buffer data in the form of a Riffa interface to a first interface connected to a first buffer of the interface conversion unit in response to the communication frame unit beginning to transmit the data in the form of the Riffa interface, and to read out the data from a second interface of the first buffer to transmit the data to the image processing module in the form of an Axis interface in response to the data transmitted by the communication frame unit being completely buffered.
In some embodiments, the interface conversion unit is configured to buffer data in the form of an Axis interface to a first interface connected to a second buffer of the interface conversion unit in response to the image processing module beginning to transmit data in the form of an Axis interface, and to read out data from the second buffer of the second interface to transmit data in the form of a Riffa interface to the communication frame unit in response to the data transmitted by the image processing module being completely buffered.
In some embodiments, the image processing module includes:
an input data stream connected to the transmission module and configured to receive the data stream sent by the transmission module;
an input parameter list connected to the input data stream, configured to acquire and extract image data from the data stream, and to fill input parameters corresponding to the image data into the input parameter list for the image processing module to process the image data based on the image data;
an output parameter list configured to be filled with output parameters corresponding to the image processing data based on the image processing data obtained by processing the image data by the image processing module;
an output data stream connected to the output parameter list and the transmission module, configured to receive the output parameters of the output parameter list and output the data stream to the transmission module based on the output parameters;
a control signal coupled to the input data stream, the input parameter list, the output parameter list, and the output data stream, configured to control the input data stream, the input parameter list, the output parameter list, and the output data stream.
In some implementations, the input parameters include a source image height and/or a source image width.
In some embodiments, the output parameters include a number of rows of image processing data and/or a number of columns of image processing data.
In some embodiments, the image processing apparatus is implemented using a field programmable gate array.
Specific embodiments of the present invention are further described below with reference to specific examples.
Fig. 1 shows an overall design of an image processing apparatus, in which high-speed data communication is performed between a CPU and an FPGA board through a PCIe interface. The CPU loads the user application based on, for example, the Ubuntu operating system to install the driver and API environments of Riffa (a framework that allows FPGAs and CPUs to communicate using PCIE buses). The FPGA board card comprises a PCIe transmission module, an image processing module and a control module. The PCIe transmission module is used for rapidly realizing high-speed data communication between the FPGA and the host through the PCIe IP core and the Riffa module, receiving PCIe data packets sent by the CPU and converting the PCIe data packets into data streams in an Axis format to the image processing module; the image processing module designs a unified input and output interface, performs high-speed data interaction with the PCIe transmission module, receives source image data, realizes a corresponding image processing application program, and sends the data back to the CPU through the PCIe transmission module according to a set specified format to complete an image processing core function; the control module provides starting control signals required by the image processing module and the PCIe transmission module, and the stable operation of the system is ensured.
Specifically, the PCIe transmission module is detailed in fig. 2, which shows the overall hardware interface of the PCIe transmission module, including a PCIe IP core, a Riffa module, and a riffa_axis module. Wherein: the PCIe IP core is a hard core integrated by the FPGA and interacts with PCIe transaction packet (namely, data packet of transaction data) data sent by the CPU. The Riffa module is connected with the PCIe IP core to realize analysis and data conversion of the PCIe transaction packet, and data transmission and reception with the PCIe hard core are realized through two data transmission interfaces of Riffa_rx and Riffa_tx designed by a user. In order to realize the direct communication between the Riffa module and the image processing module, the conversion from the Riffa to the 2 interfaces of the Axis needs to be completed through the riffa_axis module. The Riffa_Axis module comprises 2 conversion channels, one conversion channel is from a Riffa_rx interface to an output Axis interface, data is transmitted in through the Riffa_rx interface and is buffered to a BRAM through a BRAM1_PORTA interface, and after the data is completely buffered, the data is read out through the BRAM1_PORTB interface and is sent out through the output Axis interface. The other channel is a conversion channel from an input Axis interface to a Riffa_tx interface, the data is transmitted in through the input Axis interface and is buffered to a BRAM through a BRAM0_PORTA interface, and after the data is buffered completely, the data is read out through the BRAM0_PORTB interface and is sent out through the Riffa_tx interface. Through the PCIe transmission module, the image transmission module can realize high-speed data transmission with the CPU module.
As shown in fig. 3, the image processing module includes five interfaces, i.e., an input data stream in an Axis format, an output parameter list, an input parameter list, and a control signal. The input data stream in the Axis format is directly connected with the PCIe transmission module and is Axis data sent by the PCIe transmission module. The output data stream in the Axis format is also directly connected with the PCIe transmission module, and is Axis data sent to the PCIe transmission module. The input parameter list is information such as source image height, source image width and the like, is generated by a Riffa_axis module of the PCIe transmission module and is sent to the image processing module. The output parameter list is the image processing information such as the output data line number, the data column number and the like, and is generated by the image processing module and sent to the Riffa_axis module. The control signal controls the image processing module to start and operate.
As can be seen from the above embodiments, the image processing apparatus provided in the embodiments of the present invention is configured to transmit, by using a transmission module, a data packet storing image data and image processing data with a central processing unit connected to an external device through a bus, and to mutually convert input data and output data in the data packet between a packet form and a data stream form; the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data to write back to the data stream, and feed back the data stream to the transmission module; the control module is connected to the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module so that the transmission module and the image processing module can normally operate, expansibility and flexibility related to image processing can be improved, communication delay is reduced, and the requirement of image processing application on high-speed data transmission and processing is met.
Based on the above object, a second aspect of the embodiments of the present invention proposes an embodiment of an image processing system that improves expandability and flexibility involved in image processing, reduces communication delay, and satisfies the need of an image processing application for high-speed data transmission and processing. Fig. 4 is a schematic diagram showing the structure of an image processing apparatus provided by the present invention.
As shown in fig. 4, the image processing system includes:
a central processing unit;
a main line interface expander connected to the central processing unit through an uplink bus;
a plurality of image processing apparatuses connected to the main line interface extender through a downstream bus, wherein each of the image processing apparatuses includes:
the transmission module is connected to the central processing unit of the external equipment through a bus, and is configured to transmit data packets storing image data and image processing data with the central processing unit on the bus, and enable input data and output data in the data packets to be mutually converted between a data packet form and a data stream form;
the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module;
and the control module is connected with the transmission module and the image processing module and is configured to provide driving control signals for the transmission module and the image processing module.
The image processing system is realized based on a PCIe switch chip and a netFPGA (Internet enabled FPGA) development board, and the following diagram is an overall hardware interconnection structure of the system, wherein a CPU main board is provided with a plurality of PCIe physical interfaces for connecting PCIe board card devices inserted into the CPU main board, and the PCIe board card devices comprise a GPU, IO resources and storage resources, and the PCIe switch chip. The PCIe switch chip is used for expanding the number of PCIe interfaces, an upstream bus is connected with the CPU, a downstream bus is connected with a plurality of terminal devices, and the terminal devices need to support PCIe standard interfaces. In the FPGA module, a NetFPGA development board is adopted in the system, and related image processing application programs and PCIe data transmission programs are realized by an FPGA board. The CPU main board adopts Ubuntu operating system, and is preloaded with FPGA board card and PCIe switch chip driver. When the system is powered on, the CPU loads PCIe driver and application program, detects terminal equipment connected to the PCIe bus, recognizes the FPGA board card, directly accesses the FPGA board card based on the memory address, and executes the image processing application program.
In some embodiments, the interface conversion unit is configured to convert data between a Riffa interface and an Axis interface, wherein the image processing module uses the Axis interface and the communication frame unit uses the Riffa interface.
In some embodiments, the interface conversion unit buffers the data in the form of the Riffa interface to a first interface connected to a first buffer of the interface conversion unit in response to the communication frame unit beginning to transmit the data in the form of the Riffa interface, and reads out the data from a second interface of the first buffer to transmit the data to the image processing module in the form of an Axis interface in response to the data transmitted by the communication frame unit being completely buffered.
In some embodiments, the interface conversion unit buffers the data in the form of an Axis interface to a first interface connected to a second buffer of the interface conversion unit in response to the image processing module beginning to transmit the data in the form of an Axis interface, and reads out the data from the second buffer of the second interface to transmit the data to the communication frame unit in the form of a Riffa interface in response to the data transmitted by the image processing module being completely buffered.
In some embodiments, the image processing module includes:
an input data stream connected to the transmission module and configured to receive the data stream sent by the transmission module;
an input parameter list connected to the input data stream, configured to acquire and extract image data from the data stream, and to fill input parameters corresponding to the image data into the input parameter list for the image processing module to process the image data based on the image data;
an output parameter list configured to be filled with output parameters corresponding to the image processing data based on the image processing data obtained by processing the image data by the image processing module;
an output data stream connected to the output parameter list and the transmission module, configured to receive the output parameters of the output parameter list and output the data stream to the transmission module based on the output parameters;
a control signal coupled to the input data stream, the input parameter list, the output parameter list, and the output data stream, configured to control the input data stream, the input parameter list, the output parameter list, and the output data stream.
In some implementations, the input parameters include a source image height and/or a source image width.
In some embodiments, the output parameters include a number of rows of image processing data and/or a number of columns of image processing data.
In some embodiments, the image processing apparatus is implemented using a field programmable gate array.
As can be seen from the above embodiments, the image processing system provided by the embodiments of the present invention is configured to transmit, with a central processing unit connected to an external device through a bus, a data packet storing image data and image processing data on the bus, and to mutually convert input data and output data in the data packet between a packet form and a data stream form, by using a transmission module; the image processing module is connected to the transmission module and is configured to acquire a data stream from the transmission module, perform data processing based on input data in the data stream to generate output data, write the output data back to the data stream, and feed back the written data stream to the transmission module; the control module is connected to the transmission module and the image processing module and is configured to provide a technical scheme of driving control signals for the transmission module and the image processing module, so that expansibility and flexibility related to image processing can be improved, communication delay is reduced, and the requirement of image processing application on high-speed data transmission and processing is met.
It should be noted that the above-described embodiments of the image processing system use the embodiments of the image processing apparatus to specifically describe the operation of each module, and those skilled in the art can easily understand that these modules are applied to other embodiments of the image processing apparatus. Of course, since the modules in the embodiment of the image processing apparatus may be intersected, replaced, added and subtracted, these reasonable permutation and combination are changed to the image processing system shall also belong to the protection scope of the present invention, and shall not limit the protection scope of the present invention to the embodiment.
Those of ordinary skill in the art will appreciate that: the above discussion of any embodiment is merely exemplary and is not intended to imply that the scope of the disclosure of embodiments of the invention, including the claims, is limited to such examples; combinations of features of the above embodiments or in different embodiments are also possible within the idea of an embodiment of the invention, and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. of the embodiments should be included in the protection scope of the embodiments of the present invention.
Claims (8)
1. An image processing apparatus, comprising:
a transmission module connected to a central processing unit of an external device through a bus, configured to transmit a data packet storing image data and image processing data with the central processing unit on the bus, and to mutually convert input data and output data in the data packet between a data packet form and a data stream form;
an image processing module, connected to the transmission module, configured to acquire the data stream from the transmission module, perform data processing based on input data in the data stream to generate output data to write back to the data stream, and feed back the data stream after writing back to the transmission module;
a control module connected to the transmission module and the image processing module, configured to provide driving control signals to the transmission module and the image processing module;
the transmission module is connected to the central processing unit through a PCIE bus, and the transmission module includes:
the integrated hard core is connected to the central processing unit through a PCIE bus and is configured to send or receive the data packet to or from the central processing unit on the PCIE bus;
a communication framework unit connected to the integrated hard core and configured to parse data from the data packet or package the data into the data packet;
an interface conversion unit connected to the communication frame unit and the image processing module, configured to receive data from the communication frame unit and convert to an interface form compatible with the image processing module to transmit to the image processing module, and to receive data from the image processing module and convert to an interface form compatible with the communication frame unit to transmit to the communication frame unit;
the image processing module includes:
an input data stream connected to the transmission module and configured to receive the data stream sent by the transmission module;
an input parameter list connected to the input data stream, configured to acquire and extract the image data from the data stream, and to fill input parameters corresponding to the image data into the input parameter list based on the image data for the image processing module to process the image data;
an output parameter list configured to be filled with output parameters corresponding to the image processing data based on the image processing data obtained by processing the image data by the image processing module;
an output data stream connected to the output parameter list and the transmission module, configured to receive output parameters of the output parameter list and output the data stream to the transmission module based on the output parameters;
a control signal coupled to the input data stream, the input parameter list, the output parameter list, and the output data stream, configured to control the input data stream, the input parameter list, the output parameter list, and the output data stream.
2. The apparatus of claim 1, wherein the interface conversion unit is configured to convert data between a Riffa interface and an Axis interface, wherein the image processing module uses an Axis interface, and wherein the communication framework unit uses a Riffa interface.
3. The apparatus according to claim 2, wherein the interface conversion unit is configured to buffer the data in the form of a Riffa interface to a first interface connected to a first buffer of the interface conversion unit in response to the communication frame unit starting to transmit the data in the form of the Riffa interface, and to read out the data from a second interface of the first buffer to transmit the data to the image processing module in the form of an Axis interface in response to the data transmitted by the communication frame unit being completely buffered.
4. An apparatus according to claim 2, wherein the interface conversion unit is configured to buffer data in the form of an Axis interface to a first interface connected to a second buffer of the interface conversion unit in response to the image processing module starting to transmit the data in the form of the Axis interface, and to read out data from the second buffer of the second interface to transmit the data to the communication frame unit in the form of a Riffa interface in response to the data transmitted by the image processing module being completely buffered.
5. The apparatus of claim 1, wherein the input parameters comprise a source image height and/or a source image width.
6. The apparatus according to claim 1, wherein the output parameters comprise a number of image processing data lines and/or a number of image processing data columns.
7. The apparatus of any of claims 1-6, wherein the image processing apparatus is implemented using a field programmable gate array.
8. An image processing system, comprising:
a central processing unit;
a main line interface expander connected to the central processing unit through an upstream bus;
a plurality of image processing apparatuses connected to the main line interface extender via a downstream bus, wherein each of the image processing apparatuses includes:
a transmission module connected to a central processing unit of an external device through a bus, configured to transmit a data packet storing image data and image processing data with the central processing unit on the bus, and to mutually convert input data and output data in the data packet between a data packet form and a data stream form;
an image processing module, connected to the transmission module, configured to acquire the data stream from the transmission module, perform data processing based on input data in the data stream to generate output data to write back to the data stream, and feed back the data stream after writing back to the transmission module;
a control module connected to the transmission module and the image processing module, configured to provide driving control signals to the transmission module and the image processing module to make the transmission module and the image processing module operate normally;
the transmission module is connected to the central processing unit through a PCIE bus, and the transmission module includes:
the integrated hard core is connected to the central processing unit through a PCIE bus and is configured to send or receive the data packet to or from the central processing unit on the PCIE bus;
a communication framework unit connected to the integrated hard core and configured to parse data from the data packet or package the data into the data packet;
an interface conversion unit connected to the communication frame unit and the image processing module, configured to receive data from the communication frame unit and convert to an interface form compatible with the image processing module to transmit to the image processing module, and to receive data from the image processing module and convert to an interface form compatible with the communication frame unit to transmit to the communication frame unit;
the image processing module includes:
an input data stream connected to the transmission module and configured to receive the data stream sent by the transmission module;
an input parameter list connected to the input data stream, configured to acquire and extract the image data from the data stream, and to fill input parameters corresponding to the image data into the input parameter list based on the image data for the image processing module to process the image data;
an output parameter list configured to be filled with output parameters corresponding to the image processing data based on the image processing data obtained by processing the image data by the image processing module;
an output data stream connected to the output parameter list and the transmission module, configured to receive output parameters of the output parameter list and output the data stream to the transmission module based on the output parameters;
a control signal coupled to the input data stream, the input parameter list, the output parameter list, and the output data stream, configured to control the input data stream, the input parameter list, the output parameter list, and the output data stream.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627980.8A CN113472964B (en) | 2021-06-05 | 2021-06-05 | Image processing device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627980.8A CN113472964B (en) | 2021-06-05 | 2021-06-05 | Image processing device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113472964A CN113472964A (en) | 2021-10-01 |
| CN113472964B true CN113472964B (en) | 2024-04-16 |
Family
ID=77872380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110627980.8A Active CN113472964B (en) | 2021-06-05 | 2021-06-05 | Image processing device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113472964B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559357A (en) * | 2013-11-12 | 2014-02-05 | 无锡市华磊易晶微电子有限公司 | FPGA (Field Programmable Gate Array) chip for 3D (Three-Dimensional) graphics rendering acceleration |
| JP2015170221A (en) * | 2014-03-07 | 2015-09-28 | 三菱電機株式会社 | Simulation device and interface module creation device and program |
| CN106845219A (en) * | 2017-01-13 | 2017-06-13 | 北京科技大学 | A kind of intrusion detection smart machine for multiple types of data |
| CN107071324A (en) * | 2017-01-25 | 2017-08-18 | 上海电气集团股份有限公司 | A kind of visual pattern processing system and its design method |
| CN107203484A (en) * | 2017-06-27 | 2017-09-26 | 北京计算机技术及应用研究所 | A kind of PCIe based on FPGA and SRIO bus bridge systems |
| CN207458128U (en) * | 2017-09-07 | 2018-06-05 | 哈尔滨理工大学 | A kind of convolutional neural networks accelerator based on FPGA in vision application |
| WO2018120780A1 (en) * | 2016-12-27 | 2018-07-05 | 深圳开立生物医疗科技股份有限公司 | Method and system for pcie interrupt |
| CN108829530A (en) * | 2018-06-15 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of image processing method and device |
| CN109741234A (en) * | 2018-12-29 | 2019-05-10 | 深圳市太赫兹科技创新研究院 | A kind of OCT image processing unit and system |
| CN110209358A (en) * | 2019-06-05 | 2019-09-06 | 哈尔滨工业大学 | A kind of NVMe equipment storage speed method for improving based on FPGA |
| CN110824218A (en) * | 2019-11-18 | 2020-02-21 | 重庆邮电大学 | Digital storage oscilloscope system based on ZYNQ |
| CN111339003A (en) * | 2020-01-08 | 2020-06-26 | 中国船舶重工集团公司第七二四研究所 | General multichannel data sending system and method based on FPGA |
| CN112181890A (en) * | 2020-09-30 | 2021-01-05 | 北京锐马视讯科技有限公司 | PCIE _ DMA data transmission device, method and system |
| CN112600858A (en) * | 2021-01-06 | 2021-04-02 | 智道网联科技(北京)有限公司 | Automatic driving data processing method, vehicle-mounted equipment and vehicle |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6722278B2 (en) * | 2016-04-11 | 2020-07-15 | オリンパス株式会社 | Image processing device |
| US10404364B2 (en) * | 2017-05-01 | 2019-09-03 | Teradyne, Inc. | Switch matrix system |
| CN107817216B (en) * | 2017-10-31 | 2020-06-02 | 武汉精测电子集团股份有限公司 | Automatic optical detection system based on CPU + GPU + FPGA architecture |
-
2021
- 2021-06-05 CN CN202110627980.8A patent/CN113472964B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559357A (en) * | 2013-11-12 | 2014-02-05 | 无锡市华磊易晶微电子有限公司 | FPGA (Field Programmable Gate Array) chip for 3D (Three-Dimensional) graphics rendering acceleration |
| JP2015170221A (en) * | 2014-03-07 | 2015-09-28 | 三菱電機株式会社 | Simulation device and interface module creation device and program |
| WO2018120780A1 (en) * | 2016-12-27 | 2018-07-05 | 深圳开立生物医疗科技股份有限公司 | Method and system for pcie interrupt |
| CN106845219A (en) * | 2017-01-13 | 2017-06-13 | 北京科技大学 | A kind of intrusion detection smart machine for multiple types of data |
| CN107071324A (en) * | 2017-01-25 | 2017-08-18 | 上海电气集团股份有限公司 | A kind of visual pattern processing system and its design method |
| CN107203484A (en) * | 2017-06-27 | 2017-09-26 | 北京计算机技术及应用研究所 | A kind of PCIe based on FPGA and SRIO bus bridge systems |
| CN207458128U (en) * | 2017-09-07 | 2018-06-05 | 哈尔滨理工大学 | A kind of convolutional neural networks accelerator based on FPGA in vision application |
| CN108829530A (en) * | 2018-06-15 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of image processing method and device |
| CN109741234A (en) * | 2018-12-29 | 2019-05-10 | 深圳市太赫兹科技创新研究院 | A kind of OCT image processing unit and system |
| CN110209358A (en) * | 2019-06-05 | 2019-09-06 | 哈尔滨工业大学 | A kind of NVMe equipment storage speed method for improving based on FPGA |
| CN110824218A (en) * | 2019-11-18 | 2020-02-21 | 重庆邮电大学 | Digital storage oscilloscope system based on ZYNQ |
| CN111339003A (en) * | 2020-01-08 | 2020-06-26 | 中国船舶重工集团公司第七二四研究所 | General multichannel data sending system and method based on FPGA |
| CN112181890A (en) * | 2020-09-30 | 2021-01-05 | 北京锐马视讯科技有限公司 | PCIE _ DMA data transmission device, method and system |
| CN112600858A (en) * | 2021-01-06 | 2021-04-02 | 智道网联科技(北京)有限公司 | Automatic driving data processing method, vehicle-mounted equipment and vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113472964A (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9424214B2 (en) | Network interface controller with direct connection to host memory | |
| US9654342B2 (en) | Bandwidth configurable IO connector | |
| CN103902486A (en) | System, device and method for implementation of remote direct memory access | |
| CN110837486A (en) | FlexRay-CPCIe communication module based on FPGA | |
| CN116069711B (en) | Direct memory access controller, heterogeneous device, memory access method and medium | |
| CN110941577A (en) | SATA interface link layer function realization method based on FPGA | |
| CN115733549B (en) | PCIE network card, switching method of interface modes of PCIE network card, electronic equipment and storage medium | |
| CN104714907A (en) | Design method for converting PCI bus into ISA bus or APB bus | |
| US8510759B1 (en) | Scatter gather emulation | |
| US8539131B2 (en) | Root hub virtual transaction translator | |
| CN113472964B (en) | Image processing device and system | |
| CN116450554A (en) | Interrupt processing method, root complex device and electronic device | |
| US20190286606A1 (en) | Network-on-chip and computer system including the same | |
| CN115114202B (en) | Embedded WEB Server Based on Optical Fiber Communication Interface Expansion | |
| CN102708091A (en) | Double-CPU (central processing unit) system communication method based on virtual network card | |
| CN103034613A (en) | Data communication method between processors and FPGA (field programmable gate array) equipment | |
| CN114840458B (en) | Read-write module, system on chip and electronic equipment | |
| CN109634880B (en) | Data acquisition equipment, data interaction equipment and data acquisition system | |
| CN212649484U (en) | 250W high-power-consumption ten-gigabit network card | |
| US10853287B2 (en) | Information processing system, semiconductor integrated circuit, and information processing method | |
| CN218996035U (en) | RDMA high-speed data transmission system matched with XMDA | |
| CN103164370B (en) | A kind of high-speed local bus access control interface module | |
| CN217767439U (en) | Processor system | |
| CN115543908B (en) | Aurora bus data interaction system based on FPGA | |
| CN217640194U (en) | PCIE switching circuit based on sea light server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |