CN113810740A - Image transmission hardware system based on FPGA control circuit design - Google Patents

Abstract

The invention relates to the technical field of image transmission systems, in particular to an image transmission hardware system designed based on an FPGA control circuit, which comprises an FPGA development board, a CMOS image chip, a computer, a display, an image chip peripheral circuit and an FPGA peripheral circuit, wherein the input end of the FPGA development board is connected with the output end of the computer through a circuit, the output end of the FPGA development board is connected with the input end of the display through a circuit, the output end of the FPGA development board is connected with the input end of the CMOS image chip through a signal, the CMOS image chip is connected with the image chip peripheral circuit, the FPGA development board is connected with the FPGA peripheral circuit, the Spartan-7 FPGA chip is taken as a center, and the design of a logic circuit and the compiling of simulation files are carried out through the time sequence simulation of the hardware circuit by using a Verilog HDL hardware programming language. The data transmission and the image format conversion can be effectively realized, the image test can be carried out in the signal transmission process, and the display of high-quality images is realized.

Description

Image transmission hardware system based on FPGA control circuit design

Technical Field

The invention relates to the technical field of image transmission systems, in particular to an image transmission hardware system designed based on an FPGA control circuit.

Background

With the improvement of the integrated circuit technology level, a CMOS (Complementary Metal Oxide Semiconductor) image chip has been developed dramatically. An analog-to-digital converter and a level conversion circuit can be integrated in the CMOS image chip of today, so that the CMOS image chip directly outputs digital signals. With the increasing market demand for high-definition and high-frame-frequency image chips, the hardware circuit interface design is also greatly challenged. The conventional DVP (Digital Video Port) parallel Interface cannot meet the requirement of a high frame rate and high resolution image chip, and the novel CSI Serial Interface (Camera Serial Interface) gradually occupies the image chip Interface market by virtue of the advantages of high data transmission rate, strong interference resistance and the like. An imaging control circuit is designed aiming at an 1/4-inch CMOS image chip OV5647, and the circuit platform adopts a spartan 7 model FPGA chip (Field Programmable circuit) of Xilinx company and carries out circuit design by using a Verilog HDL hardware description language. The configuration of the internal register parameters of the image chip is realized, the CMOS image chip can be controlled to output image data through a CSI-2 protocol, and data analysis and image format conversion based on a CSI-2 protocol interface are realized at an FPGA receiving end. The design finally adopts a mini HDMI (High Definition Multimedia Interface ) to carry out image output test, and the test result shows that when the resolution is 1280 multiplied by 720 to carry out image acquisition, the image frame rate can reach 70fps, and the image display state is good.

Disclosure of Invention

The invention aims to provide an image transmission hardware system designed based on an FPGA control circuit to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme: the image transmission hardware system based on the FPGA control circuit design comprises an FPGA development board, a CMOS image chip, a computer, a display, an image chip peripheral circuit and an FPGA peripheral circuit, wherein the input end of the FPGA development board is connected with the output end of the computer through a line, the output end of the FPGA development board is connected with the input end of the display through a line, the output end of the FPGA development board is connected with the input end of the CMOS image chip through a signal, the CMOS image chip is connected with the image chip peripheral circuit, the FPGA development board is connected with the FPGA peripheral circuit, and a CMOS driving module, an MIPI CSI-2 data receiving module and a data conversion and display control module are arranged in the FPGA development board.

Preferably, the FPGA controls the CMOS image chip through an SCCB bus, the receiving of CMOS image data is realized through an MIPI CSI-2 bus, the FPGA configures a mini HDMI interface to be connected to a display for display testing, and the CMOS and the FPGA are respectively provided with a clock and stable driving voltage through respective peripheral circuits.

Preferably, a programmable logic device integrated development tool Vivado 2018.2 is adopted to design a logic circuit, a compiled bit file is downloaded to a Spartan-7 FPGA chip through the software, model sim 10.4 SE software is adopted to perform time sequence simulation of a hardware circuit, and a Verilog HDL hardware programming language is used to perform design of the logic circuit and writing of a simulation file.

Preferably, the output end line of the peripheral circuit of the image chip is connected with an OV5647 image chip, the output end of the CMOS driving module is in signal connection with the input end of the OV5647 image chip, the input end of the MIPI CSI-2 data receiving module is in signal connection with the output end of the OV5647 image chip, the output end of the MIPI CSI-2 data receiving module is in signal connection with the input end of the data conversion and display control module, and the output end line of the data conversion and display control module is connected with the input end of a display.

Preferably, the CMOS driving module controls DOVDD, AVDD, DVDD, and PWDN.

Preferably, the MIPI CSI-2 data receiving module includes a D-PHY layer receiving module, a channel merging management module, and a protocol layer data unpacking module.

Compared with the prior art, the invention has the beneficial effects that: the system flow and the used circuit thereof are used as a signal transmission system designed by a technical point, and are combined with a Spartan-7 FPGA chip as a center, and the design of a logic circuit and the writing of a simulation file are carried out by the time sequence simulation of a hardware circuit and by using a Verilog HDL hardware programming language. The data transmission and the image format conversion can be effectively realized, the image test can be carried out in the signal transmission process, and the display of high-quality images is realized.

Drawings

FIG. 1 is a hardware platform system framework diagram;

FIG. 2 is a schematic diagram of a development board structure;

FIG. 3 is a transition diagram of a single configuration state machine;

FIG. 4 is a block diagram of MIPI CSI-2 data reception;

FIG. 5 is a diagram of D-PHY layer data receive deserialization debug waveforms;

FIG. 6 is a diagram of D-PHY layer byte alignment emulation;

FIG. 7 is a diagram of the distribution pattern of CSI-2 data on channels;

FIG. 8 is a diagram of a channel merge module debug waveform;

FIG. 9 is a diagram of debugging waveforms for analyzing the start position of a long packet;

FIG. 10 is a diagram of a debug waveform for resolving the end position of a long packet;

FIG. 11 is a diagram of an acquired frame start packet;

FIG. 12 is a diagram of an acquired frame end packet;

FIG. 13 is a diagram of OV5647 pixel arrangement;

FIG. 14 is a diagram of four different 3 × 3 pixel matrix patterns;

FIG. 15 is a simulation diagram showing the timing of the synchronization signals.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art without creative efforts based on the technical solutions of the present invention belong to the protection scope of the present invention.

Referring to fig. 1 to 15, the present invention provides a technical solution: the image transmission hardware system based on the FPGA control circuit design comprises an FPGA development board, a CMOS image chip, a computer, a display, an image chip peripheral circuit and an FPGA peripheral circuit, wherein the input end of the FPGA development board is connected with the output end of the computer through a circuit, the output end of the FPGA development board is connected with the input end of the display through a circuit, the output end of the FPGA development board is connected with the input end of the CMOS image chip through a signal, the CMOS image chip is connected with the image chip peripheral circuit, the FPGA development board is connected with the FPGA peripheral circuit, and a CMOS driving module, an MIPI CSI-2 data receiving module and a data conversion and display control module are arranged in the FPGA development board.

The physical layer adopts a D-PHY protocol, and the interface of the physical layer adopts high-speed LSVS differential signals for image transmission. Although the general FPGA chip IO resources only support LVDS level standards (low-voltage differential signaling and low-voltage differential signaling), and the level standards are not matched, when the transmission rate is less than 800Mbit/s, the LSVS signals can be converted into LVDS signals by using a bridge resistor network to be received and processed by the FPGA terminal. Xilinx's Spartan-7 series FPGA has logic cells up to 102K, using the 28nm technology of the product of Taiwan corporation (TSMC). The transmission rate of the LVDS differential signals supported by the IO resources can reach 1.25 Gb/s. The internal Clock Management resource (CMT) integrates a Phase Locked Loop (PLL) and a Mixed Mode Clock Manager (MMCM) module, and can provide a high-quality Clock for developers. The Spartan-7 series FPGA has the advantages of low price, low power consumption, small volume and the like, and is very suitable for middle and low-end application. By adopting XC7S15 series chips, the logic resource reaches 12K, and the design requirement can be met.

The development Board used is an Edge acceleration Xilinx development Board (SEA Board). The development board integrates necessary FPGA peripheral circuits, MIPI CSI interfaces and mini HDMI interfaces.

The logic function of the platform is realized by utilizing an MIPI CSI interface and an FPGA chip, for an MIPI CSI interface pin, FPGA _ CAM _ DN1 and FPGA _ CAM _ DP1 are connected to a clock channel of a CSI-2 interface, FPGA _ CAM _ DN0 and FPGA _ CAM _ DP0 are connected to a data channel lane1 of the CSI interface, FPGA _ CAM _ DN1 and FPGA _ CAM _ DP1 are connected to a data channel lane 2 of the CSI interface, FPGA _ CAM _ SCL and FPGA _ CAM _ SDA are connected to an SCCB bus port of the CMOS image chip, and FPGA _ CAM _ RST is connected with a hardware reset end of the CMOS image chip. The differential signal port is connected with a 150 omega terminal impedance, the differential mode voltage of the receiving end is increased to 300mV due to the impedance, and the FPGA end can receive the high-speed differential signal of the D-PHY by using the LVDS level standard.

The CMOS image chip drive control module is used for realizing the power-on and configuration functions of the OV5647 image chip, enabling the OV5647 image chip to work normally, providing a correct power-on time sequence, ensuring the normal work of the image chip, realizing an SCCB bus control circuit and carrying out parameter configuration on the CMOS image chip through the interface.

The CMOS image chip register configuration, there are more than three hundred built-in parameter registers controlling the operation of the image chip in OV5647, wherein most parameter settings are related to gain control (AGC), exposure control (AEC), white balance control (AWB), gamma correction, background compensation, black level correction and the like of the image, and default values can be kept. According to the practical situation, only 80 register values are changed, most registers are kept at default values after being electrified, and the CMOS image chip is configured to be output by a 1280 multiplied by 720 MIPI serial interface, and the gain and the exposure time are automatically controlled. The relevant register configuration and its meaning are shown in the following table:

OV5647 register configuration information

Register address	Configuring parameters	Description of the invention
			0x3016	0x08	MIPI output interface enabling
0x3036	0xa8	Chip PLL frequency multiplication
			0x380a	0x05	Number of image pixel columns (high order)
0x380b	0x00	Number of image pixel columns (Low order)
			0x380c	0x02	Image pixel line number (high order)
0x380d	0xd0	Image pixel line number (high order)
			0x503d	0x80	Bar chart test open
0x3a08	0x01	Automatic gain adjustment

When the Verilog HDL programming language is used for realizing SCCB bus transmission, SIO _ D and SIO _ C are generated in a counter control mode. At 400khz clock, the time required for a single configuration is 165 clock cycles. The three-phase write transmission process for a single SCCB bus is programmed using a state machine circuit for a total of 6 states.

The module is used for receiving and analyzing MIPI CSI-2 interface data sent from the OV5647 image chip, namely the MIPI CSI-2 standard protocol receiving module is realized.

The D-PHY layer receiving module is used for converting the serial differential data stream received by the FPGA into 8-bit parallel single-ended data stream and carrying out byte alignment according to synchronous signals in the differential data stream; the channel merging management module is used for merging the 8-bit byte data streams of the two channels into a single-channel 16-bit data stream; the protocol layer data unpacking module is used for removing a protocol layer packet head and a protocol layer packet tail and outputting effective pixel information.

The D-PHY layer receives the control module, according to the time sequence change characteristic of the data stream, the data signal text under the low power consumption mode is identified by adopting the HSUL-12 level standard, and the high-speed differential signal is identified; the system adopts a selecto wiz IP core module to carry out serial-parallel conversion so as to obtain effective data, wherein an IP core is configured to be 2-channel data input and 16-bit parallel data output, and DDR double edges are used for carrying out data sampling. In the output timing of the pixel data on the LVDS channel, data _ lp _ n _ IBUF, data _ lp _ p _ IBUF are logic levels of a p pole and an n pole of the data channel No. 0, respectively, and data _ par _0 and data _ par _1 are data serial combination results of the data channels No. 0 and No. 1, respectively, particularly paying attention to the lowest order bit first output.

After the FPGA side acquires the data information, it needs to perform byte synchronization, that is, it detects whether the received data of each channel contains the synchronization sequence 00011101, and the D-PHY adopts low-order first transmission, so that the time when the sequence is transmitted to the receiving side becomes 10111000 (0 xB 8). Sequence detection is carried out by adopting two-byte FIFO, namely, one clock can always detect that a complete synchronous sequence is positioned in the FIFO, each channel is detected in such a way, and the synchronous sequences of all the channels are ensured to be positioned in the same clock period after detection is finished. The content of the data packet is the beginning of the next beat of the synchronization sequence of the four channels, and channel merging and unpacking operations can be carried out according to the CSI-2 protocol. The byte _ valid _ o is a data alignment completion flag signal, i.e., a channel valid signal, and the data _ par _0_ align are outputs after data alignment of two data channels, respectively. Both data lanes simultaneously generate the synchronization signal 0xB8 shortly after entering high speed mode and are correctly detected.

And the D-PHY layer channel merging control module is used for configuring an image chip to output data through 2 data channels, and according to the CSI-2 specification, byte data of the image chip are distributed to the two channels one by one in a zigzag form.

The data for lane0 and lane1 are stored in two FIFOs, respectively, by creating two FIFOs of 8 bits width and 3 bits depth to buffer the data for both lanes. And determining which channel receives valid data first (generally, the data delay of the two channels is small) according to the value of the channel valid signal of each channel, and taking the valid data from the FIFO buffer area according to the sequence form of the data valid flag signal and combining and outputting the valid data. For example, when it is detected that the channel valid signals of lane0 and lane1 are respectively 0 and 1, the module marks the depth [0] and the depth [1] of the two channel FIFOs at the next clock, and if the channel valid signal of the 01 sequence is detected again at the next clock, the module adds one to the FIFO depth mark signal of the lane1 channel until the channel valid signal sequence is detected to be 11, and assigns the data of the corresponding depth in the FIFO to the lane _ byte _ o output with the width of 16bit according to the sequence of lane0 and lane1 according to the depth mark signal. Lane _ byte _ o is the 16-bit output after merging the channel data, and Lane _ valid _ o is the valid signal for merging the data.

And the CSI-2 protocol data unpacking module is responsible for extracting the received image effective data, the line effective signals and the frame effective signals. The method comprises the steps of analyzing image effective Data and Data effective signals from long packets, analyzing frame starting signals and frame ending signals from short packets, extracting Data effective signals and line effective signals, analyzing contents in the long packets, wherein each long packet corresponds to a Data Id, the Data Id comprises a virtual channel number and contents of a Data Type, and the corresponding Data Type is 0x2B because only one device is arranged in the text, the virtual channel is 0, and the configured Data output format is RAW10 bit. Therefore, after receiving the valid signal after Data merging, the module can determine whether the packet is a valid image Data packet by identifying whether the Data Id of the first byte in the Data packet is 0x2B, and then output the valid image Data therein.

data _ o is always equal to lane _ byte _ o, output _ valid _ o is a valid image data output signal, and packet _ length is the total number of bytes of valid data of the packet acquired from the data packet header. The initial cursor position is 1173, the end cursor position is 1973, the difference value is 800, the double data channels are adopted to transmit data, and therefore the total effective byte number of pixels transmitted by one long packet is 1600 byte. The output resolution of the CMOS image chip is set to be 1280 × 720, the output data format is RAW10 bit, that is, the number of output row pixels is 1280, and each pixel occupies 10bit, that is, the number of theoretically output row pixels byte is 1280 × 10/8=1600 byte. The theory is consistent with the actual situation, and the correctness of the module is reasonably proved.

To extract the frame synchronization signal, it needs to be obtained from the short packet. According to the CSI-2 protocol, in the case of a single virtual channel, the DI of the start-of-frame short packet is 0x00, and the DI of the end-of-frame short packet is 0x01, so that the start-of-frame signal and the end-of-frame signal can be obtained by changing the determination conditions thereof by using the same idea and method as described above.

v _ sync is a frame effective signal, and the signal is set to be 1 when a frame start short packet is identified, which means that currently transmitted data is row data in the same frame; when the end-of-frame short packet is identified, the signal is set to be 0, namely the transmission of the current frame data is finished.

The design of the data conversion and display driving module aims at testing the driving circuit and outputs the received image data to a display screen through a mini HDMI interface according to a specified display time sequence. The module completes two functions, namely data conversion from RAW10 to RGB and display time sequence design.

And the data conversion module realizes the conversion of the RAW10 data into the RGB data because the OV5647 image chip can only output the pixel data in the RAW format. The pixel matrix form of OV5647 is arranged in a Bayer array fashion.

For each single pixel point, the RGB pixel value of the point is restored by using the primary color value of the pixel matrix with the lower right corner being 3 multiplied by 3, and the restoration mode adopts an averaging method. For the pixel points at different positions, the point-taking positions are different during color value calculation, and there are 4 pixel matrix patterns with different arrangement modes, which are Pattern 1, Pattern2, Pattern3 and Pattern4 respectively.

And (3) BRAMs with the storage capacity of 1280 multiplied by 8 bits are adopted to respectively cache the adjacent three lines of data, and one BRAM is selected for storage each time one line of data is received. Three FIFO pairs of 8bit width and 3bit depth are then used to fetch a 3 x 3 matrix of pixels near the pixel from the BRAM. When the pixel value is calculated, the pixel values of the same primary color in the FIFO are respectively taken out, and the average value is calculated, so that the RGB three-primary-color value of the point can be obtained.

The display driving module uses an IP core of HDMI provided by Xilinx official library, and a display time sequence generating circuit conforming to VESA standard (Video Electronics Standards Association) is designed for driving the IP core. For a digital image of 1280 × 720 resolution, at a 60fps frame rate, the pixel clock standard is 74.25 MHz.

Horizontal synchronization signal timing requirements

Phase name	Duration (number of pixel clocks)
		Pixel active period	1280
Displaying a leading edge	110
		Display back porch	220
Width of line synchronous signal	40

Field sync signal timing requirements

Phase name	Duration (line number)
		Pixel active period	720
Displaying a leading edge	5
		Display back porch	20
Width of field synchronous signal	5

The design adopts a counter mode to design the time sequence, the rising edge of a pixel clock is sampled according to the time sequence requirement of a line synchronizing signal, the line synchronizing signal is enabled to output a low level at the stage of 1390 to 1430 of the counter value, and the rest time period is a high level. According to the timing requirement of the field synchronizing signal, the rising edge of the sampling line synchronizing signal is enabled to output low level in the stage of the counter value being 725 to 730, and the rest period is high level.

The working principle is as follows: and (3) using a computer to realize the logic circuit of the generated bit file on the FPGA chip through a JTAG interface. The FPGA realizes the control of the CMOS image chip through an SCCB bus, realizes the receiving of CMOS image data through an MIPI CSI-2 bus, and configures a mini HDMI interface to be connected to a display for display test. The CMOS and the FPGA are respectively provided with a clock and stable driving voltage by respective peripheral circuits, a programmable logic device integrated development tool Vivado 2018.2 is adopted to carry out logic circuit design, and the compiled bit file is downloaded to a spark-7 FPGA chip through the software. Meanwhile, the design uses Modelsim 10.4 SE software developed by Mentor company to perform time sequence simulation of a hardware circuit, and uses Verilog HDL hardware programming language to perform design of a logic circuit and writing of simulation files.

It should be noted that the above-mentioned embodiments are only used for illustrating the technical solutions of the present invention and not for limiting the technical solutions, and although the applicant has described the invention in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions made on the technical solutions of the present invention can not be made within the spirit and scope of the technical solutions of the present invention and shall be covered by the claims of the present invention.

Claims (6)

1. Image transmission hardware system based on FPGA control circuit design, including FPGA development board, CMOS image chip, computer, display, image chip peripheral circuit and FPGA peripheral circuit, its characterized in that: the input end of the FPGA development board is connected with the output end of the computer through a line, the output end of the FPGA development board is connected with the input end of the display through a line, the output end of the FPGA development board is connected with the input end of the CMOS image chip through a signal, the CMOS image chip is connected with an image chip peripheral circuit, the FPGA development board is connected with the FPGA peripheral circuit, and a CMOS driving module, an MIPI CSI-2 data receiving module and a data conversion and display control module are arranged in the FPGA development board.

2. The image transmission hardware system based on FPGA control circuit design of claim 1, characterized in that: the FPGA realizes the control of the CMOS image chip through an SCCB bus, realizes the receiving of CMOS image data through an MIPI CSI-2 bus, configures a mini HDMI interface to be connected to a display for display test, and the CMOS and the FPGA are respectively provided with a clock and stable driving voltage through respective peripheral circuits.

3. The image transmission hardware system based on FPGA control circuit design of claim 2, characterized in that: a programmable logic device integrated development tool Vivado 2018.2 is adopted to design a logic circuit, a compiled bit file is downloaded to a Spartan-7 FPGA chip through software, Modelsim 10.4 SE software is adopted to perform time sequence simulation of a hardware circuit, and a Verilog HDL hardware programming language is used to design the logic circuit and compile a simulation file.

4. The image transmission hardware system based on FPGA control circuit design of claim 3, characterized in that: the output end of the peripheral circuit of the image chip is connected with an OV5647 image chip through a line, the output end of the CMOS driving module is connected with the input end of the OV5647 image chip through a signal, the input end of the MIPI CSI-2 data receiving module is connected with the output end of the OV5647 image chip through a signal, the output end of the MIPI CSI-2 data receiving module is connected with the input end of the data conversion and display control module through a signal, and the output end of the data conversion and display control module is connected with the input end of a display through a line.

5. The image transmission hardware system based on FPGA control circuit design of claim 4, characterized in that: the CMOS driving module controls DOVDD, AVDD, DVDD and PWDN.

6. The image transmission hardware system based on FPGA control circuit design of claim 5, characterized in that: the MIPI CSI-2 data receiving module comprises a D-PHY layer receiving module, a channel merging management module and a protocol layer data unpacking module.

Images