CN110933382A - Vehicle-mounted video image picture-in-picture display method based on FPGA - Google Patents

Abstract

A vehicle-mounted video image picture-in-picture display method based on FPGA relates to FPGA technology. The invention provides a picture-in-picture display method for two paths of video images, which has the advantages of strong anti-interference capability of transmitted data, high resolution of the transmitted video images, stable and smooth video images and the like. In the method, two paths of video data are input into the FPGA through pins and are respectively received by the LVDS receiving module and the acquisition module. The data input by the LVDS receiving module is mapped into two groups of 24-bit RGB888 data after passing through the RGB mapping module; the data input by the acquisition module is firstly stored in the DDR3 through the data interaction module, and when needed, the data is taken out from the DDR3 and sent to the data gating module to be combined with the mapped RGB888 signals to form a frame of image data. The data gating module is used for selecting and outputting two paths of video data, the selected data are converted into 8 paths of 7-bit parallel data through the RGB reflection module, the 8 paths of 7-bit parallel data are output to the outside of the FPGA after being subjected to parallel-serial conversion through the LVDS sending module, and finally a picture formed by recombining the two paths of video data is displayed in one frame of image. The invention is suitable for the fields of high-definition multi-channel video image processing such as automobile-mounted video transmission, multi-channel video monitoring and the like.

Description

Vehicle-mounted video image picture-in-picture display method based on FPGA

Technical Field

The invention relates to an FPGA (field programmable gate array) technology, in particular to a vehicle-mounted video image picture-in-picture display method based on an FPGA.

Background

Picture-in-picture is a method of displaying two video signals simultaneously on one screen by using a mode of overlapping two video pictures, one large video picture and one small video picture, so as to view other pictures while viewing a main picture, namely, the sub-picture is placed in the main picture for display. With the development of automotive electronics technology in recent years, research and development on video sources, display devices, and video transmission lines have been greatly advanced. For example, the separation of the navigation display from the electronic system, which allows the display to be mounted in a position convenient for the driver to view, requires the addition of a video transmission line. The new generation of cars may also be equipped with various cameras for driving assistance, such as rear-view mirror cameras, night vision mirrors, and road sign recognition cameras, each of which needs to be connected to a display device through a video transmission line. The rapidly increasing transmission lines inside the vehicle body, especially the longer and longer transmission lines, make the transmission of the analog CVBS signal of the conventional transmission system very difficult. These signal formats cannot withstand the electromagnetic interference of the automobile. One solution to reduce video interference is to replace the analog signal with a digital signal, the video signal lines themselves being unable to generate interference. Low Voltage Differential Signaling (LVDS) has proven to provide the most reasonable connection for digital video transmission.

Disclosure of Invention

A vehicle-mounted video image picture-in-picture display method based on FPGA is provided. The invention realizes the functions of processing two paths of input video digital signals through the FPGA, finally realizing one path output of two paths of video images, and combining and displaying the contents of the two paths of video images on one frame of image to realize the picture-in-picture video image display effect. The A-path video image is an LVDS signal which is sent by the vehicle-mounted entertainment host and input by the serializer chip, and comprises a pair of clock lines and 8 pairs of data lines in total, and the bit width of the serial data is 7 bits; the B-path data is RGB888 image data, a line synchronizing signal HS, a field synchronizing signal VS and an effective data strobe DE signal which are acquired by the camera and input after a series of processing. The serial LVDS signals input by the A path are parallelized through an IP core inside the FPGA, RGB three channels are mapped, then the parallel LVDS signals are combined with RGB888 image data input by the B path to be selectively output, the output data are reflected back to the parallel signals, and finally the signals are converted into serial LVDS signals through the IP core inside the FPGA to be output to an external module. The FPGA is a main control chip, and the type of the FPGA is LFE5U-45F8BG381CA7131R45 chips under ECP5 series of LATTICE company; the serializer chip is selected to be DS90UB 947-Q1; the DDR3 memory is selected to be SCB13H2G160 AF-13K; the FPGA development tool is carried out in a Diamond3.10 integrated development environment.

The invention has the beneficial effects that: a vehicle-mounted video image picture-in-picture display method based on FPGA can realize the combined display of video data sent by a vehicle-mounted entertainment host computer end and video data sent by a vehicle-mounted camera end through FPGA, and finally realizes the picture-in-picture display effect on a vehicle-mounted screen. The main picture is a video picture sent by a vehicle-mounted host computer end, the sub-picture is a video picture transmitted by a vehicle-mounted camera end, and the sub-picture data is transmitted into the FPGA and then needs to be temporarily stored into the DDR3 for waiting to be output in combination with the main picture data. Through the internal setting of the FPGA, the display area and the size of a sub-picture, namely a video input by the vehicle-mounted camera, can be freely adjusted. The method has the advantages of strong anti-interference capability of transmitted data, high resolution of transmitted video images, stable and smooth video images and the like, the video refreshing rate can reach 60Hz, the resolution can reach 1920 x 1020, and the method is suitable for the fields of high-definition multi-channel video image processing such as automobile-mounted video transmission, multi-channel video monitoring and the like.

Drawings

Fig. 1 is a schematic block diagram of the structure of the present invention.

Fig. 2 is a schematic diagram of an LVDS receiving end according to the present invention.

Fig. 3 is a schematic diagram of an LVDS transmitting terminal according to the present invention.

Fig. 4 and 5 are schematic diagrams of input and output ends of RGB888 according to the invention.

FIG. 6 is a schematic diagram of a DDR3 memory of the invention.

Detailed Description

The invention is further described with reference to the following figures and detailed description.

As shown in fig. 1, the internal implementation process of the FPGA of the present invention is mainly composed of eight modules. The input end of the FPGA is two paths of video signals, the output end of the FPGA is a video signal transmitted by one path of LVDS interface, and data interaction with DDR3 is needed in the middle of data transmission. The system comprises an LVDS receiving module A2 connected with the video data of the A1 channel, namely LVDS interface data sent by a vehicle-mounted entertainment host end, an acquisition module B2 connected with a data end B1 input through a vehicle-mounted camera, a DDR3 interface module B4 for data interaction with the DDR3, and finally data of a sending end is output to the outside of an FPGA through an LVDS sending module AB 6. The other modules comprise a data interaction module B3, an RGB mapping module A3, a data gating module AB4 and an RGB mirror mapping module AB 5. The two paths of video data are subjected to data gating and merging in one path of video image data through the data gating module, and finally the data are output through the LVDS sending module (AB 7).

In an alternative embodiment, as shown in fig. 2, the LVDS signals are input to the FPGA through an external pin connected to the FPGA, and are sent to the LVDS receiving module. The LVDS receiving module is used for receiving serial differential signals sent by a host and outputting the serial differential signals in a parallelized mode, wherein an input end comprises a pair of differential clocks and a serial LVDS data pair of 8 channels, the frequency of the input clock is 75MHz, and each channel transmits 7-bit serial data in one clock. The 8-channel serial data is divided into two groups, each group of data comprises RGB888 three-channel data, line-field synchronizing signals HS and VS and effective data strobe DE signals, and two pixel points are transmitted in one clock period. The output is a pair of differential clocks and 8-way 7-bit parallel data. The capture circuit uses a double data rate register to capture data on both the rising and falling edges of the clock. When the receiving module is operating, the low speed clock provided with the data needs to be multiplied by 3.5 times, and the clock needs to be connected with the transmitting end. Two groups of 8-path 7-bit parallel data converted by the LVDS receiving module need to be mapped to an RGB three-channel through an RGB mapping module. Each group of 4-path data comprises 24-bit RGB signals, line field synchronization and effective data strobe signals, image RGB signals in a group of 4-channel 7-bit parallel data can be extracted through a mapping module to synthesize 24-bit R, G, B three-channel image data, and line field synchronization HS, VS and strobe enable signals DE are extracted. And finally, the image data is sent to the RGB three channels of the data gating module and the acquisition module to be recombined into a new frame of picture.

In an alternative embodiment, as shown in fig. 3, the internal data of the FPGA is output by an external pin connected to the FPGA, and the data is output to the outside through the LVDS sending module. The LVDS sending module is used for converting parallel data in the FPGA into serial differential signals and outputting the serial differential signals to the outside, the output clock frequency is consistent with the input differential clock, and the output data is serial 8-path 7-bit data.

In an alternative embodiment, as shown in fig. 4 and 5, the data input by the vehicle-mounted camera is sent to the FPGA after a series of processing and received by the acquisition module. The acquisition module acquires input RGB888 image data and transmits the data to the next module. The input frame of RGB888 image has resolution 1520 × 720, data input bit width 24 bits, and line and field sync (HS/VS) valid data strobe (DE) indication signal. The rising edge of the VS signal is detected by setting a register to beat the field sync signal VS. When the rising edge of VS is detected and DE is active, indicating that a field of images begins to be transmitted and simultaneously pulling up the active image output signal, the captured images are effectively output. Setting a line number counter and a field number counter, when a field image starts to be transmitted, the line number counter starts to count, and is cleared when the line number counter counts to the 1520 th point, and the field number counter is increased by one; when the field number counter counts to the 720 th point, the field number counter is cleared and counted again, and the image acquisition of one frame is finished. And outputting the data to the next module in a packet form, setting a packet header as the first data in the first field pixels and a packet end as the 1519 th data in the 720 th field pixels, wherein when the next module detects the packet header, the next module indicates that the frame of image starts to be sent, and when the packet end is detected, the next module indicates that the transmission of the frame of image is finished.

In an alternative embodiment, as shown in fig. 6, the data input by the acquisition module needs to be temporarily stored in the DDR3 to wait for the data strobe module to select, where the data needs to pass through the data interaction module and the DDR3 interface module.

The data interaction module is used for performing interaction between data storage and data reading. RGB888 data input by the acquisition module needs to be temporarily stored in FIFO in the module for cross-clock domain reading and writing, the clock at the write end is the clock of the acquisition module, and the clock at the read end is the input clock of DDR 3. Since the data bit width of DDR3 is 64 bits, two pixels are spliced together to form 48 bits of data for storage. Its bit width is set to 64 bits, its upper 16 bits complement 0, and its depth is set to 128. When the DDR3 interface module is ready to receive data and there is data in the FIFO, the pixel data stored in the FIFO is sent to the DDR3 interface module. The data output from the DDR3 and sent to the data strobe module also needs a FIFO for reading and writing data across the clock domain, the write clock is the input clock of the DDR3, and the read clock is the clock of the data strobe module. The bit width is set to 64 bits and the depth is set to 2048, pixel data is stored in the FIFO when being read out from the DDR3, and the pixel data is read out from the FIFO when a camera acquisition data input request sent by the data strobe module is received.

The DDR3 interface module is used for interaction of DDR3 memory and external data. In this module, the initialization and register mode configuration of DDR3 is completed. In the initialization process, after the DDR3 is powered on, it needs to wait at least 500us before sending the initialization start signal, and when the DDR3 returns the initialization completion signal, the DDR3 is initialized successfully. In the mode register configuration, the read/write burst BL of the DDR3 is set to 8, and the burst command length is 32, that is, when there is one read (write) command, 32 read (write) operations are performed consecutively, and 8 16-bit data are written in one read (write) operation, so 64-bit data can be written consecutively. In the module, 3 FIFOs are arranged to read and write data across clock domains, wherein the FIFOs 1 and 2 are used for buffering write address data and read address data, clocks at a write end and a read end of the FIFOs are input clocks of DDR3, the bit width of the data is set to be 26, and the depth of the data is set to be 128. The FIFO3 is used to buffer the pixel data input by the previous module, and the write and output terminals of the FIFO are both clocked by the input clock of the DDR 3. The module sends a write command, a write address and write data to a DDR3 IP core, pixel data are stored in a DDR3, and when the data interaction module needs to read out the data, the data stored in the DDR3 are read out according to the address sequence through a read address and a read request signal sent by the previous module.

And the two paths of video data are selectively output through the data gating module, and finally, a picture formed by recombining the two paths of video data is displayed in one frame of image. The video data sources input by the module are respectively an A-path vehicle-mounted host and a B-path vehicle-mounted camera. The video image data in the A path is converted into RGB888 three-channel image data through the mapping module, and is selectively output together with the RGB888 image data input by the B path acquisition module. The line and field synchronizing signals and data strobe signals of the module are based on the line and field synchronizing and strobe signals mapped from the A-channel input. When the rising edge of a field synchronizing signal VS in the A path is detected, counting the input pixels of one line, and after the input of one line is finished, clearing the line counter and adding one to the field counter. Specifically, the image data is gated by setting a local gate signal, the input pixels of the a-channel data are 1920 × 1020, the input pixels of the B-channel data are 1520 × 720, and since two pixels are input at one time, the zero clearing condition of the line count is to count to 1920/2 th 960, and the zero clearing condition of the field count is to count to 1020 th. The area strobe signal indicates the B-way data when the line counts 110 to 960 and the field counts 80 to 840, and the pixel data output in the range of the area strobe signal is the camera capture partial data read out from the DDR 3.

The working principle and the process of the invention are as follows: the two paths of video data are input into the FPGA through pins and are respectively received by the LVDS receiving module and the acquisition module. The data input by the LVDS receiving module is mapped into two groups of 24-bit RGB888 data after passing through the RGB mapping module; the data input by the acquisition module is firstly stored in the DDR3 through the data interaction module, and when needed, the data is taken out from the DDR3 and sent to the data gating module to be combined with the mapped RGB888 signals to form a frame of image data. The data gating module is used for selecting and outputting two paths of video data, the selected data are converted into 8 paths of 7-bit parallel data through the RGB reflection module, the 8 paths of 7-bit parallel data are output to the outside of the FPGA after being subjected to parallel-serial conversion through the LVDS sending module, and finally a picture formed by recombining the two paths of video data is displayed in one frame of image.

Although the embodiments of the present invention have been described above, the contents thereof are merely embodiments adopted to facilitate understanding of the technical aspects of the present invention, and are not intended to limit the present invention. It will be apparent to persons skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (4)

1. A vehicle-mounted video image picture-in-picture display method based on FPGA is characterized by comprising an LVDS receiving module, an RGB mapping module, an acquisition module, a data interaction module, a data gating module, a DDR3 interface module, an RGB reflection module and an LVDS sending module. The signal received by the LVDS receiving module is a serial differential signal input by a serializer chip, the serial signal is sent into the FPGA and converted into a parallel signal by an internal IP core, and image data in the parallel signal is mapped into RGB888 data; the digital image data received by the acquisition module is temporarily stored in the DDR3, and when necessary, the digital image data is taken out from the DDR3 and sent to the data gating module to be combined with the RGB888 signal mapped out to form a new frame of image data. The data gating module is used for selecting and outputting two paths of video data, the selected data are converted into 8 paths of 7-bit parallel data through the RGB reflection module, the 8 paths of 7-bit parallel data are output to the outside of the FPGA after being subjected to parallel-serial conversion through the LVDS sending module, and finally a picture formed by recombining the two paths of video data is displayed in one frame of image.

2. The vehicle-mounted video image picture-in-picture display method based on the FPGA of claim 1, which is characterized in that: the FPGA chip is an LFE5U-45F8BG381CA7131R45 chip under ECP5 series of LATTICE, and the speed grade is 6.

3. The vehicle-mounted video image picture-in-picture display method based on the FPGA of claim 1, which is characterized in that: the serializer chip is selected to be a DS90UB947-Q1 chip.

4. The vehicle-mounted video image picture-in-picture display method based on the FPGA of claim 1, which is characterized in that: the DDR3 chip selects SCB13H2G160AF-13K as a memory.

Images