CN115546001A - Extensible real-time image acquisition, processing and display system and method - Google Patents

Abstract

An extensible real-time image acquisition, processing and display system and method, the system includes the image acquisition device, image processing module based on FPGA development board and image display module; the parameter of the image acquisition device can be configured, the image processing module comprises a clock module, an image acquisition device driving module, a DDR3 control module, an extensible image processing module, an HDMI display module and an LCD display module, the image acquisition device is connected with the image processing module through the image acquisition device driving module, and the image display module comprises an HDMI display and an LCD display; the expandable image processing module consists of respective image processing algorithm units; the DDR3 control module is connected with the DDR3 memory, and the DDR3 control module comprises an FIFO control module, an MIG IP core and a DDR3 read-write module. The invention has strong expandability and can be used as a basic platform to develop deep image processing application algorithms.

Description

Extensible real-time image acquisition, processing and display system and method

Technical Field

The invention belongs to the field of digital integrated circuit design, and particularly relates to an extensible real-time image acquisition, processing and display system and method.

Background

Digital image processing is an active research field in computer science, and with the improvement of social informatization level and the leap-over development of information technology, the demand of human production and life for digital image information is gradually enlarged, and the digital image processing method is widely applied to various industries such as security monitoring, biological medical treatment, military guidance, video media and the like. The field programmable logic array (FPGA) is widely applied to the fields of communication, image processing and the like by virtue of the characteristics of high flexibility, short development period, strong processing performance (parallelism) and the like. Therefore, performing image processing based on FPGAs is a popular issue to be studied.

According to the requirements of practical application occasions, different image processing algorithms are required to be adopted. And a mature, reliable and high-expansibility image acquisition and display system is built, so that the image processing effect can be debugged and verified more quickly. And the FPGA exerts the expansibility of the whole system to the maximum extent by virtue of the characteristic of flexibility. On the basis of the platform, deeper algorithms and functions can be developed, and further, the realization of a larger-scale system is realized.

Disclosure of Invention

The invention aims to provide an extensible real-time image acquisition, processing and display system and method, which can automatically configure the register parameters of a camera according to an application scene, automatically add a corresponding image processing unit, have very good extensibility, and can use the system as a basic platform and continuously develop a deep image processing application algorithm based on the platform.

In order to achieve the purpose, the invention has the following technical scheme:

the first aspect provides an extensible real-time image acquisition, processing and display system, which comprises an image acquisition device, an image processing module constructed based on an FPGA development board, and an image display module; the parameter of the image acquisition device is configurable, the image processing module comprises a clock module, an image acquisition device driving module, a DDR3 control module, an extensible image processing module, an HDMI display module and an LCD display module, the image acquisition device is connected with the image processing module through the image acquisition device driving module, the image display module comprises an HDMI display and an LCD display, and the HDMI display and the LCD display are respectively connected with the HDMI display module and the LCD display module of the image processing module; the extensible image processing module is composed of respective image processing algorithm units and can realize real-time processing of the acquired image; the DDR3 control module is connected with the DDR3 memory, the DDR3 control module comprises an FIFO control module, an MIG IP core and a DDR3 read-write module, the buffer memory between the read-write data is realized through the read-write FIFO and the MIG IP core, and the DDR3 read-write module judges when the data is written into and read out of the DDR3 memory through the data volume in the read-write FIFO; the HDMI display module and the LCD display module follow respective display protocols to transmit image data, and the HDMI display and the LCD display complete image display.

As a preferred scheme of the system of the invention, the image acquisition device adopts an OV5640 camera, and registers in the image acquisition device are configured through an SCCB protocol interface, so that parameter configurability is realized; the DDR3 memory adopts a chip with a specification of 128M 16bit, and the chip is interactively controlled through an MIG IP core developed by Xilinx; the image processing module is built based on an FPGA development board of an Artix-7 series chip of Xilinx.

As a preferred scheme of the system of the present invention, the image capturing device driving module includes an I2C driving module, a register configuration module, and an image capturing module; the register configuration module completes the power-on operation of the OV5640 camera and the configuration of a corresponding register according to the power-on requirement of the OV5640 camera and a data manual and a corresponding time sequence; the I2C driving module records the number of the configuration registers according to the I2C configuration completion signal and configures corresponding parameters for each register according to the counter; the image acquisition module completes bit width conversion from 8 bits to 16 bits according to a clock signal, a line field synchronizing signal and image data output by the OV5640 camera and outputs the converted signal to the DDR3 control module.

As a preferred scheme of the system of the invention, a read-write FIFO IP core is instantiated inside the FIFO control module, the FIFO control module controls the interaction of data between the DDR3 control module and the image acquisition device, the HDMI display module and the LCD display module, wherein the write FIFO acquires data from the image acquisition module, and writes the data written into the FIFO into the DDR3 memory according to the read enable of the DDR3 control module; the read FIFO acquires data from the read FIFO according to the read request signals of the HDMI display module and the LCD display module, and the DDR3 control module generates write enable of the read FIFO and writes the data in the DDR3 memory into the read FIFO.

As a preferred scheme of the system of the present invention, the HDMI display module is composed of a video driver module, a coding module, and a parallel-serial conversion module; the image format adopted by the image acquisition module is RGB565, HDMI transmission is carried out according to RGB888 format, the video drive module generates line-field synchronizing signal, enabling signal and RGB888 image data according to VGA time sequence protocol and time sequence parameter corresponding to resolution, and RGB565 is converted into RGB888 format by filling 0 low bit; the encoding module performs 8-bit/10-bit encoding conversion on R, G and B three-channel data of the image according to the requirements of a TMDS algorithm; the parallel-serial conversion module realizes 10 parallel-serial transmission based on OSERDESE2 primitives, wherein OSERDESE2 adopts DDR working mode, and finally converts R, G, B and clock signals into differential signal transmission according to TMDS requirements through OBUFDS primitives.

As a preferred scheme of the system of the present invention, the MIG IP core is an IP developed specifically for DDR3 memory, and a control module of the memory is integrated inside the MIG IP core; the MIG IP core comprises two interfaces, one is a physical interface of the storage, and the other is an application interface.

As a preferred scheme of the system of the present invention, the DDR3 control module completes control of the MIG IP core according to an application interface signal of the MIG IP core, and further the MIG IP core completes control of the DDR3 memory.

As a preferred scheme of the system of the present invention, the clock module provides clock drive for the MIG IP core, the image acquisition device driving module, the HDMI display module, and the LCD display module; the clock module generates both 50MHz and 200MHz clocks through the PLL IP core.

As a preferred scheme of the system of the invention, the LCD display module comprises an LCD clock frequency division module, an LCD ID capture module, an LCD drive module and an LCD display module; the ID of the LCD is obtained according to the corresponding electric potentials of three resistors fixed on the LCD screen, wherein the electric potentials are the highest positions of R, G and B respectively, the resolution of the screen is determined according to the ID of the LCD, and then the driving clock of the LCD is determined according to the ID; the LCD clock frequency division module divides frequency according to an input clock, and functions in a pulse effective mode in a clock enabling mode; the LCD driving module generates a line-field synchronizing signal and an LCD control signal according to the time sequence parameter of the resolution ratio and the transmission standard of the VGA; the LCD display module selectively controls the displayed image position and the input image according to the positions of the pixel points.

In a second aspect, a scalable real-time image capture, processing and display method is provided, comprising the steps of:

generating a required clock by a clock module;

modifying and configuring the register parameters of the image acquisition device according to application requirements;

writing the parameters into corresponding registers according to addresses according to the writing operation time sequence of the SCCB protocol;

after an image acquisition device is stabilized, acquiring original pixel data according to a pixel clock, and completing bit width conversion;

transmitting the data signal which completes bit width conversion into an expandable image processing module according to the requirement to complete corresponding processing;

the processed data enters a write FIFO, and the write operation corresponds to one write of the DDR3 memory every set number of times; completing the conversion of read-write logic according to the burst length;

reading data from the DDR3 memory once every set number of reading operations, writing the data into a read FIFO, and completing bit width decomposition;

and finishing real-time display of data according to the display protocols of the HDMI and the LCD.

Compared with the prior art, the invention has the following beneficial effects:

the parameters of the image acquisition device can be configured, the register parameters of the image acquisition device can be configured according to an application scene, and a corresponding image processing unit is added automatically. The DDR3 control module realizes the cache between the read data and the write data through the read-write FIFO and the MIG IP core, the DDR3 read-write module judges when the data is written into and read out of the DDR3 memory through the data volume in the read-write FIFO, and generates a correct read-write enabling signal according to the signal of the read-write port. The extensible image processing module is composed of respective image processing algorithm units, is placed on a data path between the image acquisition device driving module and the DDR3 control module, and achieves real-time processing of acquired images. Of course, the scalable image processing module may be placed on the data path between the output of the DDR3 control module and the HDMI display module, depending on the image processing algorithm. The extensible image processing module provides common image processing algorithms including gray processing, binarization, median filtering, sobel edge detection, corrosion, expansion, histogram equalization, white balance, inter-frame filtering and the like, and realizes the functions of image identification, moving target detection and the like based on the common image processing algorithms. The developer can also develop the image processing module according to the application requirement of the developer, and the platform is used for debugging and verifying the algorithm and is finally embedded into the platform for use. The invention can be expanded to configure the image processing module according to different application scene requirements, and besides the provided well-debugged common image processing algorithm unit, the user can also debug and verify the image processing algorithm designed by the user on board level and quickly arrange the image processing algorithm in the system. On the basis, the scale of the image processing module can be continuously expanded so as to realize more complex functions such as image recognition, moving target detection and the like.

Drawings

FIG. 1 is a block diagram of an extensible real-time image acquisition, processing and display system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of SCCB transmission protocol according to the embodiment of the invention;

fig. 3 (a) is a block diagram of a camera driving module of OV5640 according to an embodiment of the present invention;

fig. 3 (b) is a diagram of an OV5640 camera driving module RTL according to an embodiment of the present invention;

FIG. 4 (a) is a block diagram of a DDR3 control module according to an embodiment of the invention;

FIG. 4 (b) is a diagram of a DDR3 control module RTL according to the embodiment of the invention;

FIG. 5 (a) is a block diagram of an HDMI display module according to an embodiment of the present invention;

FIG. 5 (b) is a diagram of an HDMI display module RTL according to an embodiment of the present invention;

FIG. 6 (a) is a block diagram of an LCD display module according to an embodiment of the present invention;

FIG. 6 (b) is a diagram of an LCD display module RTL according to one embodiment of the present invention;

FIG. 7 (a) is a block diagram of an extensible image processing module according to an embodiment of the present invention;

fig. 7 (b) is a diagram of an expandable image processing module RTL according to an embodiment of the present invention;

fig. 8 is a diagram of an expandable real-time image capture, processing and display system RTL according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings.

As shown in fig. 1, in the scalable real-time image capturing, processing and displaying system according to the embodiment of the present invention, except for the clock module, each module includes a plurality of sub-modules to implement corresponding functions. The image acquisition device driving module comprises three sub-modules, namely an I2C driving module, a register configuration module and an image acquisition module. The DDR3 control module comprises a DDR3 read-write module, an FIFO control module and an MIG IP core module. The extensible image processing module is composed of respective image processing algorithm units, and can realize real-time processing of the acquired images. The HDMI display module comprises a video driving module, a coding module and a parallel-serial conversion module. The LCD display module comprises an LCD clock frequency division module, an LCD ID capture module, an LCD driving module and an optional LCD display module. The details of each module are explained below:

referring to fig. 2, the SCCB transmission protocol according to an embodiment of the present invention firstly sends a start signal of one bit, and then sends a device address of 7 bits for a write operation timing sequence; next, the register address to be written is transmitted, and since the internal register address of the OV5640 camera is 16 bits, it is necessary to transmit twice. The first time the upper 8 bits are transmitted and the second time the lower 8 bits are transmitted. After the register address is written, configured 8-bit data can be written, and after the writing is completed, a stop signal is sent to complete one configuration operation. It can be seen that after each write of an address or data, there is a one-bit X signal indicating that the value returned by the slave is not of concern, whereas for the I2C protocol the slave must return a response signal indicating that the write is valid. In addition, there is no difference in write operations of the SCCB protocol and the I2C protocol. For a read operation, the SCCB needs to send a stop bit after completing one write operation of the device address and the register address (i.e., one dummy write operation) to perform the next read operation. Whereas the I2C protocol does not need to send a stop bit but a restart operation. This is the greatest difference between the two. The extensible real-time image acquisition, processing and display system only needs to configure a register in the OV5640 camera and does not relate to reading operation, so that the I2C drive is fixed to be in a writing mode.

Referring to fig. 3 (a) and 3 (b), the image pickup device driving module includes an I2C driving module, a register configuration module, and an image pickup module. The register configuration module completes the power-on operation of the OV5640 camera and the configuration of the corresponding register according to the power-on requirement of the OV5640 camera and a data manual and the corresponding time sequence. Firstly, after being electrified, the register can be configured only by waiting for at least 20 ms; secondly, recording the number of the configuration registers according to the I2C configuration completion signal, and configuring corresponding parameters for each register according to the counter. The image acquisition module completes bit width conversion from 8 bits to 16 bits according to a clock signal, a line-field synchronizing signal and image data output by the camera and outputs the converted data to the DDR3 control module.

Referring to fig. 4 (a) and fig. 4 (b), the DDR3 control module includes a DDR3 read/write module, an FIFO control module, and an MIG IP core, where the FIFO control module instantiates a read/write FIFO IP core internally. The FIFO control module controls data interaction between the DDR3 memory and the OV5640 camera and the HDMI/LCD display module, the write FIFO acquires data from the image acquisition module, and each write data of the DDR3 memory is 128 bits, so that 8 data are required to be spliced into effective write data, and write enable is generated. Similarly, the write FIFO needs to write the data written into the FIFO into the DDR3 memory according to the read enable of the DDR3 read/write module. The read FIFO also obtains data from the read FIFO according to a read request signal of the HDMI or LCD module, and because the data in the FIFO is set according to the data bit width of the DDR3 memory and the display adopts the format of RGB565 or RGB888, a read operation of reading the FIFO is correspondingly requested every 8 times. And the DDR3 read-write module generates write enable of the read FIFO and writes the data in the DDR3 into the read FIFO. The caching of the acquired image data is completed through such a mechanism. The MIG IP core is an IP specially developed for a DDR3 memory, and a control module of the memory is integrated inside the MIG IP core. The MIG IP core mainly comprises two interfaces, one is a physical interface of the storage, and the other is an application interface. The DDR3 read-write module needs to complete control over the MIG IP core according to an application interface signal of the MIG IP core, and then the MIG IP core completes control over the DDR3 memory. The DDR3 clock frequency configured in the embodiment is the maximum value 400MHz of the normal operation of the chip, the ratio from the PHY to the control clock is fixed to be 4. In order to generate a maximum physical clock of 400MHz, an input clock provided by a clock module to an MIG IP core is set to be 200MHz, and memory address coding adopts a BANK + Row + COLUMN form, so that the memory address coding is in accordance with a conventional addressing mode, and power consumption is reduced.

Referring to fig. 5 (a) and 5 (b), the HDMI display module is mainly composed of a video driving module, an encoding module, and a parallel-serial conversion module. The image format adopted by the image acquisition module is RGB565, and HDMI transmission is performed according to RGB888 format. Therefore, the video driver module is required to generate the line-field synchronization signal, the enable signal and the RGB888 image data according to the timing protocol of the VGA and the timing parameter corresponding to the resolution, and in the embodiment of the present invention, the RGB565 is converted into the RGB888 format by filling 0 low bit. And the coding module performs 8-bit/10-bit coding conversion on the R, G and B three-channel data of the image according to the requirements of the TMDS algorithm. After being coded by the TMDS algorithm, the signal has better electromagnetic compatibility, the algorithm can reduce the overshoot and undershoot in the signal transmission process, and can realize long-distance and high-quality digital signal transmission. The parallel-serial conversion module is based on OSERDESE2 primitive and realizes 10. Wherein OSERDESE2 adopts DDR mode, therefore only 5 times pixel clock is needed to complete the transmission of 10. And finally, converting R, G, B and clock signals into differential signal transmission according to TMDS requirements through OBUFDS primitives.

Referring to fig. 6 (a) and 6 (b), the LCD display module is used as an optional display module, mainly for assisting in displaying some extra information in the image. For example, in the histogram equalization algorithm of the image, in order to observe the change before and after the histogram equalization more intuitively, the image before and after the processing is directly compared, the histogram can be displayed through an LCD display screen, and the change before and after the histogram equalization can be seen better. The LCD display module comprises an LCD clock frequency division module, an LCD ID capture module, an LCD drive module and an optional LCD display module. The ID of the LCD is obtained according to the corresponding electric potentials of three resistors fixed on the LCD screen, and is the highest bit of R, G and B. The input 50MHz clock is divided according to the ID as the clock of the LCD. And obtaining corresponding parameters of the VGA time sequence according to the ID. The LCD driving module generates an enabling signal, a data request signal and x and y coordinates of pixel points according to the corresponding time sequence parameters and a VGA transmission protocol. The LCD panel has two data synchronization modes, one is a line-field synchronization Mode (HV Mode) and the other is a data enable synchronization Mode (DE Mode). Here, for convenience of the device, the data enable sync mode is selected so that the line field sync signal can be set to a high level. Similarly, the reset signal of the LCD is active low, set to 1, and no reset operation is performed. The LCD also has a backlight control signal, which can be used to adjust the backlight of the screen by generating a PWM signal, which is also set to a high level, i.e., the screen is normally on at maximum brightness. And finally, the optional LCDdisplay module selectively sends out a request signal according to coordinates and x and y coordinates of pixel points transmitted by the LCD driving module to realize flexible change of the image position, and the function needs to be configured and selected according to a specific use scene.

Referring to fig. 7 (a) and 7 (b), the interior of the expandable image processing module may include a plurality of image processing algorithm units, some image processing algorithms may be directly cascaded, and some image processing algorithms may not be needed, so that the connection lines between the interior are not identified. For simple image processing units, such as grayscale processing and binarization, only calculation processing needs to be performed on input image data according to a corresponding algorithm. For algorithms such as median filtering, edge detection, erosion, dilation, white balance, etc., convolution operations are required. Therefore, a shift RAM needs to be constructed by two RAM IP cores to realize the generation of a 3 × 3 matrix, based on the matrix, the convolution operation can be performed according to the corresponding algorithm convolution core, and finally, corresponding image data is output. The extensible image processing module in fig. 7 according to the embodiment of the present invention includes a gray processing, median filtering, binarization, sobel edge detection, erosion, expansion, and histogram equalization algorithm unit. The detection of moving objects using frame difference method is also illustrated. And finally, performing board-level debugging and verification on the built complete system on the FPGA, wherein an RTL schematic diagram is shown in FIG. 8.

Another embodiment of the present invention further provides an extensible real-time image acquisition, processing and display method, including the following steps:

s1: and synthesizing the PLL IP core of the clock module to generate the required clock.

S2: parameters of up to 248 registers of the OV5640 camera are modified and configured according to application requirements.

S3: and writing the parameters into the corresponding registers according to the address according to the writing operation sequence of the SCCB protocol.

S4: and after the OV5640 camera stabilizes a certain number of frames, acquiring original pixel data according to a pixel clock, and completing bit width conversion from 8 bits to 16 bits.

S5: and transmitting the data signal which completes the bit width conversion into an expandable image processing module according to the requirement, and completing a corresponding processing algorithm.

S6: and the processed data enters a write FIFO, and each 8 write operations correspond to one 128-bit write of the DDR3 memory. And completing the conversion of the read-write logic according to the burst length.

S7: and reading data from the DDR3 memory once every 8 times of reading operation, writing the data into a read FIFO, and completing the decomposition from 128bit to 16 bit.

S8: and finishing the real-time display of the data according to the display protocols of the HDMI and the LCD.

The image acquisition device driving module adopts an I2C protocol, can realize the configurability of image acquisition parameters (the maximum resolution of 1920 multiplied by 1080 can be realized), and converts the initial 8bit data acquired by the camera into image data in an RGB565 format. And the DDR3 control module realizes the cache between the read data and the write data through the read-write FIFO and the MIG IP core, and completes the control on the MIG IP core according to an application interface signal of the MIG IP core, thereby controlling the work of the DDR3 memory. The HDMI display module is used for displaying processed images, encodes the images according to a TMDS algorithm, and then serially transmits the encoded images at a speed of 10. The optional LCD display module is used for displaying some additional information of the image, so that the effect before and after the image processing can be observed more visually. The system of the invention can be expanded, the image processing module can be configured according to the requirements of different application scenes, besides the provided well-debugged common image processing algorithm unit, a user can also utilize the platform to carry out board-level debugging and verification on the image processing algorithm designed by the user, and the image processing algorithm can be rapidly arranged in the system. On the basis, the scale of the image processing module can be continuously expanded so as to realize more complex functions such as image recognition, moving object detection and the like.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present application, and they should be construed as being included in the present application.

Claims (10)

1. An extensible real-time image acquisition, processing and display system is characterized by comprising an image acquisition device, an image processing module constructed based on an FPGA development board, and an image display module; the parameter of the image acquisition device can be configured, the image processing module comprises a clock module, an image acquisition device driving module, a DDR3 control module, an extensible image processing module, an HDMI display module and an LCD display module, the image acquisition device is connected with the image processing module through the image acquisition device driving module, the image display module comprises an HDMI display and an LCD display, and the HDMI display and the LCD display are respectively connected with the HDMI display module and the LCD display module of the image processing module; the extensible image processing module is composed of respective image processing algorithm units and can realize real-time processing of the acquired images; the DDR3 control module is connected with the DDR3 memory, the DDR3 control module comprises an FIFO control module, an MIG IP core and a DDR3 read-write module, the buffer memory between read-write data is realized through the read-write FIFO and the MIG IP core, and the DDR3 read-write module judges when to write and read data into the DDR3 memory according to the data volume in the read-write FIFO; the HDMI display module and the LCD display module follow respective display protocols to transmit image data, and the HDMI display and the LCD display complete image display.

2. The scalable real-time image capture, processing and display system of claim 1, wherein the image capture device employs an OV5640 camera, and registers therein are configured via SCCB protocol interface to enable parameter configurability; the DDR3 memory adopts a chip with a specification of 128M 16bit, and the chip is interactively controlled through an MIG IP core developed by Xilinx; the image processing module is built based on an FPGA development board of an Artix-7 series chip of Xilinx.

3. The scalable real-time image capture, processing and display system of claim 2, wherein the image capture device driver module comprises an I2C driver module, a register configuration module and an image capture module; the register configuration module completes the power-on operation of the OV5640 camera and the configuration of a corresponding register according to the power-on requirement of the OV5640 camera and a data manual and a corresponding time sequence; the I2C driving module records the number of the configuration registers according to the I2C configuration completion signal and configures corresponding parameters for each register according to the counter; the image acquisition module completes bit width conversion from 8 bits to 16 bits according to a clock signal, a line field synchronizing signal and image data output by the OV5640 camera and outputs the converted signal to the DDR3 control module.

4. The scalable real-time image capture, processing and display system of claim 3, wherein the FIFO control module instantiates a read-write FIFO IP core inside, the FIFO control module controlling data interaction between the DDR3 control module and the image capture device and between the HDMI display module and the LCD display module, wherein the write FIFO obtains data from the image capture module, the write FIFO writes the data written into the FIFO into the DDR3 memory according to read enable of the DDR3 control module; the read FIFO acquires data from the read FIFO according to the read request signals of the HDMI display module and the LCD display module, and the DDR3 control module generates write enable of the read FIFO and writes the data in the DDR3 memory into the read FIFO.

5. The scalable real-time image capture, processing and display system of claim 3, wherein the HDMI display module is comprised of a video driver module, an encoding module and a parallel-to-serial conversion module; the image format adopted by the image acquisition module is RGB565, HDMI transmission is carried out according to RGB888 format, the video drive module generates line-field synchronizing signals, enabling signals and RGB888 image data according to the time sequence protocol of VGA and time sequence parameters corresponding to resolution, and the RGB565 is converted into the RGB888 format by adopting a low-order filling 0 mode; the encoding module performs 8-bit/10-bit encoding conversion on the data of the three channels of R, G and B of the image according to the requirement of a TMDS algorithm; the parallel-serial conversion module is based on OSERDESE2 primitive, and realizes 10-degree parallel-serial transmission, wherein OSERDESE2 adopts DDR working mode, and finally converts R, G, B and clock signals into differential signal transmission according to TMDS requirement through OBUFDS primitive.

6. The scalable real-time image capture, processing, and display system of claim 1, wherein the MIG IP core is an IP developed specifically for DDR3 memory, with a memory control module integrated inside; the MIG IP core comprises two interfaces, one is a physical interface of a memory, and the other is an application interface.

7. The scalable real-time image capture, processing, and display system of claim 6, wherein the DDR3 control module controls the MIG IP core according to the MIG IP core application interface signal, and further the MIG IP core controls the DDR3 memory.

8. The scalable real-time image capture, processing, and display system of claim 1, wherein the clock module provides clock driving for the MIG IP core, the image capture device driver module, the HDMI display module, and the LCD display module; the clock module generates both 50MHz and 200MHz clocks through the PLL IP core.

9. The scalable real-time image capture, processing and display system of claim 1, wherein the LCD display module comprises an LCD clock divider module, an LCDID capture module, an LCD driver module and an LCD display module; the ID of the LCD is obtained according to the corresponding electric potentials of three resistors fixed on the LCD screen, wherein the electric potentials are the highest positions of R, G and B respectively, the resolution of the screen is determined according to the ID of the LCD, and then the driving clock of the LCD is determined according to the ID; the LCD clock frequency division module divides frequency according to an input clock, and functions in a pulse effective mode in a clock enabling mode; the LCD driving module generates a line-field synchronous signal and an LCD control signal according to the time sequence parameter of the resolution ratio and the transmission standard of the VGA; the LCD display module selectively controls the displayed image position and the input image according to the position of the pixel point.

10. An extensible real-time image acquisition, processing and display method is characterized by comprising the following steps:

generating a required clock by a clock module;

modifying and configuring the register parameters of the image acquisition device according to application requirements;

writing the parameters into corresponding registers according to addresses according to the writing operation time sequence of the SCCB protocol;

after an image acquisition device is stabilized, acquiring original pixel data according to a pixel clock, and completing bit width conversion;

transmitting the data signal which completes bit width conversion into an expandable image processing module according to the requirement to complete corresponding processing;

the processed data enters a write FIFO, and the write operation corresponds to one write of the DDR3 memory every set number of times; completing the conversion of read-write logic according to the burst length;

reading data from the DDR3 memory once every set number of reading operations, writing the data into a read FIFO, and completing bit width decomposition;

and finishing real-time display of data according to the display protocols of the HDMI and the LCD.

Images