CN115002304B - Video image resolution self-adaptive conversion device - Google Patents
Video image resolution self-adaptive conversion device Download PDFInfo
- Publication number
- CN115002304B CN115002304B CN202210380813.2A CN202210380813A CN115002304B CN 115002304 B CN115002304 B CN 115002304B CN 202210380813 A CN202210380813 A CN 202210380813A CN 115002304 B CN115002304 B CN 115002304B
- Authority
- CN
- China
- Prior art keywords
- ddr3
- module
- bram
- write control
- control module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 15
- 238000003860 storage Methods 0.000 claims abstract description 72
- 238000012545 processing Methods 0.000 claims abstract description 64
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 8
- 239000013078 crystal Substances 0.000 claims description 7
- 238000000034 method Methods 0.000 claims description 6
- 230000003139 buffering effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 229920002430 Fibre-reinforced plastic Polymers 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011151 fibre-reinforced plastic Substances 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/907—Television signal recording using static stores, e.g. storage tubes or semiconductor memories
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a video image resolution self-adaptive conversion device, which comprises: the device comprises a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module. The invention aims to solve the problem that when a CMOS camera is used, the resolution of a real-time video stream and the resolution of a terminal display are difficult to adaptively convert. The invention provides a CMOS camera DVP interface, a CMOS camera MIPI interface, a VGA video output interface, an HDMI video output interface, 1080P high-definition video stream display and DDR3 large-capacity video data frame storage, which is suitable for centralized access and provides a brand new solution for real-time adjustment of resolution of a CMOS camera video stream and on-screen display of multiple video streams.
Description
Technical Field
The invention relates to the fields of image video acquisition technology, image encoding and decoding technology, digital image processing analysis and the like. In particular to a video image processing device which is used in a scene of a multi-terminal display screen and is used for real-time adaptation and scaling of the display resolution of a video stream and adjustment of the display direction so as to achieve the optimal display effect.
Technical Field
Video images are the most direct means of human observation and perception, and video image processing techniques are widely used in various fields today, such as industrial production, public transportation, aerospace, clinical medicine, and the like. The video image processing technology greatly improves the expressive force of pictures and images, and especially people enlarge and reduce the pictures by using an image interpolation algorithm, so as to obtain richer detail information in the pictures and the images.
The traditional video image processing technology based on a computer uses a CPU as a central processing unit, and builds a digital image processing system by means of computer software and an operating system. People complete program development by means of computer languages C, C ++ and the like, and can complete interaction with underlying hardware to realize image processing. Image processing algorithms have been implemented by such systems in most scenarios to date. Although the performance of the CPU of the computer is stronger and the processing speed is faster, the processing speed is limited by the principle of a serial processing mode of single instruction and single data, and the computer also needs to run an operating system, so that the application field of the computer is greatly limited, and the computer is not suitable for scenes with high real-time requirements.
In recent years, with rapid progress and development of technologies such as image acquisition and video processing, resolution, storage size and processing speed of images are all higher and higher, while requirements on hardware size of an image processing system are smaller and smaller, and processing instantaneity is also required to be higher and higher. This presents a more serious challenge to conventional computer software-based image processing platforms. Therefore, it is necessary to jump out of the traditional processing mode based on the computer software platform, and search for another processing platform and mode capable of meeting new requirements.
Disclosure of Invention
The invention aims to solve the problem of adaptive optimization of the resolution of a real-time video stream and the resolution of a terminal display when a CMOS camera is used in various terminal display screen scenes, and provides a video image resolution adaptive conversion device which can well realize multi-screen display of an aircraft cockpit, one-screen multi-display of mobile phone camera shooting, multi-screen monitoring of a factory production line, multi-screen real-time monitoring of public transportation and the like in various application scenes, can amplify and reduce the resolution of the video stream by any multiple in real time and high efficiency according to requirements, and can stably and clearly display the video image resolution at any target position of the terminal display screen.
The specific technical scheme for realizing the aim of the invention is as follows:
the video image resolution self-adaptive conversion device is characterized by comprising a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module, wherein the camera decoding module is connected with the DDR3 storage read-write control module and the clock module; the DDR3 storage read-write control module is connected with the camera decoding module, the BRAM storage read-write control module and the clock module; the BRAM storage read-write control module is connected with the DDR3 storage read-write control module, the Scaler algorithm processing module, the video stream output module and the clock module; the Scaler algorithm processing module is connected with the BRAM storage read-write control module and the clock module; the video stream output module is connected with the BRAM storage read-write control module and the clock module; the clock module is connected with the camera decoding module, the DDR3 storage read-write control module, the BRAM storage read-write control module, the Scaler algorithm processing module and the video stream output module; the camera decoding module comprises a CMOS camera, a DVP interface or an MIPI interface on an FPGA chip; the DDR3 storage read-write control module comprises a DDR3 chip, a DDR3 input FIFO in an FPGA chip and a DDR3 output FIFO in the FPGA chip; the BRAM storage read-write control module comprises a BRAM in the FPGA chip and a BRAM output FIFO in the FPGA chip; the Scaler algorithm processing module comprises a Scaler algorithm circuit and matrix keys in the FPGA chip; the video stream output module comprises a VGA interface or an HDMI interface on the display screen and the FPGA chip; the clock module comprises a crystal oscillator and a phase-locked loop in the FPGA chip; the DVP interface or MIPI interface, DDR3 input FIFO, DDR3 output FIFO, BRAM, BRAM output FIFO, scaler algorithm circuit, VGA interface or HDMI interface and phase-locked loop are all circuits inside the FPGA chip.
And a DVP interface or an MIPI interface of the camera decoding module is connected with the CMOS camera, the phase-locked loop and the DDR3 input FIFO.
The DDR3 input FIFO of the DDR3 storage read-write control module is connected with the DDR3 chip, the DVP interface or the MIPI interface and the phase-locked loop; the DDR3 chip is connected with the DDR3 input FIFO and the DDR3 output FIFO; the DDR3 output FIFO is connected with the DDR3 chip, the BRAM and the phase-locked loop; the DDR3 memory read-write control module provides high-speed frame buffering.
The BRAM of the BRAM storage read-write control module is connected with the DDR3 output FIFO, the Scaler algorithm circuit and the phase-locked loop; the BRAM output FIFO is connected with the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop; the BRAM storage read-write control module realizes the secondary cache of three rows of pixel points and provides accurate original pixel points for the Scaler algorithm processing module.
The Scaler algorithm circuit of the Scaler algorithm processing module is connected with the matrix keys, the BRAM output FIFO and the phase-locked loop; the Scaler algorithm circuit in the module realizes the Scaler algorithm innovated by the invention, and interpolates the image.
And a VGA interface or an HDMI interface of the video stream output module is connected with the display screen, the BRAM output FIFO and the phase-locked loop.
The phase-locked loop of the clock module is connected with a crystal oscillator, a DVP interface or an MIPI interface, a DDR3 input FIFO, a DDR3 output FIFO, BRAM, BRAM output FIFO, a Scaler algorithm circuit and a VGA interface or an HDMI interface.
Compared with the prior art, the invention has the beneficial effects that:
(1) The FPGA is used as a main video image data processing chip, tasks can be executed in a parallel mode, and the method has shorter processing time and stronger task responsiveness. And the internal hardware modules of the FPGA can be customized conveniently and reused.
(2) The invention innovatively optimizes the bilinear interpolation algorithm, optimizes floating point operations related to multiplication and division in the traditional algorithm into unsigned integer multiplication operations, ensures that the algorithm is innovatively adapted to the operation mode of the FPGA, ensures the best effect of the algorithm, simultaneously greatly exerts the high speed of processing data by the FPGA and consumes fewer internal logic units of the FPGA.
(3) The DDR3 chip with high reading and writing speed and large storage capacity is adopted as the cache chip, so that the video image data caching capacity is improved, the video image data acquisition capacity is greatly improved, more dense and more complete video image frames can be cached, and effective information extraction can be performed on the video image frames in time. The video image data throughput in the overall function is extremely high.
(4) The invention adopts an integrated solution of picture information acquisition, storage control and display control, and can finish the processes of extracting video picture information from the CMOS camera, storing and controlling video image frames and displaying and controlling a terminal display screen through an independent module. The topology structure and the implementation way of the scheme are simplified.
Drawings
FIG. 1 is a block diagram of the apparatus of the present invention;
fig. 2 is a flow chart of the operation of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and examples.
Referring to fig. 1, the invention comprises a camera decoding module 1, a DDR3 memory read-write control module 2, a BRAM memory read-write control module 3, a Scaler algorithm processing module 4, a video stream output module 5 and a clock module 6; the camera decoding module 1 is connected with the DDR3 storage read-write control module 2 and the clock module 6 and is used for receiving and decoding video stream data of the CMOS camera; the DDR3 storage read-write control module 2 is connected with the camera decoding module 1, the BRAM storage read-write control module 3 and the clock module 6, and is used for carrying out frame buffering on an input video stream, providing pixel point information for a subsequent algorithm, ensuring the integrity of the video stream in the transmission process and preventing data loss; the BRAM storage read-write control module 3 is connected with the DDR3 storage read-write control module 2, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6, and is used for caching three lines of video data in a high-speed ping-pong manner, so that the Scaler algorithm processing module 4 can read pixel points to be calculated at any time at high speed; the Scaler algorithm processing module 4 is connected with the BRAM storage read-write control module 3 and the clock module 6, and is used for reading parameters input by a user through keys, completing a core Scaler algorithm and adjusting target display resolution in real time; the video stream output module 5 is connected with the BRAM storage read-write control module 3 and the clock module 6, and is used for reading the DDC of the terminal display equipment and displaying the image processed by the algorithm; the clock module 6 is connected with the camera decoding module 1, the DDR3 memory read-write control module 2, the BRAM memory read-write control module 3, the Scaler algorithm processing module 4 and the video stream output module 5, and is used for providing reference clocks for all the modules.
The camera decoding module 1 comprises a DVP interface or HDMI interface 12 and a CMOS camera 11 on the FPGA chip, wherein the DVP interface or HDMI interface 12 is connected with the CMOS camera 11. The camera decoding module 1 completes the decoding and data splicing integration of the CMOS camera input video stream to obtain the standard RGB video stream time sequence with RGB565 data format and line field synchronizing signals.
The DDR3 memory read-write control module 2 comprises a DDR3 chip 22, a DDR3 input FIFO21 in the FPGA chip and a DDR3 output FIFO23 in the FPGA chip, wherein the DDR3 input FIFO21 is connected with the DDR3 chip 22, and the DDR3 output FIFO23 is connected with the DDR3 chip 21. The module drives the DDR3 chip, and ensures that the time sequence of address bus and data bus signals is accurate when the DDR3 chip reads and writes. The internal space of the DDR3 chip is divided into 256 parts, each of which has a size of 32Mbits, each of which is used to store 1 frame of picture.
The BRAM storage read-write control module 3 comprises a BRAM31 in the FPGA chip and a BRAM output FIFO32 in the FPGA chip. The module controls the 3 lines of pixels to be read from the DDR3 storage read-write control module and cached in the BRAM, and the BRAM caches 3 lines of pixel data altogether and provides ping-pong reading function for the Scaler algorithm processing module 4. At the same time, the module controls the pixel value to be output to be read from the Scaler algorithm processing module 4 and stored in the BRAM output FIFO32.
The Scaler algorithm processing module 4 comprises a Scaler algorithm circuit 41 and a matrix key 42 in the FPGA chip, and the Scaler algorithm circuit 41 is connected with the matrix key 42. The module reads parameters input by a user through keys and adjusts the resolution and the display orientation of a picture on a display screen in real time. The Scaler algorithm circuit 41 employs a bilinear interpolation algorithm. And carrying out linear weighted summation on the four pixel points read from the BRAM storage read-write control module 3, solving the value of the target pixel point, and writing the value into the BRAM output FIFO32 for the video stream output module 5 to read and display.
The video stream output module 5 comprises a display screen 51, a VGA interface on an FPGA chip or an HDMI interface 52 on the FPGA chip, and the display screen 51 is connected with the VGA interface or the HDMI interface 52. The module reads the DDC of the terminal display screen and determines the optimal display resolution of the display screen. And reading the pixel points processed by the algorithm from the BRAM output FIFO32, and outputting the pixel values of the corresponding coordinate points according to the line-field synchronous time sequence of the display screen to form a stable display picture.
The core algorithm of the present invention consists in the bilinear interpolation algorithm employed by the Scaler algorithm circuit 41. The Scaler algorithm circuit 41 reads the matrix keys 42 in real time to obtain parameters input by a user: displaying the starting abscissa X 0 Display of the initial ordinate Y 0 A display length L, and a display width H. When the amount of data in the BRAM output FIFO32 is less than half, the BRAM output FIFO32 issues a calculation request to the Scaler algorithm circuit 41. After receiving the calculation request, the Scaler algorithm circuit 41 starts to execute the algorithm according to the mapping formula of the display screen coordinates: (x, y) → (x a, y x B) to obtain a coordinate point (x) 0 ，y 0 ) Mapped coordinate point (x 0 *A，y 0 * B) A. The invention relates to a method for producing a fibre-reinforced plastic composite Wherein A and B are the length and width of the video source resolution, respectively. Then according to the mapping formula of the video source coordinates: (u, v) → (u×l, v×h), find (u) satisfying the following set of inequalities 0 ，v 0 )：
Thereafter, the Scaler algorithm circuit 41 sequentially reads the coordinates (u) from the BRAM31 0 ，v 0 )、(u 0 ，v 0 +1)、(u 0 +1，v 0 ) Sum (u) 0 +1，v 0 Four pixel values of +1): p (u) 0 ，v 0 )、P(u 0 ，v 0 +1)、P(u 0 +1，v 0 ) And P (u) 0 +1，v 0 +1), and then according to the bilinear interpolation formula:
P(x 0 ，y 0 )＝[P(u 0 ，v 0 )+P(u 0 ，v 0 +1)+P(u 0 +1，v 0 )+P(u 0 +1，v 0 +1)]calculating to obtain the pixel value P (x 0 ，y 0 ) Finally, the pixel value is stored in the BRAM output FIFO32, and the coordinate point to be solved is updated to be (x) 0 +1，y 0 )。
The working process of the invention is as follows:
after the FPGA development platform is electrified, initializing, reading DDC of the terminal display equipment, and setting target display resolution and azimuth by a user through keys. The camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6 start to work normally.
When the CMOS camera 11 captures a video image, the video stream is transferred to the DVP interface or HDMI interface 12. The camera decoding module 1 decodes the video stream in the DVP format or the HDMI format, converts the video stream into RGB565 color signals and line-field synchronous time sequence signals HS and VS, calculates display coordinate points x and y corresponding to RGB colors of an RGB color effective signal DE according to the line-field synchronous signals, and sends the display coordinate points x and y to the DDR3 input FIFO21 in the DDR3 storage read-write control module 2.
After receiving RGB565, HS, VS, DE, x and y signals, DDR3 memory read write control module 2 stores RGB565 data into asynchronous DDR3 input FIFO21 for buffering. The DDR3 input FIFO21 has a write clock frequency of 24MHz, a read clock frequency of 100Mhz, and a memory space of 8kbits, and can store 512 16bits of RGB565 data. Meanwhile, the DDR3 input FIFO21 also carries out bit width conversion on input data, 1 RGB565 data with 16bits is written into the FIFO every 1 clock period at the input end, 16 data is read out once every 1 clock period at the output end, 1 data with 256bits is spliced, the data is transmitted to the DDR3 chip 22, and the DDR3 chip 22 writes or reads out 256bits of data once.
The DDR3 memory read-write control module 2 continuously completes the read-write state control of the DDR3 chip 22, the whole DDR3 state machine works at the frequency of 100MHz, and the read-write throughput of the DDR3 chip 22 is up to 3.2GBps. The internal total space 1GBytes of the DDR3 chip 22 is divided into 256 pieces, each piece having a size of 4MBytes space, each piece being for storing 1 frame of picture. The jump procedure of the DDR3 state machine is as follows: the DDR3 chip 22 reads a complete frame of picture from the DDR3 input FIFO21, stores it in the first slice address space, and then sequentially reads data from the first slice address space and stores it in the DDR3 output FIFO23 until the data amount in the DDR3 output FIFO23 reaches half, completing the initialization of the state machine, and the state machine enters the idle mode. Then, when the data in the DDR3 output FIFO23 is less than half, the DDR3 chip 22 immediately enters a read mode, and half of the data amount of the DDR3 output FIFO23 is read from the address after the last reading is completed and stored in the DDR3 output FIFO23. When new data can be read in the DDR3 input FIFO21, the DDR3 chip 22 enters a data writing mode, reads new data from the DDR3 input FIFO21, and then writes the address last time, or just writes a complete image last time, jumps to the next address space, and continues writing new data. Because DDR3 chip 22 cannot read and write at the same time, the state machine design has a higher read state priority than the write state, and the read address always lags the write address by one address space.
The DDR3 output FIFO23 in the DDR3 memory read write control module 2 is used to buffer the data read out in the DDR3 chip 22. The read-write clock frequency of the DDR3 output FIFO23 is 100Mhz, the storage space is 8kbits, and 32 DDR3 chips 22 with 256bits can store output data. Meanwhile, the DDR3 output FIFO23 also carries out bit width conversion on the data, 1 DDR3 data of 256bits is written into the FIFO every 1 clock period DDR3 at the input end, 256 DDR3 data is split according to the sequence from high bit to low bit at the output end, 1 RGB565 data of 16bits is read out every 1 clock period, and the data is supplied to the subsequent BRAM storage read-write control module 3.
The BRAM31 in the BRAM storage read-write control module 3 is 37.5kbits in size and is used for storing exactly 3 lines of RGB565 data. The module firstly reads 2 lines of RGB565 data from DDR3 output FIFO23 and stores the data into the first 2 lines of BRAM31, thus completing the initialization of BRAM. Because the Scaler algorithm processing module 4 needs to read two lines of data at the same time, when the Scaler algorithm processing module 4 reads the 1 st line data and the 2 nd line data, the BRAM31 completes filling the 3 rd line data; when the Scaler algorithm processing module 4 reads the 2 nd and 3 rd line data, the BRAM31 completes filling the 1 st line data; when the Scaler algorithm processing module 4 reads the 3 rd line data and the 1 st line data, the BRAM31 completes filling the 2 nd line data, so that ping-pong operation of 3 rd line data access is realized.
The Scaler algorithm processing module 4 implements the core algorithm bilinear interpolation algorithm of the present invention. The module first reads the parameters entered by the user via matrix key 42: l, H, X0, Y0. Since the resolution of the input image is 800×480, when the resolution of the target output image is l×h and the pixel value of the coordinates (x, y) is required, the pixel values of the four original coordinate points (x×800/L, y×480/H), (x×800/l+1, y×480/H), (x×800/L, y×480/h+1), (x×800/l+1, y×480/h+1) are required to be weighted and averaged, so that the Scaler algorithm processing module 4 extracts 4 required coordinate points from the two lines of effective data of the BRAM31 each time at the working frequency of 400Mhz, performs weighted and averaged, calculates the pixel value of the target point, and outputs the pixel value to the BRAM output FIFO32.
Finally, the video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points through the VGA or HDMI interface 52 according to the line-field synchronous time sequence of the display screen 51, so as to form a stably displayed picture.
Examples
Referring to fig. 1, the present embodiment includes: the video camera comprises a camera decoding module 1, a DDR3 storage read-write control module 2, a BRAM storage read-write control module 3, a Scaler algorithm processing module 4, a video stream output module 5 and a clock module 6. The DVP interface or MIPI interface 12, DDR3 input FIFO21, DDR3 output FIFO23, BRAM31, BRAM output FIFO32, scaler algorithm circuit 41, VGA or HDMI interface 52, and phase-locked loop 62 are all circuits designed using the logic resources inside the FPGA chip 7.
The camera decoding module 1 is connected with the DDR3 storage read-write control module 2 and the clock module 6.
The DDR3 memory read-write control module 2 is connected with the camera decoding module 1, the BRAM memory read-write control module 3 and the clock module 6.
The BRAM storage read-write control module 3 is connected with the DDR3 storage read-write control module 2, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6.
The Scaler algorithm processing module 4 is connected with the BRAM storage read-write control module 3 and the clock module 6.
The video stream output module 5 is connected with the BRAM storage read-write control module 3 and the clock module 6.
The clock module 6 is connected with the camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4 and the video stream output module 5.
The camera decoding module 1 of the present embodiment includes a DVP interface or MIPI interface 12 connected to a CMOS camera 11. The camera decoding module 1 completes the decoding and data splicing integration of the video stream input by the CMOS camera 11, and obtains the standard RGB video stream time sequence with RGB565 data format and line-field synchronous signals.
The DDR3 memory read write control module 2 of the present embodiment includes a DDR3 input FIFO21, a DDR3 chip 22, and a DDR3 output FIFO23. The DDR3 chip 22 is connected to the DDR3 input FIFO21 and the DDR3 output FIFO23. The internal space of the DDR3 chip 22 is divided into 256 parts, each of which has a size of 32Mbits, each of which is used to store 1 frame of picture.
The BRAM storage read-write control module 3 of this embodiment includes a BRAM31 and a BRAM output buffer FIFO32. The BRAM storage read-write control module 3 reads three lines of pixel caches from the DDR3 output FIFO23 to the BRAM31, and the BRAM31 caches three lines of pixel data altogether, so as to provide ping-pong reading function for the Scaler algorithm processing module 4. At the same time, the BRAM storage read-write control module 3 reads the pixel value to be output from the Scaler algorithm circuit 41 and writes the pixel value into the BRAM output FIFO32.
The Scaler algorithm processing module 4 of the present embodiment includes a Scaler algorithm circuit 41 and a matrix key 42. The Scaler algorithm circuit 41 scans the key values of the matrix keys 42 in real time to acquire user input parameters. Meanwhile, a bilinear interpolation algorithm is adopted, the four pixel points read from the BRAM31 are subjected to linear weighted summation, the value of the target pixel point is obtained, and the value is written into the BRAM output FIFO32 for being read and displayed by the video stream output module 5.
The video stream output module 5 of the present embodiment includes a VGA or HDMI interface 52 and a display screen 51, and the display screen 51 is connected to the VGA or HDMI interface 52. The video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points according to the line-field synchronous time sequence of the target display screen 51, so as to form a stable display picture.
The clock module 6 of the present embodiment includes a crystal oscillator 61 and a phase locked loop 62. The phase locked loop 62 divides and multiplies the 100M differential clock source provided by the crystal oscillator 61 to supply a reference clock source to the remaining blocks.
Referring to fig. 2, this embodiment works as follows:
after the device is electrified, the device is initialized, and the camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6 start to work normally. The phase locked loop 62 begins to divide and multiply the 100M differential clock source provided by the crystal oscillator 61, providing a reference clock source for the remaining blocks. The video stream output module 5 reads the DDC of the display screen 51 and sets the target display resolution and orientation by the user through the matrix key 42.
When the CMOS camera 11 captures a video image, the video stream is transferred to the DVP interface or MIPI interface 12. After finishing decoding the video stream in the DVP format or the HDMI format, the camera decoding module 1 sends the decoded RGB565 video stream to the DDR3 input FIFO21 in the DDR3 storage read-write control module 2.
After receiving the RGB565 video stream, the DDR3 memory read write control module 2 stores the RGB565 data into the DDR3 input FIFO21 for buffering. At the same time, DDR3 input FIFO21 performs bit width conversion on input data, and 16bits are converted into 256bits, and then the data is transmitted to DDR3 chip 22. The DDR3 stores data read from the DDR3 chip 22 by the DDR3 output FIFO23 in the read/write control module 2. Meanwhile, the DDR3 output FIFO23 performs bit width conversion on data, and 256bits are converted into 16bits and supplied to a subsequent BRAM storage read-write control module 3.
The BRAM storage read-write control module 3 reads 2 lines of RGB565 data from the DDR3 output FIFO23 and stores the data into the first 2 lines of the BRAM31, and the BRAM31 is initialized. Because the Scaler algorithm processing module 4 needs to read two lines of data at the same time, when the Scaler algorithm processing module 4 reads the 1 st line data and the 2 nd line data, the BRAM31 completes filling the 3 rd line data; when the Scaler algorithm processing module 4 reads the 2 nd and 3 rd line data, the BRAM31 completes filling the 1 st line data; when the Scaler algorithm processing module 4 reads the 3 rd line data and the 1 st line data, the BRAM31 completes filling the 2 nd line data, so that ping-pong operation of 3 rd line data access is realized.
The Scaler algorithm processing module 4 implements the core algorithm bilinear interpolation algorithm of the present embodiment. The Scaler algorithm circuit 41 scans the key values of the matrix keys 42 in real time to acquire user input parameters. The Scaler algorithm circuit 41 extracts 4 required coordinate points from two lines of effective data of the BRAM31 each time, performs weighted averaging according to the requirement of user input parameters, calculates a pixel value of a target point, and writes the pixel value into the BRAM output FIFO32 in the BRAM storage read-write control module 3.
Finally, the video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points through the VGA or HDMI interface 52 according to the line-field synchronous time sequence of the display screen 51, so as to form a stably displayed picture.
The embodiment completes the self-adaptive adjustment function of the resolution of the real-time video stream and the resolution of the terminal display, can effectively amplify and reduce the resolution of the video stream by any multiple in real time, and stably and clearly displays the video stream at any target position of the terminal display screen.
Claims (6)
1. A video image resolution adaptive conversion device, characterized in that: the device comprises a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module, wherein the camera decoding module is connected with the DDR3 storage read-write control module and the clock module; the DDR3 storage read-write control module is connected with the camera decoding module, the BRAM storage read-write control module and the clock module; the BRAM storage read-write control module is connected with the DDR3 storage read-write control module, the Scaler algorithm processing module, the video stream output module and the clock module; the Scaler algorithm processing module is connected with the BRAM storage read-write control module and the clock module; the video stream output module is connected with the BRAM storage read-write control module and the clock module; the clock module is connected with the camera decoding module, the DDR3 storage read-write control module, the BRAM storage read-write control module, the Scaler algorithm processing module and the video stream output module; the camera decoding module comprises a CMOS camera, a DVP interface or an MIPI interface on an FPGA chip; the DDR3 storage read-write control module comprises a DDR3 chip, a DDR3 input FIFO in an FPGA chip and a DDR3 output FIFO in the FPGA chip; the BRAM storage read-write control module comprises a BRAM in the FPGA chip and a BRAM output FIFO in the FPGA chip; the Scaler algorithm processing module comprises a Scaler algorithm circuit and matrix keys in the FPGA chip; the video stream output module comprises a VGA interface or an HDMI interface on the display screen and the FPGA chip; the clock module comprises a crystal oscillator and a phase-locked loop in the FPGA chip; the DVP interface or MIPI interface, DDR3 input FIFO, DDR3 output FIFO, BRAM, BRAM output FIFO, scaler algorithm circuit, VGA interface or HDMI interface and phase-locked loop are all circuits inside the FPGA chip;
the Scaler algorithm circuit of the Scaler algorithm processing module is connected with the matrix keys, the BRAM output FIFO and the phase-locked loop; the Scaler algorithm circuit realizes a Scaler algorithm and interpolates images;
the Scaler algorithm circuit reads matrix keys in real time to obtain parameters input by a user: displaying the starting abscissa X 0 Display of the initial ordinate Y 0 Display length L, display width H; when the data amount in the BRAM output FIFO is less than half, the BRAM output FIFO sends a calculation request to the Scaler algorithm circuit; after receiving the calculation request, the Scaler algorithm circuit starts to execute the algorithm, and firstly, the algorithm is executed according to the mapping formula of the display screen coordinates: (x, y) → (x a, y x B) to obtain a coordinate point (x) 0 ，y 0 ) Mapped coordinate point (x 0 *A，y 0 * B) The method comprises the steps of carrying out a first treatment on the surface of the Wherein A and B are the length and width of the video source resolution respectively; then according to the mapping formula of the video source coordinates: (u, v) → (u×l, v×h), find (u) satisfying the following set of inequalities 0 ，v 0 )：
Then, the Scaler algorithm circuit sequentially reads the coordinates (u) from the BRAM 0 ，v 0 )、(u 0 ，v 0 +1)、(u 0 +1，v 0 ) Sum (u) 0 +1，v 0 Four pixel values of +1): p (u) 0 ，v 0 )、P(u 0 ，v 0 +1)、P(u 0 +1，v 0 ) And P (u) 0 +1，v 0 +1), and then according to the bilinear interpolation formula:
P(x 0 ，y 0 )＝[P(u 0 ，v 0 )+P(u 0 ，v 0 +1)+P(u 0 +1，v 0 )+P(u 0 +1，v 0 +1)]/4
calculating a pixel value P (x) 0 ，y 0 ) Finally, the pixel value is stored in BRAM output FIFO, and the coordinate point to be solved is updated as (x) 0 +1，y 0 )。
2. A video image resolution adaptive conversion apparatus according to claim 1, wherein: and a DVP interface or an MIPI interface of the camera decoding module is connected with the CMOS camera, the phase-locked loop and the DDR3 input FIFO.
3. A video image resolution adaptive conversion apparatus according to claim 1, wherein: the DDR3 input FIFO of the DDR3 storage read-write control module is connected with the DDR3 chip, the DVP interface or the MIPI interface and the phase-locked loop; the DDR3 chip is connected with the DDR3 input FIFO and the DDR3 output FIFO; the DDR3 output FIFO is connected with the DDR3 chip, the BRAM and the phase-locked loop; the DDR3 memory read-write control module provides high-speed frame buffering.
4. A video image resolution adaptive conversion apparatus according to claim 1, wherein: the BRAM of the BRAM storage read-write control module is connected with the DDR3 output FIFO, the Scaler algorithm circuit and the phase-locked loop; the BRAM output FIFO is connected with the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop; the BRAM storage read-write control module realizes the secondary cache of three rows of pixel points and provides accurate original pixel points for the Scaler algorithm processing module.
5. A video image resolution adaptive conversion apparatus according to claim 1, wherein: and a VGA interface or an HDMI interface of the video stream output module is connected with the display screen, the BRAM output FIFO and the phase-locked loop.
6. A video image resolution adaptive conversion apparatus according to claim 1, wherein: the phase-locked loop of the clock module is connected with a crystal oscillator, a DVP interface or an MIPI interface, a DDR3 input FIFO, a DDR3 output FIFO, BRAM, BRAM output FIFO, a Scaler algorithm circuit and a VGA interface or an HDMI interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380813.2A CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380813.2A CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115002304A CN115002304A (en) | 2022-09-02 |
| CN115002304B true CN115002304B (en) | 2024-03-22 |
Family
ID=83023957
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210380813.2A Active CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115002304B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320248B (en) * | 2023-05-18 | 2023-08-29 | 深圳曦华科技有限公司 | Memory chip sharing system, method, related device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010090113A (en) * | 2000-03-23 | 2001-10-18 | 김효근 | Method and apparatus for obtaining 2-dimensional high resolution image |
| CN104717485A (en) * | 2015-03-26 | 2015-06-17 | 金陵科技学院 | VGA interface naked-eye 3D display system based on FPGA |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
| WO2018076402A1 (en) * | 2016-10-26 | 2018-05-03 | 昆山软龙格自动化技术有限公司 | Multi-frame caching dual-camera twin-test test card |
| CN109587421A (en) * | 2018-12-10 | 2019-04-05 | 南京威翔科技有限公司 | A kind of HD-SDI/3G-SDI transmitting-receiving and real-time picture-in-picture switch output processing method |
| WO2019103240A1 (en) * | 2017-11-21 | 2019-05-31 | (주)루먼텍 | System and method for multi-camera video distribution and combination through ip |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108924621B (en) * | 2018-07-12 | 2019-10-29 | 深圳创维－Rgb电子有限公司 | Display methods, device and television set, storage medium |
-
2022
- 2022-04-12 CN CN202210380813.2A patent/CN115002304B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010090113A (en) * | 2000-03-23 | 2001-10-18 | 김효근 | Method and apparatus for obtaining 2-dimensional high resolution image |
| CN104717485A (en) * | 2015-03-26 | 2015-06-17 | 金陵科技学院 | VGA interface naked-eye 3D display system based on FPGA |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
| WO2018076402A1 (en) * | 2016-10-26 | 2018-05-03 | 昆山软龙格自动化技术有限公司 | Multi-frame caching dual-camera twin-test test card |
| WO2019103240A1 (en) * | 2017-11-21 | 2019-05-31 | (주)루먼텍 | System and method for multi-camera video distribution and combination through ip |
| CN109587421A (en) * | 2018-12-10 | 2019-04-05 | 南京威翔科技有限公司 | A kind of HD-SDI/3G-SDI transmitting-receiving and real-time picture-in-picture switch output processing method |
Non-Patent Citations (10)
| Title |
|---|
| A novel control method for horizontal and vertical scaler in the arbitrary resolution LCD panel;Yuan-Hao Huang;IEEE;全文 * |
| Development and Optimization of the Tb3+/Ce3+ co-doped Gd2O3 Scintillation Glass Fiber Faceplate for Cold Neutron Microscope;Li Deyuan;IEEE;全文 * |
| 基于FPGA 的ARM 图像扩展显示;黄俊;《计算机技术与应用》;131-137 * |
| 基于ＦＰＧＡ的视频格式转换系统设计;刘一清;中国有线电视;31-35 * |
| 基于FPGA的视频源无缝切换系统设计;周家耀;张俊杰;李正璇;;电子测量技术(第15期);全文 * |
| 基于FPGA的非VESA标准视频信号转换显示系统的设计;赵钱;章鹏;曹允;窦亮;;光电子技术(第04期);全文 * |
| 基于双线性插值算法的缩放IP核设计;邹学瑜;刘昌禄;胡敬营;;计算技术与自动化(第01期);全文 * |
| 李蒙胜.高斯－拉普拉斯金字塔法重建多分辨率锥束ＣＴ 三维图像.核电子学与探测技术.2022,50-56. * |
| 视频缩放在FPGA中的应用和实现;张梁;王景存;梅镖;;电子技术应用(第06期);全文 * |
| 高速CMOS图像存储与实时显示系统设计;马林;李锦明;张虎威;侯天喜;降帅;;电子技术应用(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115002304A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8619085B2 (en) | Method and system for compressing tile lists used for 3D rendering | |
| CN104616243B (en) | A kind of efficient GPU 3 D videos fusion method for drafting | |
| CN101000755B (en) | Multi-screen display splicing controller | |
| US8738891B1 (en) | Methods and systems for command acceleration in a video processor via translation of scalar instructions into vector instructions | |
| US9471955B2 (en) | Multiple display pipelines driving a divided display | |
| US20110221743A1 (en) | Method And System For Controlling A 3D Processor Using A Control List In Memory | |
| CN110996170B (en) | Video file playing method and related equipment | |
| KR20070041507A (en) | Method and system for displaying a sequence of image frames | |
| US20090225224A1 (en) | System and method for processing a television picture-out-picture | |
| CN115002304B (en) | Video image resolution self-adaptive conversion device | |
| CN114302092A (en) | One-display frame insertion circuit, method, device, chip, electronic device and medium | |
| US7876996B1 (en) | Method and system for time-shifting video | |
| US10672367B2 (en) | Providing data to a display in data processing systems | |
| CN200983644Y (en) | Multi-screen display and combination control device | |
| CN114613306A (en) | Display control chip, display panel and related equipment, method and device | |
| KR20070063580A (en) | Video processing | |
| EP1141892B1 (en) | Method and apparatus for stretch blitting using a 3d pipeline processor | |
| Berekovic et al. | Architecture of a coprocessor module for image compositing | |
| JP2003348447A (en) | Image output apparatus | |
| JP2012028997A (en) | Image processing device and camera | |
| CN115511709A (en) | Self-adaptive super-resolution image sampling system | |
| US8704834B2 (en) | Synchronization of video input data streams and video output data streams | |
| JP4735572B2 (en) | Image data encoding apparatus, image data decoding apparatus, image processing apparatus, and electronic apparatus | |
| JP5157419B2 (en) | Image processing apparatus, image processing method, and electronic apparatus | |
| JP2005322233A (en) | Memory efficient method and apparatus for compression encoding large overlaid camera image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |