CN115002304A - Video image resolution self-adaptive conversion device - Google Patents
Video image resolution self-adaptive conversion device Download PDFInfo
- Publication number
- CN115002304A CN115002304A CN202210380813.2A CN202210380813A CN115002304A CN 115002304 A CN115002304 A CN 115002304A CN 202210380813 A CN202210380813 A CN 202210380813A CN 115002304 A CN115002304 A CN 115002304A
- Authority
- CN
- China
- Prior art keywords
- ddr3
- module
- bram
- write control
- control module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 12
- 238000003860 storage Methods 0.000 claims abstract description 80
- 238000012545 processing Methods 0.000 claims abstract description 63
- 239000011159 matrix material Substances 0.000 claims description 12
- 239000013078 crystal Substances 0.000 claims description 7
- 230000003139 buffering effect Effects 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims 3
- 238000005516 engineering process Methods 0.000 description 6
- 238000000034 method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 239000000872 buffer Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/907—Television signal recording using static stores, e.g. storage tubes or semiconductor memories
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a video image resolution self-adaptive conversion device, which comprises: the device comprises a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module. The invention aims to solve the problem that the resolution of a real-time video stream and the resolution of a terminal display are difficult to adaptively convert when a CMOS camera is used. The invention provides a CMOS camera DVP interface, a CMOS camera MIPI interface, a VGA video output interface, an HDMI video output interface, 1080P high-definition video stream display and DDR3 high-capacity video data frame storage, is suitable for centralized access, and provides a brand-new solution for the resolution real-time adjustment of the CMOS camera video stream and the on-screen display of multiple paths of video streams.
Description
Technical Field
The invention relates to the fields of image video acquisition technology, image coding and decoding technology, digital image processing analysis and the like. In particular to a video image processing device which is used in a multi-terminal display screen scene and adapts and scales the display resolution of a video stream in real time and adjusts the display direction so as to achieve the optimal display effect.
Technical Field
Video images are the most direct means for human beings to observe and perceive the world, and the processing technology of video images has quite wide application in various fields such as industrial production, public transportation, aerospace, clinical medicine and the like. The video image processing technology greatly improves the expressive force of pictures and images, and particularly people enlarge and reduce the pictures by using an image interpolation algorithm so as to obtain richer detail information in the pictures and the images.
In the traditional computer-based video image processing technology, a Central Processing Unit (CPU) is used as a central processing unit, and a digital image processing system is built by means of computer software and an operating system. People complete program development by means of computer languages C, C + + and the like, and can complete interaction with underlying hardware to realize image processing. So far, most of the image processing algorithms in the scene are realized by the system. Although the performance of computer CPU is getting stronger and the processing speed is getting faster and faster, it is still limited by the principle of serial processing mode of single instruction and single data in nature, and the computer needs to run the operating system, so its application field is very limited, and it is not suitable for the scenario with high real-time requirement.
In recent years, with rapid progress and development of technologies such as image acquisition and video processing, the resolution, storage size, and processing speed of images are increasing, and the requirements for hardware size of image processing systems are decreasing and the requirements for processing real-time performance are increasing. This presents more serious challenges to conventional computer software-based image processing platforms. Therefore, it is necessary to jump from the traditional processing method based on a computer software platform and find another processing platform and method that can meet new requirements.
Disclosure of Invention
The invention aims to solve the problem of self-adaptive optimization of the resolution of a real-time video stream and the resolution of a terminal display when a CMOS camera is used in various terminal display screen scenes, and provides a video image resolution self-adaptive conversion device which can well realize multi-screen display of an aircraft cockpit, multi-screen display of one screen of a mobile phone camera, multi-screen monitoring of a factory production line, multi-azimuth real-time monitoring of public transportation and the like in various application scenes, can amplify and reduce the resolution of the video stream by any multiple in real time and high efficiently according to requirements, and can stably and clearly display the video stream at any target position of the terminal display screen.
The specific technical scheme for realizing the purpose of the invention is as follows:
a video image resolution self-adaptive conversion device is characterized by comprising a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module, wherein the camera decoding module is connected with the DDR3 storage read-write control module and the clock module; the DDR3 storage read-write control module is connected with the camera decoding module, the BRAM storage read-write control module and the clock module; the BRAM storage read-write control module is connected with the DDR3 storage read-write control module, the Scaler algorithm processing module, the video stream output module and the clock module; the Scaler algorithm processing module is connected with the BRAM storage read-write control module and the clock module; the video stream output module is connected with the BRAM storage read-write control module and the clock module; the clock module is connected with the camera decoding module, the DDR3 storage read-write control module, the BRAM storage read-write control module, the Scaler algorithm processing module and the video stream output module; the camera decoding module comprises a CMOS camera and a DVP interface or an MIPI interface on an FPGA chip; the DDR3 storage read-write control module comprises a DDR3 chip, a DDR3 input FIFO in the FPGA chip and a DDR3 output FIFO in the FPGA chip; the BRAM storage read-write control module comprises a BRAM in the FPGA chip and a BRAM output FIFO in the FPGA chip; the Scaler algorithm processing module comprises a Scaler algorithm circuit and a matrix key in an FPGA chip; the video stream output module comprises a display screen and a VGA interface or HDMI interface on the FPGA chip; the clock module comprises a crystal oscillator and a phase-locked loop in the FPGA chip; the DVP interface or MIPI interface, the DDR3 input FIFO, the DDR3 output FIFO, the BRAM output FIFO, the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop are all circuits inside the FPGA chip.
And a DVP interface or an MIPI interface of the camera decoding module is connected with the CMOS camera, the phase-locked loop and the DDR3 input FIFO.
The DDR3 storage read-write control module is characterized in that a DDR3 input FIFO is connected with a DDR3 chip, a DVP interface or an MIPI interface and a phase-locked loop; the DDR3 chip is connected with the DDR3 input FIFO and the DDR3 output FIFO; the DDR3 output FIFO is connected with a DDR3 chip, a BRAM and a phase-locked loop; the DDR3 storage read-write control module provides high-speed frame buffering.
The BRAM of the BRAM storage read-write control module is connected with the DDR3 output FIFO, the Scaler algorithm circuit and the phase-locked loop; the BRAM output FIFO is connected with the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop; the BRAM storage read-write control module realizes the secondary cache of three rows of pixel points and provides accurate original pixel points for the Scaler algorithm processing module.
The Scaler algorithm circuit of the Scaler algorithm processing module is connected with the matrix key, the BRAM output FIFO and the phase-locked loop; the Scaler algorithm circuit in the module realizes the innovative Scaler algorithm of the invention to interpolate the image.
And a VGA interface or HDMI interface of the video stream output module is connected with the display screen, the BRAM output FIFO and the phase-locked loop.
The phase-locked loop of the clock module is connected with the crystal oscillator, the DVP interface or the MIPI interface, the DDR3 input FIFO, the DDR3 output FIFO, the BRAM output FIFO, the Scaler algorithm circuit and the VGA interface or the HDMI interface.
Compared with the prior art, the invention has the beneficial effects that:
(1) the FPGA is used as a main video image data processing chip, tasks can be executed in a parallel mode, and the method has shorter processing time and stronger task responsiveness. And the internal hardware modules of the FPGA can be customized very conveniently and multiplexed.
(2) The invention innovatively optimizes the bilinear interpolation algorithm, totally optimizes floating point operations related to multiplication and division in the traditional algorithm into unsigned integer multiplication, innovatively adapts the algorithm to the operation mode of the FPGA, greatly plays the high speed of data processing of the FPGA while ensuring the best effect of the algorithm, and consumes less logic units in the FPGA.
(3) The DDR3 chip with high reading and writing speed and large storage capacity is used as the cache chip, the capacity of video image data cache is improved, the acquisition capacity of the video image data is greatly improved, more dense and more complete video image frames can be cached, and effective information extraction can be performed on the video image frames in time. The video image data throughput in the overall function is extremely strong.
(4) The invention adopts the solution of integrating picture information acquisition, storage control and display control, and can finish the processes of extracting video picture information from the CMOS camera, storing and controlling video image frames and displaying and controlling a display screen of a terminal through an independent module. The topological structure of the scheme and the implementation way are simplified.
Drawings
FIG. 1 is a block diagram of the apparatus of the present invention;
FIG. 2 is a flow chart of the present invention.
Detailed Description
The invention is described in detail below with reference to the drawings and examples.
Referring to fig. 1, the invention includes a camera decoding module 1, a DDR3 storage read-write control module 2, a BRAM storage read-write control module 3, a Scaler algorithm processing module 4, a video stream output module 5 and a clock module 6; the camera decoding module 1 is connected with the DDR3 storage read-write control module 2 and the clock module 6 and is used for receiving and decoding video stream data of the CMOS camera; the DDR3 storage read-write control module 2 is connected with the camera decoding module 1, the BRAM storage read-write control module 3 and the clock module 6, and is used for performing frame caching on an input video stream, providing pixel point information for a subsequent algorithm, and ensuring the integrity of the video stream in the transmission process without data loss; the BRAM storage read-write control module 3 is connected with the DDR3 storage read-write control module 2, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6, and is used for caching three lines of video data in a ping-pong cache manner, so that the Scaler algorithm processing module 4 can read pixel points needing to be calculated at high speed at any time; the Scaler algorithm processing module 4 is connected with the BRAM storage read-write control module 3 and the clock module 6 and is used for reading parameters input by a user through keys, completing a core Scaler algorithm and adjusting target display resolution in real time; the video stream output module 5 is connected with the BRAM storage read-write control module 3 and the clock module 6, and is used for reading the DDC of the terminal display equipment and displaying the image processed by the algorithm; the clock module 6 is connected with the camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4 and the video stream output module 5, and is used for providing a reference clock for each module.
The camera decoding module 1 comprises a DVP interface or an HDMI interface 12 and a CMOS camera 11 on the FPGA chip, and the DVP interface or the HDMI interface 12 is connected to the CMOS camera 11. The camera decoding module 1 completes decoding and data splicing integration of the input video stream of the CMOS camera to obtain a standard RGB video stream time sequence with an RGB565 data format and a line-field synchronous signal.
The DDR3 storage read-write control module 2 comprises a DDR3 chip 22, a DDR3 input FIFO21 in the FPGA chip and a DDR3 output FIFO23 in the FPGA chip, wherein the DDR3 input FIFO21 is connected with the DDR3 chip 22, and the DDR3 output FIFO23 is connected with the DDR3 chip 21. The module drives the DDR3 chip, and ensures that the time sequence of signals of an address bus and a data bus is accurate and correct when the DDR3 chip reads and writes. The internal space of the DDR3 chip is divided into 256 parts, each part has the size of 32Mbits space, and each part is used for storing 1 frame picture.
The BRAM storage read-write control module 3 comprises a BRAM31 in an FPGA chip and a BRAM output FIFO32 in the FPGA chip. The module controls the DDR3 storage read-write control module to read 3 rows of pixel cache to the BRAM, the BRAM caches 3 rows of pixel data, and a ping-pong read function is provided for the Scaler algorithm processing module 4. And simultaneously, the module controls the pixel value to be output read from the Scaler algorithm processing module 4 and stored in the BRAM output FIFO 32.
The Scaler algorithm processing module 4 comprises a Scaler algorithm circuit 41 and a matrix key 42 in the FPGA chip, and the Scaler algorithm circuit 41 is connected with the matrix key 42. The module reads parameters input by a user through keys and adjusts the resolution and the display direction of a picture on a display screen in real time. The Scaler algorithm circuit 41 employs a bilinear interpolation algorithm. And performing linear weighted summation on the four pixel points read from the BRAM storage read-write control module 3 to obtain the value of a target pixel point, and writing the value into a BRAM output FIFO32 for the video stream output module 5 to read and display.
The video stream output module 5 includes a display screen 51, a VGA interface on the FPGA chip or an HDMI interface 52 on the FPGA chip, and the display screen 51 is connected to the VGA interface or the HDMI interface 52. The module reads DDC of the terminal display screen and determines the optimal display resolution of the display screen. And reading pixel points processed by the algorithm from the BRAM output FIFO32, and outputting pixel values of corresponding coordinate points according to the line-field synchronous time sequence of the display screen to form a stably displayed picture.
The core algorithm of the present invention is a bilinear interpolation algorithm adopted by the Scaler algorithm circuit 41. The Scaler algorithm circuit 41 reads the matrix key 42 in real time to obtain the parameters input by the user: display the starting abscissa X 0 Showing the starting ordinate Y 0 Display length L, display width H. When the amount of data in the BRAM output FIFO32 is less than half, the BRAM output FIFO32 issues a calculation request to the Scaler algorithm circuit 41. After receiving the calculation request, the Scaler algorithm circuit 41 starts to execute the algorithm, and according to the mapping formula of the display screen coordinates: (x, y) → (x a, y B), and obtaining a coordinate point (x) to be obtained 0 ，y 0 ) Mapped coordinate point (x) 0 *A，y 0 B). Wherein, A and B are the length and width of the video source resolution, respectively. Then according to the mapping formula of the video source coordinates: (u, v) → (u x L, v x H), finding (u) satisfying the following inequality group 0 ，v 0 )：
The Scaler algorithm circuit 41 then reads the coordinates of (u) from the BRAM31 in sequence 0 ，v 0 )、(u 0 ，v 0 +1)、(u 0 +1，v 0 ) And (u) 0 +1，v 0 Four pixel values of + 1): p (u) 0 ，v 0 )、P(u 0 ，v 0 +1)、P(u 0 +1，v 0 ) And P (u) 0 +1，v 0 +1), and then according to the bilinear interpolation formula:
P(x 0 ，y 0 )＝[P(u 0 ，v 0 )+P(u 0 ，v 0 +1)+P(u 0 +1，v 0 )+P(u 0 +1，v 0 +1)]the pixel value P (x) of the target point is obtained through calculation 0 ，y 0 ) Finally, the pixel value is stored in the BRAM output FIFO32, and the coordinate point to be calculated is updated to be (x) 0 +1，y 0 )。
The working process of the invention is as follows:
after the FPGA development platform is powered on, initialization is carried out, the DDC of the terminal display equipment is read, and a user sets the target display resolution and the target display direction through keys. The camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6 start to work normally.
When the CMOS camera 11 acquires a video image, the video stream is transmitted to the DVP interface or HDMI interface 12. The camera decoding module 1 decodes the DVP format or HDMI format video stream, converts the video stream into RGB565 color signals and line-field synchronization timing signals HS and VS, calculates display coordinate points x and y corresponding to RGB colors and RGB effective signals DE according to the line-field synchronization signals, and sends the display coordinate points x and y to the DDR3 input FIFO21 in the DDR3 storage read-write control module 2.
After the DDR3 storage read-write control module 2 receives RGB565, HS, VS, DE, x, and y signals, the RGB565 data is stored in the asynchronous DDR3 input FIFO21 for buffering. The DDR3 input FIFO21 has a write clock frequency of 24MHz, a read clock frequency of 100Mhz, and a memory space of 8kbits, and can store 512 RGB565 data with 16 bits. Meanwhile, the DDR3 input FIFO21 also carries out bit width conversion on input data, 1 RGB565 data with 16bits is written into the FIFO every 1 clock cycle at the input end, 16 data is read out once every 1 clock cycle at the output end, 1 data with 256bits is spliced and transmitted to the DDR3 chip 22, and the DDR3 chip 22 writes or reads 256bits of data once.
The DDR3 storage read-write control module 2 continues to complete read-write state control on the DDR3 chip 22, the whole DDR3 state machine works at the frequency of 100MHz, and the read-write throughput of the DDR3 chip 22 is up to 3.2 GBps. The internal total space 1GBytes of the DDR3 chip 22 is divided into 256 pieces, each piece having a size of 4MBytes space, and each piece is used for storing 1 frame of pictures. The jump process of the DDR3 state machine is as follows: the DDR3 chip 22 reads a complete frame of picture from the DDR3 input FIFO21 and stores the picture in the first address space, and then reads data from the first address space and stores the data in the DDR3 output FIFO23 in sequence until the data amount in the DDR3 output FIFO23 reaches half, the initialization of the state machine is completed, and the state machine enters an idle mode. Then, every time the data in the DDR3 output FIFO23 is less than half, the DDR3 chip 22 enters the read mode immediately, and the data amount of the DDR3 output FIFO23 is read by half from the last read address and stored in the DDR3 output FIFO 23. When new data can be read from the DDR3 input FIFO21, the DDR3 chip 22 enters a data writing mode, reads the new data from the DDR3 input FIFO21, and then jumps to the next address space to continue writing the new data when the last written address or the last time just finishes writing a complete image frame. Because the DDR3 chip 22 cannot read and write simultaneously, the read state is prioritized over the write state in the state machine design, and the read address is always one address space behind the write address.
The DDR3 stores the DDR3 output FIFO23 in the read-write control module 2 for buffering the data read out from the DDR3 chip 22. The read-write clock frequency of the DDR3 output FIFO23 is 100Mhz, the storage space is 8kbits, and 32 DDR3 chips 22 with 256bits can be stored to output data. Meanwhile, the DDR3 output FIFO23 also carries out bit width conversion on the data, 1 DDR3 data with 256bits is written into the FIFO at the input end every 1 clock cycle DDR3, the 256 DDR3 data is split at the output end according to the sequence from high bit to low bit, 1 RGB565 data with 16bits is read out every 1 clock cycle, and the data is supplied to a subsequent BRAM storage read-write control module 3.
The size of the BRAM31 in the BRAM storage read-write control module 3 is 37.5kbits and is used for storing exactly 3 lines of RGB565 data. The module reads 2 lines of RGB565 data from DDR3 output FIFO23 and stores the data in the first 2 lines of BRAM31, and the initialization of BRAM is completed. Because the Scaler algorithm processing module 4 needs to read two lines of data at the same time, when the Scaler algorithm processing module 4 reads the 1 st and 2 nd data, the BRAM31 completes the filling of the 3 rd data; when the Scaler algorithm processing module 4 reads the data of the 2 nd and 3 rd rows, the BRAM31 completes the filling of the data of the 1 st row; when the Scaler algorithm processing module 4 reads the data of the 3 rd and 1 st rows, the BRAM31 completes the filling of the data of the 2 nd row, so as to realize the ping-pong operation of accessing the data of the 3 nd row.
The Scaler algorithm processing module 4 realizes the core algorithm bilinear interpolation algorithm of the invention. The module first reads the parameters entered by the user via the matrix key 42: l, H, X0 and Y0. Since the resolution of the input image is 800 × 480, when the resolution of the target output image is L × H and the pixel value with the coordinate (x, y) needs to be obtained, the weighted average of the pixel values of the four original coordinate points (x × 800/L, y × 480/H), (x × 800/L + 1, y 480/H), (x × 800/L, y × 480/H +1), (x 800/L + 1, y × 480/H +1) is needed, so the Scaler algorithm processing module 4 takes out 4 required coordinate points from two rows of valid data of the BRAM31 at the operating frequency of 400Mhz each time, performs the weighted average, calculates the pixel value of the target point, and outputs the pixel value to the BRAM output FIFO 32.
Finally, the video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points through the VGA or HDMI interface 52 according to the line-field synchronous timing sequence of the display screen 51, thereby forming a stably displayed picture.
Examples
Referring to fig. 1, the present embodiment includes: the device comprises a camera decoding module 1, a DDR3 storage read-write control module 2, a BRAM storage read-write control module 3, a Scaler algorithm processing module 4, a video stream output module 5 and a clock module 6. The DVP interface or MIPI interface 12, the DDR3 input FIFO21, the DDR3 output FIFO23, the BRAM31, the BRAM output FIFO32, the Scaler algorithm circuit 41, the VGA or HDMI interface 52, and the phase-locked loop 62 are all circuits designed by using internal logic resources of the FPGA chip 7.
The camera decoding module 1 is connected with the DDR3 storage read-write control module 2 and the clock module 6.
The DDR3 storage read-write control module 2 is connected with the camera decoding module 1, the BRAM storage read-write control module 3 and the clock module 6.
The BRAM storage read-write control module 3 is connected with the DDR3 storage read-write control module 2, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6.
The Scaler algorithm processing module 4 is connected with the BRAM storage read-write control module 3 and the clock module 6.
The video stream output module 5 is connected with the BRAM storage read-write control module 3 and the clock module 6.
The clock module 6 is connected with the camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4 and the video stream output module 5.
The camera decoding module 1 of the present embodiment includes a DVP interface or a MIPI interface 12 connected to the CMOS camera 11. The camera decoding module 1 completes decoding and data splicing integration of the input video stream of the CMOS camera 11 to obtain a standard RGB video stream timing sequence having an RGB565 data format and a line-field synchronization signal.
The DDR3 storage read-write control module 2 of the embodiment of the present invention includes a DDR3 input FIFO21, a DDR3 chip 22, and a DDR3 output FIFO 23. The DDR3 chip 22 is connected to the DDR3 input FIFO21 and the DDR3 output FIFO 23. The internal space of the DDR3 chip 22 is divided into 256 shares, each having a size of 32Mbits of space, each for storing 1 frame of picture.
The BRAM memory read-write control module 3 of the embodiment includes a BRAM31 and a BRAM output buffer FIFO 32. The BRAM memory read-write control module 3 reads three lines of pixel buffers from the DDR3 output FIFO23 to the BRAM31, and the BRAM31 buffers three lines of pixel data in total, so that a ping-pong read function is provided for the Scaler algorithm processing module 4. Meanwhile, the BRAM memory read-write control module 3 reads the pixel value to be output from the Scaler algorithm circuit 41 and writes the pixel value into the BRAM output FIFO 32.
The Scaler algorithm processing module 4 of the present embodiment includes a Scaler algorithm circuit 41 and a matrix key 42. The Scaler algorithm circuit 41 scans the key values of the matrix keys 42 in real time to obtain the user input parameters. And meanwhile, linear weighted summation is carried out on the four pixel points read from the BRAM31 by adopting a bilinear interpolation algorithm, the value of a target pixel point is solved, and the value is written into the BRAM output FIFO32 for the video stream output module 5 to read and display.
The video stream output module 5 of the present embodiment includes a VGA or HDMI interface 52 and a display screen 51, and the display screen 51 is connected to the VGA or HDMI interface 52. The video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points according to the line-field synchronization timing sequence of the target display screen 51, so as to form a stably displayed picture.
The clock module 6 of the present embodiment includes a crystal oscillator 61 and a phase-locked loop 62. The phase-locked loop 62 frequency-divides and frequency-multiplies the 100M differential clock source provided by the crystal oscillator 61, and supplies reference clock sources to the other modules.
Referring to fig. 2, the present embodiment works as follows:
after the device is powered on, initialization is carried out, and the camera decoding module 1, the DDR3 storage read-write control module 2, the BRAM storage read-write control module 3, the Scaler algorithm processing module 4, the video stream output module 5 and the clock module 6 start to work normally. The phase-locked loop 62 starts to frequency-divide and frequency-multiply the 100M differential clock source provided by the crystal oscillator 61, and supplies a reference clock source to the rest modules. The video stream output module 5 reads the DDC of the display screen 51 and sets the target display resolution and orientation by the user through the matrix key 42.
When the CMOS camera 11 acquires a video image, the video stream is transmitted to the DVP interface or the MIPI interface 12. After the camera decoding module 1 completes decoding of the DVP format or HDMI format video stream, the decoded RGB565 video stream is sent to the DDR3 storage read-write control module 2 in the DDR3 input FIFO 21.
After the DDR3 storage read-write control module 2 receives the RGB565 video stream, the RGB565 data is stored in the DDR3 input FIFO21 for buffering. Meanwhile, the DDR3 input FIFO21 carries out bit width conversion on input data, 16bits are converted into 256bits, and then the converted data are transmitted to the DDR3 chip 22. The DDR3 stores data read from the DDR3 chip 22 by the DDR3 output FIFO23 in the read-write control module 2. Meanwhile, the DDR3 output FIFO23 carries out bit width conversion on the data, and 256bits are converted into 16bits, and the data are supplied to the subsequent BRAM storage read-write control module 3.
The BRAM memory read-write control module 3 reads 2 lines of RGB565 data from the DDR3 output FIFO23 and stores the data into the previous 2 lines of BRAM31, and the initialization of BRAM31 is completed. Because the Scaler algorithm processing module 4 needs to read two lines of data at the same time, when the Scaler algorithm processing module 4 reads the 1 st and 2 nd line of data, the BRAM31 completes the filling of the 3 rd line of data; when the Scaler algorithm processing module 4 reads the 2 nd and 3 rd row data, the BRAM31 completes the filling of the 1 st row data; when the Scaler algorithm processing module 4 reads the 3 rd and 1 st line data, the BRAM31 completes the filling of the 2 nd line data, thereby realizing the ping-pong operation of 3 line data access.
The Scaler algorithm processing module 4 implements the core algorithm bilinear interpolation algorithm of the present embodiment. The Scaler algorithm circuit 41 scans the key values of the matrix keys 42 in real time to obtain the user input parameters. The Scaler algorithm circuit 41 takes out 4 needed coordinate points from two rows of effective data of the BRAM31 each time, carries out weighting and averaging according to the requirement of user input parameters, calculates the pixel value of a target point, and writes the pixel value into the BRAM output FIFO32 in the BRAM storage read-write control module 3.
Finally, the video stream output module 5 reads the pixel points processed by the algorithm from the BRAM output FIFO32, and outputs the pixel values of the corresponding coordinate points through the VGA or HDMI interface 52 according to the line-field synchronous timing sequence of the display screen 51, thereby forming a stably displayed picture.
Therefore, the embodiment completes the function of adaptively adjusting the resolution of the real-time video stream and the resolution of the terminal display, can effectively amplify and reduce the resolution of the video stream by any multiple in real time, and can stably and clearly display the video stream at any target position of the terminal display screen.
Claims (7)
1. An apparatus for adaptive resolution conversion of video images, comprising: the device comprises a camera decoding module, a DDR3 storage read-write control module, a BRAM storage read-write control module, a Scaler algorithm processing module, a video stream output module and a clock module, wherein the camera decoding module is connected with the DDR3 storage read-write control module and the clock module; the DDR3 storage read-write control module is connected with the camera decoding module, the BRAM storage read-write control module and the clock module; the BRAM storage read-write control module is connected with the DDR3 storage read-write control module, the Scaler algorithm processing module, the video stream output module and the clock module; the Scaler algorithm processing module is connected with the BRAM storage read-write control module and the clock module; the video stream output module is connected with the BRAM storage read-write control module and the clock module; the clock module is connected with the camera decoding module, the DDR3 storage read-write control module, the BRAM storage read-write control module, the Scaler algorithm processing module and the video stream output module; the camera decoding module comprises a CMOS camera and a DVP interface or an MIPI interface on an FPGA chip; the DDR3 storage read-write control module comprises a DDR3 chip, a DDR3 input FIFO in the FPGA chip and a DDR3 output FIFO in the FPGA chip; the BRAM storage read-write control module comprises a BRAM in the FPGA chip and a BRAM output FIFO in the FPGA chip; the Scaler algorithm processing module comprises a Scaler algorithm circuit and a matrix key in the FPGA chip; the video stream output module comprises a display screen and a VGA interface or HDMI interface on the FPGA chip; the clock module comprises a crystal oscillator and a phase-locked loop in an FPGA chip; the DVP interface or the MIPI interface, the DDR3 input FIFO, the DDR3 output FIFO, the BRAM output FIFO, the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop are all circuits inside the FPGA chip.
2. A video image resolution adaptive conversion apparatus according to claim 1, characterized in that: and a DVP interface or an MIPI interface of the camera decoding module is connected with the CMOS camera, the phase-locked loop and the DDR3 input FIFO.
3. The apparatus according to claim 1, wherein: the DDR3 storage read-write control module DDR3 input FIFO is connected with a DDR3 chip, a DVP interface or an MIPI interface and a phase-locked loop; the DDR3 chip is connected with the DDR3 input FIFO and the DDR3 output FIFO; the DDR3 output FIFO is connected with a DDR3 chip, a BRAM and a phase-locked loop; the DDR3 memory read-write control module provides high-speed frame buffering.
4. The apparatus according to claim 1, wherein: the BRAM of the BRAM storage read-write control module is connected with the DDR3 output FIFO, the Scaler algorithm circuit and the phase-locked loop; the BRAM output FIFO is connected with the Scaler algorithm circuit, the VGA interface or the HDMI interface and the phase-locked loop; the BRAM storage read-write control module realizes the secondary cache of three rows of pixel points and provides accurate original pixel points for the Scaler algorithm processing module.
5. The apparatus according to claim 1, wherein: the Scaler algorithm circuit of the Scaler algorithm processing module is connected with the matrix key, the BRAM output FIFO and the phase-locked loop; the Scaler algorithm circuit realizes a Scaler algorithm to perform interpolation of images.
6. The apparatus according to claim 1, wherein: and a VGA interface or HDMI interface of the video stream output module is connected with the display screen, the BRAM output FIFO and the phase-locked loop.
7. A video image resolution adaptive conversion apparatus according to claim 1, characterized in that: the phase-locked loop of the clock module is connected with the crystal oscillator, the DVP interface or the MIPI interface, the DDR3 input FIFO, the DDR3 output FIFO, the BRAM output FIFO, the Scaler algorithm circuit and the VGA interface or the HDMI interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380813.2A CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210380813.2A CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115002304A true CN115002304A (en) | 2022-09-02 |
| CN115002304B CN115002304B (en) | 2024-03-22 |
Family
ID=83023957
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210380813.2A Active CN115002304B (en) | 2022-04-12 | 2022-04-12 | Video image resolution self-adaptive conversion device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115002304B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320248A (en) * | 2023-05-18 | 2023-06-23 | 深圳曦华科技有限公司 | Memory chip sharing system, method, related device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010090113A (en) * | 2000-03-23 | 2001-10-18 | 김효근 | Method and apparatus for obtaining 2-dimensional high resolution image |
| CN104717485A (en) * | 2015-03-26 | 2015-06-17 | 金陵科技学院 | VGA interface naked-eye 3D display system based on FPGA |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
| WO2018076402A1 (en) * | 2016-10-26 | 2018-05-03 | 昆山软龙格自动化技术有限公司 | Multi-frame caching dual-camera twin-test test card |
| CN109587421A (en) * | 2018-12-10 | 2019-04-05 | 南京威翔科技有限公司 | A kind of HD-SDI/3G-SDI transmitting-receiving and real-time picture-in-picture switch output processing method |
| WO2019103240A1 (en) * | 2017-11-21 | 2019-05-31 | (주)루먼텍 | System and method for multi-camera video distribution and combination through ip |
| US20210044776A1 (en) * | 2018-07-12 | 2021-02-11 | Shenzhen Skyworth-Rgb Electronic Co., Ltd. | Display method and display device, television and storage medium |
-
2022
- 2022-04-12 CN CN202210380813.2A patent/CN115002304B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010090113A (en) * | 2000-03-23 | 2001-10-18 | 김효근 | Method and apparatus for obtaining 2-dimensional high resolution image |
| CN104717485A (en) * | 2015-03-26 | 2015-06-17 | 金陵科技学院 | VGA interface naked-eye 3D display system based on FPGA |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
| WO2018076402A1 (en) * | 2016-10-26 | 2018-05-03 | 昆山软龙格自动化技术有限公司 | Multi-frame caching dual-camera twin-test test card |
| WO2019103240A1 (en) * | 2017-11-21 | 2019-05-31 | (주)루먼텍 | System and method for multi-camera video distribution and combination through ip |
| US20210044776A1 (en) * | 2018-07-12 | 2021-02-11 | Shenzhen Skyworth-Rgb Electronic Co., Ltd. | Display method and display device, television and storage medium |
| CN109587421A (en) * | 2018-12-10 | 2019-04-05 | 南京威翔科技有限公司 | A kind of HD-SDI/3G-SDI transmitting-receiving and real-time picture-in-picture switch output processing method |
Non-Patent Citations (10)
| Title |
|---|
| LI DEYUAN: "Development and Optimization of the Tb3+/Ce3+ co-doped Gd2O3 Scintillation Glass Fiber Faceplate for Cold Neutron Microscope", IEEE * |
| YUAN-HAO HUANG: "A novel control method for horizontal and vertical scaler in the arbitrary resolution LCD panel", IEEE * |
| 刘一清: "基于ＦＰＧＡ的视频格式转换系统设计", 中国有线电视, pages 31 - 35 * |
| 周家耀;张俊杰;李正璇;: "基于FPGA的视频源无缝切换系统设计", 电子测量技术, no. 15 * |
| 张梁;王景存;梅镖;: "视频缩放在FPGA中的应用和实现", 电子技术应用, no. 06 * |
| 李蒙胜: "高斯－拉普拉斯金字塔法 重建多分辨率锥束ＣＴ 三维图像", 核电子学与探测技术, pages 50 - 56 * |
| 赵钱;章鹏;曹允;窦亮;: "基于FPGA的非VESA标准视频信号转换显示系统的设计", 光电子技术, no. 04 * |
| 邹学瑜;刘昌禄;胡敬营;: "基于双线性插值算法的缩放IP核设计", 计算技术与自动化, no. 01 * |
| 马林;李锦明;张虎威;侯天喜;降帅;: "高速CMOS图像存储与实时显示系统设计", 电子技术应用, no. 06 * |
| 黄俊: "基于FPGA 的ARM 图像扩展显示", 《计算机技术与应用》, pages 131 - 137 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320248A (en) * | 2023-05-18 | 2023-06-23 | 深圳曦华科技有限公司 | Memory chip sharing system, method, related device and storage medium |
| CN116320248B (en) * | 2023-05-18 | 2023-08-29 | 深圳曦华科技有限公司 | Memory chip sharing system, method, related device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115002304B (en) | 2024-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10282805B2 (en) | Image signal processor and devices including the same | |
| EP3134804B1 (en) | Multiple display pipelines driving a divided display | |
| CN101000755B (en) | Multi-screen display splicing controller | |
| JP2008506295A (en) | Method and system for displaying a series of image frames | |
| EP2274908B1 (en) | Video multiviewer system using direct memory access (dma) registers and multi ported block ram and related method | |
| US20070024524A1 (en) | Preventing image tearing where a single video input is streamed to two independent display devices | |
| US8259233B2 (en) | System and method for processing a television picture-out-picture | |
| US20060133695A1 (en) | Display controller, electronic instrument, and image data supply method | |
| CN112862659B (en) | Method and device for generating a series of frames by means of a synthesizer | |
| US20200005719A1 (en) | Data processing systems | |
| CN109587421B (en) | HD-SDI/3G-SDI transceiving and real-time picture-in-picture switching output processing method | |
| US10672367B2 (en) | Providing data to a display in data processing systems | |
| US9239697B2 (en) | Display multiplier providing independent pixel resolutions | |
| CN115002304B (en) | Video image resolution self-adaptive conversion device | |
| CN200983644Y (en) | Multi-screen display and combination control device | |
| CN102625086B (en) | DDR2 (Double Data Rate 2) storage method and system for high-definition digital matrix | |
| CN101127847A (en) | A screen display synthesis method and synthesis device | |
| JP2012028997A (en) | Image processing device and camera | |
| CN101257588A (en) | Image processing system and method for processing television divided pictures | |
| JP2003348447A (en) | Image output apparatus | |
| CN109873954B (en) | Method for realizing color recovery of Bayer array based on FPGA | |
| CN113962842B (en) | Dynamic non-polar despinning system and method based on high-level synthesis of large-scale integrated circuit | |
| CN109102770A (en) | A kind of low-power consumption low bandwidth display panel driving chip towards high-performance calculation | |
| TWI783625B (en) | Chip set for frame rate control and associated image processing method | |
| WO2023178576A1 (en) | Display control method, display control apparatus, and intelligent terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |