KR102258501B1 - Multiple channel image combining output apparatus based on fpga - Google Patents

Abstract

Disclosed is a multichannel image combining and outputting apparatus which combines input image signals input through each of a plurality of input channels to output an integrated image to a display. According to the present invention, the multichannel image combining and outputting apparatus comprises: a first scaler group (110) for synchronizing input image signals and adjusting the resolutions to be the same; line buffer pairs (120) alternately storing odd line data and even line data; a second scaler group for reducing and scaling the input image signals; a sync generator (170) providing a reference sync signal and a reference resolution for synchronization and resolution adjustment; a frame buffer unit (140) including a first frame buffer and a second frame buffer to store the input image signals reduced and scaled by the second scalers included in the second scaler group in frame units; a first memory (M1) and a second memory (M2) for storing the frames stored in the frame buffer unit; a control unit (151) reading the frames stored in the frame buffer unit to write the read frames to the first memory and the second memory and reading the frames stored in the first memory and the second memory to provide the read frames to an image synthesizing unit; and an image synthesizing unit (160) synthesizing the frames stored in the first memory and the second memory to generate and output an integrated image to a display.

Description

FPGA-based multi-channel video combination output device {MULTIPLE CHANNEL IMAGE COMBINING OUTPUT APPARATUS BASED ON FPGA}

The present invention relates to a multi-channel image combination output device for combining input images input through a plurality of input channels and displaying them on a single display device, and specifically, a multi-channel image based on an FPGA (Field Programmable Gate Array). It relates to a combination output device.

Among various image signal processing methods, a number of multi-matrix image signal processing systems have been developed that enable multiple inputs and multiple outputs of image signals and convert processing between image signals. For example, Registered Patent No. 10-0951875 (registered on April 01, 2010), Registered Patent No. 10-1498667 (registered on February 26, 2015), and Registered Patent No. 10-1498674 (registered on February 26, 2015) Registration) and the like are disclosed. In addition, in such multi-input multi-output image processing, in particular, a synchronous combination of ultra-high-quality UHD input video signals and a generation lock matrix system for preventing delay in video and audio conversion is disclosed. (Registration on February 02).

However, in the prior art as described above, in processing images input through multiple input channels, it is necessary to implement a system by configuring all of a plurality of scalers (scaling units) in a chip form, and thus, an increase in cost is inevitable. In addition, for the output of the integrated image through the display, the process of matching the resolution of the input images by the scalers, the process of storing them in the line buffers, the process of fetching and scaling the data stored in the line buffers, and the frame There is a problem that a lot of delay occurs in the overall image processing process, such as the process of storing as a unit and the process of importing and synthesizing images, causing flicker or errors in the finally displayed integrated image, resulting in deterioration of image quality. . This problem occurs even in FHD (Full High Definition) having a resolution of 1920 X 1080, but is particularly noticeable in real-time image processing in Ultra High Definition (UHD) having a resolution of 3840 X 2160.

Korean Patent Registration No. 10-1856177 (registered on May 02, 2018)

The problem to be solved by the present invention is a process of matching the resolution of input images by scalers, a process of storing them in line buffers, and a reduction scaling by bringing data stored in the line buffers to output an integrated image through a display. There are many delays in the overall processing process, such as the process of processing, the process of storing in units of frames, and the process of importing and synthesizing the images, resulting in defects or errors in the finally displayed integrated image, resulting in deterioration of image quality. It is to provide a field programmable gate array (FPGA)-based multi-channel image combination output device capable of solving the problem of solving the problem and solving the problem of increasing the cost due to the implementation of the image processing system with a plurality of scaler chips.

In order to solve the above problem, according to an aspect of the present invention, a multi-channel image combination output device for outputting an integrated image to a display by combining input image signals input through each of a plurality of input channels, N (N is 2 The above natural number) In order to synchronize the N input image signals input through each of the input channels and adjust the resolution equally, the N first scalers provided corresponding to each of the N input channels are used. A first scaler group including-Each of the input image signals is input through each of the input channels, and one of the first scalers is provided to correspond to each of the input channels-and, by the first scaler group Includes an odd line buffer for storing Odd Line Data among the synchronized and resolution-adjusted input image signals and an even line buffer for storing Even Line Data among the input image signals. N number of line buffer pairs-one line buffer pair of the line buffer pairs is connected to one of the first scalers and includes one odd line buffer and one even line buffer, the odd Line data and the even line data are alternately stored in the odd line buffer and the even line buffer, respectively-and, by reading input image signals stored in the pair of line buffers, the integrated image to be displayed on the display A second scaler group that reduces and scales the read input image signals so that they are respectively reduced and displayed according to the screen configuration-In the second scaler group, one second scaler corresponds to the one line buffer pair. Includes N second scalers as much as possible-And, in the synchronization of the input image signals in the first scalers in the first scaler group, a reference synchronization signal is provided, and the resolution of the input image signals is the same. In order to store the input image signals reduced and scaled by a sync generator and the second scalers in the second scaler group in a frame unit to provide a reference resolution in the second scaler group. A frame buffer unit including a first frame buffer and a second frame buffer, a first memory and a second memory for storing frames stored in the frame buffer unit, and frames stored in the frame buffer unit are read and the first A control unit that writes to a memory and the second memory and reads the frames stored in the first memory and the second memory to provide the following image synthesis unit, and the first memory and the second memory provided from the control unit. And an image synthesizing unit for synthesizing the stored frames to generate the integrated image, and outputting the integrated image to the display.

According to an embodiment, while odd line data among input image signals synchronized and resolution-adjusted by one of the first scalers are stored in the odd line buffer corresponding to the one first scaler, Even line data among input image signals synchronized by the one first scaler and adjusted for resolution are read by one second scaler connected to an even line buffer corresponding to the one first scaler among the second scalers While the even line data among input image signals synchronized by the one first scaler and adjusted for resolution are stored in the even line buffer corresponding to the one first scaler, the one first scaler The odd line data of the input image signals synchronized by and resolution-adjusted are read by the one second scaler connected to the odd line buffer corresponding to the one first scaler among the second scalers.

According to an embodiment, while the control unit reads a frame stored in the first frame buffer and writes it to the first memory, input image signals reduced and scaled by the second scalers in the second scaler group are An input image signal that is reduced and scaled by the second scalers in the second scaler group while reading the frame stored in the second frame buffer and writing it to the first memory. Are written to the first frame buffer.

According to an embodiment, the control unit controls to provide the frames stored in the second memory to the image synthesis unit while writing frames stored in the first frame buffer or the second frame buffer to the first memory, Conversely, while the frames stored in the first frame buffer or the second frame buffer are written to the second memory, the frames stored in the first memory are provided to the image synthesizing unit.

According to an embodiment, the multi-channel image combination output device further includes a plurality of output channels to output the integrated image generated by the image synthesizing unit 160 to a plurality of displays.

According to an embodiment, the multi-channel image combination output device is implemented as one field programmable gate array (FPGA).

The present invention provides an FPGA (Field Programmable Gate Array)-based multi-channel image combination output device, so that the resolution of input images is matched by scalers in outputting an integrated image through a display in an existing image processing system. There are a lot of delays in the overall processing process, such as the process of storing in buffers, the process of fetching and scaling data stored in the line buffers, and the process of storing in frame units, and the process of fetching and synthesizing images. It is possible to solve the problem of deterioration of image quality due to defects or errors in the finally displayed integrated image, and by implementing the existing image processing system implemented with multiple scaler chips based on FPGA, the cost can be greatly reduced. There is an effect that can be.

1 is a block diagram illustrating an FPGA-based multi-channel image combination output device 100 according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the accompanying drawings and embodiments described with reference to the drawings are simplified and illustrated with the intention of helping those of ordinary skill in the art to understand the present invention.

Referring to FIG. 1, the FPGA-based multi-channel image combination output device 100 of the present invention includes a first scaler group 110, a plurality of line buffer pairs 120, a second scaler group 130, and a frame buffer. A unit 140, memory units M1 and M2, a control unit 151, an image synthesis unit 160, and a sync generator 170 are included. Although not specifically shown in FIG. 1, the integrated image output to the display unit 20 through the image synthesis unit 160 is a plurality of displays B1, B2, B3, ..., A plurality of output channels are provided corresponding to Bm).

The first scaler group 110 synchronizes N input image signals input through N (N is a natural number greater than or equal to 2) input channels, and adjusts all the same resolutions. For example, N input image signals input through input channels are input images provided from various input sources (A1 to An, collectively 10) such as PC (A1), DVD (A2), and camera (A3). These are the signals. Since the input image signals provided from the various input sources 10 may have different resolutions such as HD, FUD, and UHD, in order to finally display an integrated image of a certain level or higher on a display connected to an output channel, the resolution thereof is adjusted. The same needs to be adjusted. In addition, synchronization of these input video signals is also performed. In this way, by synchronizing the input image signals and matching the resolution to the same, the start and end points of the input image signals are made the same. In this specification, an integrated image is a combination of images (segmented images) coming through a plurality of input channels and displayed on a single display. In this case, it refers to a combination of divided images displayed on the display.

The first scaler group 110 includes N first scalers S11, S12, S13, ..., S1n corresponding to each of the N input channels. That is, since one first scaler is provided to correspond to one input channel, each of the input image signals is configured to be input through each of the input channels. The number of first scalers is 8 (that is, N = 8), and the number of input channels corresponding thereto may also be 8, but the number is not limited thereto.

The N line buffer pairs 120 separate and store input image signals synchronized and resolution-adjusted by the first scaler group 110 for each of odd line data and even line data. For example, in the case of an FHD image with a resolution of 3840 X 2160, there are 2160 horizontal lines, and data for odd line data (Odd Line Data) and even line data (Even Line Data) They are alternately stored in the N number of line buffer pairs 120. That is, when looking at one line buffer pair (LB11, LB12) as a reference, among the input video signals that are connected to the first scaler S11 and have the same resolution as other input video signals by the first scaler S11, The odd line data is stored in the odd line buffer LB11, and the even line data is stored in the even line buffer LB12. As described above, the odd-numbered data and the even-numbered data are stored alternately in the line buffers LB11 and LB12, respectively.

Each of the N line buffer pairs 120 includes a pair of line buffers as described above. That is, a pair of line buffers are connected to correspond to each of the first scalers. LB11 and LB12 are connected to S11, LB21 and LB22 are connected to S12, LB31 and LB32 are connected to S13, and LBn1 and LBn2 are connected to S1n.

The second scaler group 130 reads the input image signals stored in the line buffer pairs 120 so that the read input image signals are reduced and displayed according to the screen configuration of the integrated image to be displayed on the display 20. In order to do this, the read input image signals are reduced and scaled. The second scaler group 130 is provided to correspond to one line buffer pair. The screen configuration of the integrated image to be displayed on the display 20 is achieved by appropriately inputting a control command through input/output devices (not shown) such as a mouse or a keyboard. In the present specification, read means importing data from one component to another, and write means storing data from one component to another component.

Therefore, looking at the relationship between the first scaler group 110, the N number of line buffer pairs 120, and the second scaler group 130 in more detail, one line buffer pair in one first scaler (eg, S11) The LB11 and LB12 are correspondingly connected, and one second scaler S21 is correspondingly connected to the line buffer pair LB11 and LB12. For all the first scalers S11, S12, S13, ..., S1n, one line buffer pair and one second scaler are correspondingly connected.

In this way, Odd Line Data among input image signals synchronized and resolution-adjusted by one of the first scalers (e.g., S11) corresponds to one first scaler (S11). While being stored in the odd line buffer LB11, the Even Line Buffer among the input image signals synchronized and resolution-adjusted by one first scaler S11 is one of the second scalers 130 An input image read by one second scaler S21 connected to the even line buffer LB12 corresponding to the first scaler S11 of, and synchronized and resolution-adjusted by one first scaler S11 While even line data among the signals is stored in the even line buffer LB12 corresponding to one first scaler S11, the odds among input image signals synchronized and resolution-adjusted by one first scaler S11 The line data is read by one second scaler S21 connected to the odd line buffer LB11 corresponding to the first scaler S11 among the second scalers 120. That is, the first scaler S11 ), between one line buffer pair (LB11, LB12), and the second scaler (S21), while the odd line data is written from the first scheduler (S11) to the odd line buffer (LB11), the second scheduler (S21) The even line data is read from the even line buffer LB12 and, while the even line data is written to the even line buffer LB12 from the first scheduler S11, the odd line data is read from the second scheduler S21. Even line data and odd line data are continuously alternately present in the input video signal, so that the even line buffer LB12 and the odd line buffer LB11 are also included in the line buffer pairs 120. Alternately, data is written, and accordingly, the second scaler (S21) is also Each data is read alternately from the in buffer LB12. A write or read process between the first scaler 110, the line buff pairs 120, and the second scaler 130 is not directly expressed in FIG. 1, but may be controlled by the control unit 151.

The process of storing (i.e., writing) in one line buffer pair LB11 and LB12 in the first scaler S11 and the process of reading in the second scaler S21 from one line buffer pair LB11 and LB12 are each Since it is difficult to do this at the same time, if there is only one line buffer in the past, an interval for writing and reading is required, which causes flicker or defects in the integrated image in the real-time display on the display 20 side. . Accordingly, the present invention solves such an existing problem by providing a pair of line buffers including an odd line buffer and an even line buffer to correspond to each of the first and second scalers.

The sync generator 170 provides a reference synchronization signal for synchronization of input image signals from the first scalers S11, S12, S13, ..., S1n in the first scaler group 110 In addition, in the first scalers S11, S12, S13, ..., S1n, the first scalers S11, S12, S13...

The frame buffer unit 140 is a component for storing the input image signals reduced and scaled by the second scaler group 130 in units of frames. The frame buffer unit 140 includes a first frame buffer FB1 and a second frame buffer FB2. The frame buffer unit 140 converts the input image signals reduced and scaled by the second scalers S21, S22, S23, ..., S2n of the second scaler group 130 to the first memory M1 and the second It is stored in a frame unit to store in the memory M2. While the input video signal is stored (written) in the first memory M1, that is, during the read process in the first frame buffer FB1, the next frame is written to the first frame buffer FB1. ), the second frame buffer FB2 is provided to store the next frame. While the input image signal is stored (written) in the second memory M2 from the second frame buffer FB2, that is, during a read process in the second frame buffer FB2, the first frame buffer FB1 is The next frame is lighted. As described above, it is possible to prevent data loss (frame loss) by allowing one frame and the next frame to be stored alternately.

The first memory M1 and the second memory M2 are elements for storing (writing) frames stored in the frame buffer unit 140, that is, an input image signal in units of frames.

The control unit 151 reads frames stored in the frame buffer unit 140, that is, an input image signal in units of frames, and writes them to the first memory M1 and the second memory M2, and the first memory M1 And frames stored in the second memory M2, that is, the input image signal in units of frames are read and provided to the image synthesizing unit 160. The first memory M1 and the second memory M2 and the control unit 151 may be configured as one unit 150.

In the operation between the first memory M1 and the second memory M2, the control unit 151, and the frame buffer unit 140, similarly to the above, (151) stored in the first frame buffer FB1 by the control unit. While reading the frame and writing it to the first memory M1, the input image signals reduced and scaled by the second scalers S21, S22, S23, ..., S2n in the second scaler group 130 are 2 While the frame is written to the frame buffer FB2 and, on the contrary, the frame stored in the second frame buffer FB2 is read and written to the first memory M1, the second scalers S21 in the second scaler group 130 , S22, S23, ..., S2n), the input image signals scaled down are written to the first frame buffer FB1, so that one frame and the next frame can be alternately stored. Data loss (frame loss) can be prevented.

The image synthesis unit 160 generates an integrated image by synthesizing frames stored in the first memory M1 and the second memory M2 provided from the control unit 151 to display the generated integrated image. It is a component for output to the (20) side. That is, the image synthesizing unit 160 reduces a plurality of input image signals input through an input channel from the input sources 10 and synthesizes the input image signals to display all the divided images into one display. Thus, it plays a role of synthesizing images to be displayed on one display screen at a desired location and size. The synthesized image data, that is, the integrated data, is imaged to the display 20 through an output port through a high definition multimedia interface (HDMI).

Similar to the above, the first frame buffer FB1 or the operation between the frame buffer unit 140, the control unit 151, the first memory M1, the second memory M2, and the image synthesis unit 160 While writing the second frame buffer FB2 to the first memory M1, the frames stored in the second memory M2 are controlled to be provided to the image synthesizing unit 160, and on the contrary, the first frame buffer FB1 Alternatively, while the frames stored in the second frame buffer FB2 are written to the second memory M2, the frames stored in the first memory M1 are controlled to be provided to the image synthesizing unit 160.

In addition, as shown in FIG. 1, the multi-channel image combination output apparatus of the present invention includes a plurality of input image signals input through input channels from N input sources 10 to each of a plurality of displays 20. A plurality of output channels for outputting an integrated image to each of the plurality of displays 20 may be included so that all of them can be displayed on one display screen.

And, as mentioned above, the multi-channel image combination output device of the present invention is implemented as a single FPGA (Field Programmable Gate Array), so that the number of components is increased due to the configuration of all of the existing scalers in the form of a chip. The resulting cost increase can be reduced.

100: Multi-channel video combination output device of the present invention
110: first scaler group
120: line buffer pairs
LB11, LB21, LB31, LBn1: Odd line buffer
LB12, LB22, LB32, LBn2: Even Line Buffer
130: second scaler group
140: frame buffer unit
151: control unit
160: image synthesis unit
170: sync generator

Claims (6)

As a multi-channel image combination output device 100 for outputting an integrated image to a display by combining input image signals input through each of a plurality of input channels, implemented as one field programmable gate array (FPGA),
In order to synchronize the N input image signals input through each of the N input channels (N is a natural number of 2 or more) and adjust the resolution equally, N is provided corresponding to each of the N input channels. A first scaler group 110 including ten first scalers-each of the input image signals is input through each of the input channels, and one of the first scalers is provided to correspond to each of the input channels -;
An odd line buffer for storing Odd Line Data among input image signals synchronized by the first scaler group and whose resolution is adjusted, and storing Even Line Data among the input image signals. N number of line buffer pairs 120 including an even line buffer for the line buffer pair. One line buffer pair of the line buffer pairs is connected to one of the first scalers, and is connected to one of the first scalers. It includes an even line buffer, wherein the odd line data and the even line data are alternately stored in the odd line buffer and the even line buffer, respectively, and the odd line data and the even line data are the same input among the input channels. -Data provided from the same input source through the channel;
The read input image so that the input image signals stored in the line buffer pairs are read, and the read input image signals are respectively reduced and displayed according to the screen configuration of the integrated image to be displayed on the display. A second scaler group 130 for reducing and scaling signals, the second scaler group including N second scalers such that one second scaler corresponds to the one line buffer pair;
A sync generator for providing a reference synchronization signal in synchronizing the input image signals in the first scalers in the first scaler group and providing a reference resolution in equally adjusting the resolution of the input image signals ( Sync Generator) 170;
A frame buffer unit 140 including a first frame buffer and a second frame buffer to store the input image signals reduced and scaled by the second scalers in the second scaler group in a frame unit;
A first memory (M1) and a second memory (M2) for storing frames stored in the frame buffer unit;
A control unit 151 for reading frames stored in the frame buffer unit, writing them to the first memory and the second memory, reading frames stored in the first memory and the second memory, and providing them to the following image synthesis unit; And
And an image synthesizing unit 160 for synthesizing the frames stored in the first memory and the second memory provided from the control unit to generate the integrated image, and outputting the integrated image to the display; and ,
While the odd line data of input image signals synchronized and resolution-adjusted by one of the first scalers are stored in the odd line buffer corresponding to the one first scaler, the one first scaler Even line data among input image signals synchronized by and resolution-adjusted are read by one second scaler connected to an even line buffer corresponding to the one first scaler among the second scalers, and the While even line data among input image signals synchronized and resolution-adjusted by one first scaler is stored in an even line buffer corresponding to the one first scaler, it is synchronized and resolution is adjusted by the one first scaler. Among the input image signals, the odd line data is read by the one second scaler connected to the odd line buffer corresponding to the one first scaler among the second scalers, and-corresponding to the one first scaler. One second scaler connected to the even line buffer and one second scaler connected to the odd line buffer corresponding to the one first scaler are the same scaler -,
While the control unit reads the frame stored in the first frame buffer and writes it to the first memory, the input image signals reduced and scaled by the second scalers in the second scaler group are stored in the second frame buffer. On the contrary, while reading the frame stored in the second frame buffer and writing it to the first memory, the input image signals reduced and scaled by the second scalers in the second scaler group are the first frame buffer. The multi-channel video combination output device, characterized in that the light is on.
delete delete The method according to claim 1,
The control unit,
While the frames stored in the first frame buffer or the second frame buffer are written to the first memory, the frames stored in the second memory are controlled to be provided to the image synthesis unit, and conversely, the first frame buffer or the The apparatus for outputting a multi-channel image combination, comprising controlling to provide the frames stored in the first memory to the image synthesizing unit while writing frames stored in the second frame buffer to the second memory.
The method according to claim 1,
The multi-channel video combination output device,
And a plurality of output channels to output the integrated image generated by the image synthesizing unit 160 to a plurality of displays.
delete

Images