KR20190009972A - An image processing apparatus and method for image parallel rendering processing - Google Patents

Abstract

Disclosed are an apparatus for processing an image for a multi-display system and a method thereof which can extract a division image signal to be displayed in each graphic adaptor from a source image signal to simultaneously transmit the division image signal to each graphic adaptor in parallel without performing any image processing on a source image.

Description

TECHNICAL FIELD [0001] The present invention relates to an image processing apparatus and a method for image parallel rendering,

The present invention relates to an image processing apparatus and method for image parallel rendering, and more particularly, to an image processing apparatus and method for performing image parallel rendering, And simultaneously transmitting and displaying the image data in parallel with each of the graphic adapters, and to an image processing apparatus and method for image parallel rendering processing.

Due to the development of information and communication technology, various types of display devices are widely used. Particularly, the necessity for a large display screen is increasing day by day due to personal taste of users, specific use (for example, performance / sport relay, control of control situation) and the like. Techniques for constructing one large multi-display device with a plurality of individual display devices to constitute such a large display screen are widely used due to technical limitations for manufacturing a large display screen and expensive manufacturing costs.

1 is a schematic block diagram of a multi-display system according to the prior art. A multi-display system according to the related art will be briefly described with reference to FIG. A multi-display system according to the prior art includes a multi-display device 100 and an image processing apparatus 10 according to the prior art.

The multi-display device 100 shown in FIG. 1 is a 24-screen multi-display device composed of a total of twelve individual display devices 110a to 110l, and displays image data output from the image processing device 10. FIG. In particular, the display screen of the multi-display device 100 shown in FIG. 1 includes a first sub-area 120a, a second sub-area 120b, a third sub-area 120c, And each of the sub-areas 120a to 120d is controlled by a corresponding graphics adapter 12a to 12d in the image processing apparatus 10 according to the related art.

The image processing apparatus 10 according to the related art is configured to receive a source image from an image source 300 and output the received source image through the multi-display device 100. More specifically, the image processing apparatus 10 according to the related art includes a first graphics adapter 12a, a second graphics adapter 12b, a third graphics adapter 12c, a fourth graphics adapter 12d, a first PCI Slot 14a, a second PCI slot 14b, a third PCI slot 14c, a fourth PCI slot 14d, and a decoding module 16.

The decoding module 16 is configured to receive the encoded source image from the image source 300 and to decode the received source image into raw image data 320 and output to the graphics adapters. The first to fourth graphic adapters 12a to 12d are installed in the first to fourth PCI slots 14a to 14d respectively and decode the decoded raw video data output from the decoding module 16 Area 120a to the fourth sub-area 120d on the corresponding sub-area of the multi-display device 100. The display of the first sub-

It should be noted that the image processing apparatus 10 according to the related art constructed as described above is configured to process the decoded raw image data 320 in a sequential and serial manner. Let's look at this in more detail. First, as shown in the drawing, the row image data 320 includes first divided image data 320a to be displayed in the first sub-area 120a by the first graphic adapter 12a, The second divided image data 320b to be displayed in the second sub-area 120b by the third graphic adapter 12b, the third divided image data 320b to be displayed in the third sub-area 120c by the third graphic adapter 12c, Image data 320c and fourth divided image data 320d to be displayed in the fourth sub-area 120d by the fourth graphic adapter 12d.

The decoding module 16 transmits the entire low image data 320 including the first divided image data 320a to the fourth divided image data 320d to the first graphic adapter 12a, The graphic adapter receives the entire row image data 320 including all the divided image data and extracts the first divided image data 320a of the entire row image data 320 and outputs the extracted first divided image data 320a on the first sub- Display. Then, the first graphic adapter 12a transmits the row image data 320 from which the first divided image data 320a has been removed to the second graphic adapter 12b, and the second graphic adapter 12b transmits the row image data 320, And displays the second divided image data 320b on the second sub-area 120b by receiving the data 320 'and extracting the second divided image data 320b from the received row image data 320'. Similarly, the second graphic adapter 12b transmits the row image data 320 '', from which the first divided image data 320a and the second divided image data 320b have been removed, to the third graphic adapter 12c, The third graphic adapter 12c receives the row image data 320 "and extracts the third divided image data 320c from the received row image data 320" . Finally, the third graphic adapter 12c receives the row image data 320 '' '(i.e., the fourth divided image data 320d') from which the first divided image data 320a to the third divided image data 320c are removed ) To the fourth graphics adapter 12d and the fourth graphics adapter 12d receives the row image data 320 '' 'and the fourth segment image data 320d to the fourth sub-region 120d ).

As described above, the image processing apparatus 10 according to the related art is configured to process image data in a sequential and serial manner in order to support the multi-display device 100. This sequential and serial rendering method requires a considerable amount of time and resources for image processing.

An object of the present invention is to solve the above-mentioned problems of the prior art.

It is an object of the present invention to extract a divided video signal to be displayed from each graphics adapter from a source video signal and to simultaneously output the divided video signals to each graphics adapter in parallel And to provide a video processing apparatus and an image processing method capable of transmitting video data.

It is another object of the present invention to provide an image processing apparatus and an image processing method capable of displaying an extracted divided image signal synchronously in time and frame.

In order to achieve the above-described object of the present invention and to achieve the specific effects of the present invention described below, the characteristic structure of the present invention is as follows.

According to an aspect of the present invention, there is provided an image processing method performed by an image processing apparatus to perform an image parallel rendering processing on a multi-display apparatus, the image processing apparatus comprising: (A) comparing and matching the display area between the image source and the multi-display device, and based on the display area matching result and the sub-area setting information for each graphics adapter, Determining a divided image area of a source image to be displayed by each of the plurality of graphic adapters, and allocating a video memory area in which divided image data for each graphic adapter is to be stored; (b) receiving a source image frame from the image source and decoding the received source image frame into raw image data; (c) storing corresponding divided image data of the decoded raw image data based on the determined divided image area for each graphic adapter in a video memory area allocated to the corresponding graphic adapter; And (d) synchronizing the respective divided image data stored in the video memory area allocated for each of the graphics adapters on a frame-by-frame basis and simultaneously transmitting the image data in parallel to the corresponding graphics adapter.

Advantageously, step (a) comprises the steps of: (a1) extracting a list of the plurality of graphics adapters connected to the multi-display device; (a2) extracting or receiving display information for each of the plurality of graphic adapters, the display information for each of the plurality of graphic adapters including the sub-area setting information for each of the plurality of graphic adapters; (a3) comparing the resolution of the source image from the image source with the full resolution of the multi-display device to determine a pixel ratio between the source image and the multi-display device; And (a4) determining, based on the determined pixel ratio and the graphics adapter-specific sub-area setting information, a divided image area of the source image to be displayed by each of the plurality of graphics adapters, And allocating a video memory area in which the video data is to be stored.

Advantageously, the step (b) may further comprise the step of providing a data time stamp (DTS) to the received source image frame based on the reception time of the source image frame.

Advantageously, the step (c) may store each divided image data together with frame attribute information including the data time stamp in a video memory area allocated to the corresponding graphics adapter.

Advantageously, the frame attribute information may include the data time stamp, frame per second (FPS), and sequence number.

Preferably, the step (d) comprises: receiving each of the divided image data and the frame attribute information stored in the video memory area allocated for each graphic adapter, and, based on the frame attribute information, May be synchronized on a frame-by-frame basis and simultaneously transmitted in parallel to a corresponding graphics adapter.

Advantageously, step (d) comprises the steps of: (d1) synchronizing the main handler and the plurality of distributor handlers corresponding to each of the plurality of graphics adapters to each other in time; (d2) receiving the divided image data and the frame attribute information from a video memory area allocated to a corresponding graphic adapter of each of the plurality of distribution handlers; (d3) transmitting, by the plurality of distribution handlers, image reception information including the frame attribute information to the main handler; (d4) receiving, by the main handler, the image reception information transmitted from each of the plurality of distribution handlers, generating image reproduction information based on the received image reception information, and transmitting the generated image reproduction information to each of the plurality of distribution handlers, The video playback information includes a playback start time, a data time stamp of the playback subject division video data, and a sequence number; And (d5) receiving, by the plurality of distribution handlers, the image reproduction information transmitted from the main handler, generating, based on the received image reproduction information, the data time stamp and the divided image data corresponding to the sequence number To a corresponding graphics adapter in parallel.

Advantageously, said plurality of distribution handlers may be dynamically generated by said main handler.

According to another aspect of the present invention, there is provided an image processing apparatus for performing an image parallel rendering processing on a multi-display apparatus, the apparatus comprising: a plurality of graphic adapters each controlling display of a predetermined sub-area of the multi-display apparatus; A decoding module that receives the source image frame from the image source and decodes the received source image frame into raw image data; And a display area between the image source and the multi-display device are compared and matched to determine a divided image area of a source image to be displayed by each of the plurality of graphic adapters, and the divided image data for each graphic adapter is stored And allocating corresponding video data of the decoded raw video data to the video memory area allocated to the corresponding graphics adapter based on the determined divided video area of the graphics adapter, There is provided an image processing apparatus including an image distributing and rendering module for simultaneously synchronizing each divided image data stored in a video memory area on a frame-by-frame basis and simultaneously transmitting the image data in parallel to a corresponding graphics adapter.

Preferably, the image distribution rendering module is configured to perform a division of the source image to be displayed by each of the plurality of graphic adapters based on the pixel ratio between the source image and the multi-display device and the sub- A display area processing unit for determining an image area and allocating a video memory area in which divided image data for each graphic adapter is to be stored; A video memory in which a plurality of pieces of divided image data of decoded row image data are stored in respective video memory areas assigned to corresponding graphic adapters; And storing the corresponding divided image data of the decoded raw image data based on the determined divided image area for each graphics adapter in a video memory area allocated to the corresponding graphic adapter, And an image division processing unit for simultaneously transmitting the divided image data in parallel to the corresponding graphic adapter in synchronization with each frame.

Preferably, the image segmentation processing unit may provide a data time stamp (DTS) to the received source image frame based on the reception time of the source image frame.

Preferably, the image segmentation processing unit may store the segmented image data together with the frame attribute information including the data time stamp in a video memory area allocated to the corresponding graphic adapter.

Advantageously, the frame attribute information may include the data time stamp, frame per second (FPS), and sequence number.

Preferably, the image division processing unit receives each of the divided image data and the frame attribute information stored in the video memory area allocated for each graphic adapter, and stores the received respective divided image data on the basis of the frame attribute information It can be synchronized on a frame-by-frame basis and simultaneously transmitted in parallel to a corresponding graphics adapter.

Preferably, the image division processing unit includes a plurality of distribution handlers corresponding to each of the plurality of graphic adapters, wherein each of the plurality of distribution handlers extracts, from the video memory area allocated to the corresponding graphics adapter, Receiving frame attribute information, transmitting image reception information including the frame attribute information to the main handler, receiving image playback information transmitted from the main handler, and receiving the playback start time To the corresponding graphics adapter in parallel, the data time stamp and the segmented image data corresponding to the sequence number in parallel; And image reproduction information including a reproduction start time, a data time stamp of the reproduction target division image data, and a sequence number, based on the received image reception information And a main handler for transmitting to each of the plurality of distribution handlers.

Advantageously, said plurality of distribution handlers may be dynamically generated by said main handler.

According to a preferred embodiment of the present invention, in a multi-display system, a video signal to be displayed by each graphics adapter is extracted from a source video signal without performing any image processing on the source video, It is possible to expect the effect of greatly reducing the time and resources required for image processing by transmitting in parallel.

1 is a schematic block diagram of a multi-display system according to the prior art.
2 is a schematic block diagram of a multi-display system according to an embodiment of the present invention.
3 is a block diagram of an image distribution and rendering module according to an embodiment of the present invention.
4 is a flowchart illustrating an image parallel rendering process performed by an image processing apparatus according to an embodiment of the present invention.
5 is a flowchart illustrating a display area comparison / matching process performed by the image processing apparatus according to an exemplary embodiment of the present invention.
FIG. 6A is a diagram conceptually showing the acquisition time of each divided image.
FIG. 6B is a diagram conceptually illustrating display times of divided images reproduced in synchronization with the present invention.
7 is a flowchart illustrating a process of synchronizing a reproduction time and a reproduction frame performed by an image processing apparatus according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which the claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

[Preferred Embodiment of the Present Invention]

In an embodiment of the present invention, the term 'multi-display device' means a display device in which a display screen is divided into a plurality of sub-areas and each of the sub-areas is supported by one graphic adapter. This 'multi-display device' may be, for example, a multi-vision display device in which a plurality of individual display devices constitute one multi-display device. However, the present invention is not limited to a specific display device, and covers all kinds of display devices capable of performing the functions as described above.

The term 'divided image region of a source image' means one region of a source image that one graphic adapter should display, and 'segmented image data' means image data corresponding to a segmented image region of the entire source image data .

An overview of multi-display systems

2 is a schematic block diagram of a multi-display system according to an embodiment of the present invention. Hereinafter, a schematic configuration and functions of the multi-display system according to the present invention will be described with reference to FIG.

First, prior to a detailed description of the multi-display system according to the present invention, the most important concept of the present invention will be described. As described above, the present invention is intended to solve the problems caused by processing image data in a sequential and serial manner in the image processing apparatus according to the related art. Therefore, the basic idea of the present invention is to extract the segmented image data to be displayed in each graphic adapter from the source image data, and simultaneously transmit them in parallel to each of the graphic adapters. In order to realize such a concept, the present invention provides an image distributing system configured to extract segment image data for each graphic adapter from the entire source image data, and simultaneously transmit the segment image data for each of the graphic adapters to the corresponding graphic adapter in parallel, A rendering module 230 is proposed. The image distribution and rendering module 230 will be described in detail with reference to FIGS. 2 and 3. FIG.

2, a multi-display system according to the present invention includes a multi-display device 100, an image processing device (not shown) for performing image parallel rendering processing on the multi-display device 100 200) (hereinafter referred to as an 'image processing apparatus 200'), and an image source 300.

The image source 300 of FIG. 2 may be various information communication devices capable of transmitting various encoded source images to the image processing apparatus 200 by wire and / or wirelessly. For example, the video source 300 may include, but is not limited to, a CCTV, a digital camera, a digital camcorder, a personal computer, a notebook, a tablet, a smart phone, and the like. Also, the video source 300 of FIG. 2 may be substantially the same video source as the video source 300 of FIG. The image source 300 may transmit one still image, or may transmit a video stream including a plurality of sequential image frames. Hereinafter, it should be noted that although the video source describes an embodiment in which a video stream is transmitted, the idea and principle of the present invention can be equally applied to an embodiment for transmitting a still image.

The multi-display device 100 of FIG. 2 may be substantially the same display device as the multi-display device 100 of FIG. 1, or may be a different display device. Hereinafter, in order to further clarify the comparison between the prior art and the present invention, it is assumed that the same multi-display device 100 having the same sub-areas as in FIG. 1 is used and that the same row image data is used, A multi-display system according to the present invention will be described. The multi-display device 100 shown in FIG. 2 is a 24-screen multi-display device composed of twelve individual display devices 110a to 110l, and displays image data output from the image processing device 200 according to the present invention . In particular, the display screen of the multi-display device 100 shown in FIG. 2 includes a first sub-area 120a, a second sub-area 120b, a third sub-area 120c, And each sub-region 120a to 120d is controlled by a corresponding graphic adapter 210a to 210d in the image processing apparatus 200 according to the present invention. However, it should be noted that the display settings such as the position, size, and graphics adapter for controlling the position and size of the sub-areas of the multi-display device 100 may be changed as needed or according to the user's setting.

The image processing apparatus 200 according to the present invention is configured to receive a source image (e.g., a source image stream) from an image source 300 and output the received source image through the multi-display device 100. More specifically, the image processing apparatus 200 according to the present invention includes a first graphics adapter 210a, a second graphics adapter 210b, a third graphics adapter 210c, a fourth graphics adapter 210d, a first PCI And may include a slot 212a, a second PCI slot 212b, a third PCI slot 212c, a fourth PCI slot 212d, a decoding module 220, and an image scattering rendering module 230. The remaining components except for the image distribution and rendering module 230 are substantially the same as the corresponding components described with reference to FIG. 1, so that detailed description of these components will be omitted herein do. Also, each of the plurality of graphics adapters above may be a commonly used conventional graphics adapter. Also, such a graphics adapter used in the present invention may include a back buffer and a front buffer for accelerating graphics processing.

Hereinafter, the configuration and functions of the image processing apparatus 200 according to the present invention will be described, focusing on the image distribution and rendering module 230, which is a characteristic component of the present invention. As described above, the image distribution rendering module 230 according to the present invention can be configured to process the raw image data 320 decoded by the decoding module 220 in a simultaneous and parallel manner. Let's look at this in more detail. 1, the row image data 320 includes first divided image data 320a to be displayed in the first sub-area 120a by the first graphic adapter 210a, The second divided image data 320b to be displayed in the second sub-area 120b by the third graphic adapter 210b and the third divided image data 320b to be displayed in the third sub- Data 320c and fourth divided image data 320d to be displayed in the fourth sub-area 120d by the fourth graphic adapter 210d. The image distribution rendering module 230 extracts the first divided image data 320a from the row image data 320 and stores the extracted first divided image data 320a in the first memory area of the video memory allocated to the first graphic adapter 210a, Extracts the second divided image data 320b from the data 320 and stores the extracted second divided image data 320b in the second memory area of the video memory allocated to the second graphic adapter 210b, And extracts the fourth divided image data 320d from the row image data 320 and stores it in the third memory area of the video memory allocated to the third graphic adapter 210c, And may be stored in the fourth memory area of the video memory allocated to the adapter 210d. 2, the image distribution rendering module 230 may include a graphics adapter corresponding to the first divided image data 320a to the fourth divided image data 320d in parallel at the same time (more specifically, To the back buffer of the corresponding graphics adapter), thereby performing the parallel rendering process of the image. The detailed configuration and function of the image distributing and rendering module 230 according to the present invention for performing the image parallel rendering process will be described in detail with reference to FIG.

Configuration and Function of Image Distributed Rendering Module

3 is a block diagram of an image distribution and rendering module according to an embodiment of the present invention. Hereinafter, the configuration and functions of the image distribution and rendering module 230 according to the present invention will be described in detail with reference to FIG.

3, the image distribution and rendering module 230 according to the present invention may include a display area processing unit 240, a video memory 242, and an image division processing unit 246.

First, the display area processing unit 240 compares and matches the source image transmitted from the image source 300 with the display screen of the multi-display device 100, and outputs the matching result and the sub-area setting information Based on a plurality of graphics adapters, each of the plurality of graphics adapters may be configured to determine a segmented image area of a source image to display. That is, as described above, the image distribution rendering module 230 separately extracts only the divided image data to be displayed on the multi-display device 100 by the respective graphic adapters from the row image data 320 It is necessary to first determine which part of the source image corresponds to the sub-area of the multi-display device 100 in which each graphics adapter is responsible. In order to perform this function, first, sub-area setting information for each graphics adapter is required. Such sub-area setting information per graphics adapter can be input by a user (e.g., an operator) at system startup. For example, the display area processing unit 240 is configured to provide a display setting interface capable of inputting display information for each of the graphic adapters to the user, and receive / store display information for each of the graphic adapters input through the display setting interface. Such display information for each graphics adapter may include sub-area setting information, resolution information, and the like, which the graphics adapter is responsible for. The display setting process can be performed only once in the initial setting process of the image processing apparatus 200, and after that, the stored setting information can be extracted and used. Of course, the user can change the sub-area settings at any time as needed.

In addition, the display area processing unit 240 compares the source image transmitted from the image source 300 with the pixel ratio between the multi-display device 100 to configure the source image and the display screen of the multi- . For example, if the source image is a resolution of 1920 x 1080, and the 12 display devices having the same resolution of the multi-display device 100 are arranged in a matrix of 4 x 3 arrays, The image can be divided into 4 * 3 masks, and the pixel range of the divided image area to be processed by each graphics adapter can be determined according to the mask ratio. Similarly, when the source image is a resolution of 1920 x 1080, and the 30 display devices having the same resolution of the multi-display device 100 are arranged in a matrix of 6 x 5 arrays, the display area processing unit 240 generates a source image Can be divided into 6 * 5 masks, and the pixel range of the divided image area to be processed by each graphics adapter can be determined according to the mask ratio.

The display area processing unit 240 is configured to allocate the video memory area in which the divided image data for each graphics adapter is to be stored in the video memory 242. In this case, For example, in the embodiment shown in FIG. 2, the first divided image data 320a corresponding to the first graphic adapter 210a is allocated to be stored in the first memory area in the video memory 242, The second divided image data 320b corresponding to the second graphics adapter 210b is allocated to be stored in the second memory area in the video memory 242 and the like. As described above, since the image processing apparatus 200 according to the present invention is configured to receive and process a source video stream, in order to achieve a frame-by-frame synchronized display of divided images, Need to be stored. Thus, the size of the memory area allocated for each graphics adapter can be set to be sufficient to store the divided image data of the predetermined number of frames.

When the video memory area for storing the divided image area and the divided image data for each graphic adapter is determined through the above-described process, the display area processing part 240 transmits the information to the image division processing part 246, (246) is configured to receive this information and use it for image distribution rendering processing.

The video memory 242 is configured to store a plurality of segmented image data of the decoded row image data in respective video memory areas assigned to corresponding graphic adapters. Since this video memory 242 itself uses a known technique, the detailed description will be omitted.

The image division processing unit 246 according to the present invention stores the corresponding divided image data of the decoded raw image data based on the determined divided image area for each graphic adapter in the video memory area allocated to the corresponding graphic adapter, And synchronizes the respective divided image data stored in the allocated video memory area on a frame-by-frame basis and simultaneously transmits them in parallel to the corresponding graphics adapter.

The image segmentation processing unit 246 according to the present invention further includes a source image frame from the image source 300 in order to perform frame- And to provide a data time stamp (DTS) to the received source video frame based on the reception time of the source video frame. In addition, the image division processing unit 246 may be configured to add frame attribute information including a data time stamp to each divided image data and store the divided image data in a video memory area allocated to a corresponding graphics adapter. Such frame attribute information may include a data time stamp, a frame per second (FPS), and a sequence number.

This frame attribute information is used to frame-synchronize the segmented image data stored in the video memory 242 by the image segmentation processing unit 246 thereafter and to transmit them in parallel at the same time. More specifically, the image division processing unit 246 receives (or extracts or reads) the frame attribute information added to each of the divided image data and the divided image data stored in the video memory area allocated for each graphic adapter, And synchronizes the received divided image data on a frame-by-frame basis and simultaneously transmits the synchronized image data to a corresponding graphics adapter in parallel.

In order to achieve this synchronization function, the image division processing unit 246 according to the present invention includes a plurality of distributor handlers 250a to 250d corresponding to each of a plurality of graphic adapters, and a main handler ) ≪ / RTI > The main handler 248 is configured to control the plurality of distribution handlers 250a to 250d as a whole so that each of the distribution handlers 250a to 250d simultaneously transmits the divided image data synchronized frame by frame to the corresponding graphics adapter . More specifically, the main handler 248 according to the present invention performs temporal synchronization with the plurality of distribution handlers 205a to 250d and frame synchronization of the divided image data to be output to the graphics adapters 210a to 210d can do. That is, the main handler 248 according to the present invention receives image reception information transmitted from a plurality of distribution handlers 205a to 250d, and based on the received image reception information, A data time stamp, and a sequence number, and transmits the generated image playback information to the plurality of distribution handlers 205a to 250d. This image reproduction information is used to designate the divided image data corresponding to the frame to be transmitted to the graphic adapters 210a to 210d, which are handled by the distribution handlers 205a to 250d, and the time to be transmitted.

Each of the plurality of distribution handlers 205a to 250d is configured to simultaneously transmit the divided image data synchronized on a frame-by-frame basis to each of the plurality of graphic adapters 210a to 210d under the control of the main handler 248. [ More specifically, each of the plurality of distribution handlers 205a to 250d receives or reads the divided image data and frame attribute information from the video memory area allocated to the corresponding graphic adapter, To the main handler 248, receives the image reproduction information transmitted from the main handler 248, specifies the designated divided image data based on the received image reproduction information, and simultaneously specifies the corresponding graphics adapters 210a to 210d, In parallel. In the embodiment shown in FIG. 3, since the embodiment of FIG. 2 including four graphic adapters is used as a reference, the image division processing unit 246 corresponds to the first divided image for the first graphic adapter 210a A first distribution handler 250a for synchronizing and transferring the data 320a and a second distribution handler 250b for synchronizing and transferring the second divisional image data 320b to the second graphics adapter 210b, A third distribution handler 250c for synchronizing and transferring the third divisional image data 320c to the third graphic adapter 210c and a fourth divisional image data 320d for the fourth graphic adapter 210d, And a fourth distribution handler 250d responsible for synchronization and transmission of the data. Those skilled in the art will appreciate that the number of distribution handlers can vary depending on the number of graphics adapters.

Meanwhile, the main handler 248 and the distribution handlers 205a to 250d as described above can be implemented by software, hardware, or a combination thereof. In addition, the distribution handlers 205a-d may be dynamically generated by the main handler 248. [ That is, in one embodiment, the distribution handlers 205a-d may be program threads that are dynamically generated by the main handler 248 according to the number of graphics adapters. In this embodiment, the main handler 248 generates the distribution handlers 205a to 250d and transmits the system time of the image processing apparatus 200 as the reference time to the generated distribution handlers 205a to 250d The main handler 248 can synchronize the time between the distribution handlers 205a through 250d. In addition, the main handler 248 transmits the processing request information (the graphics adapter ID, the divided image information of the corresponding graphics adapter, and the corresponding graphics adapter) to the distributed handlers 205a to 250d, , By assigning a graphics adapter corresponding to each of the distribution handlers 205a through 250d by sending a video memory area, an output resolution, a display sub-area, etc.,

The above-described synchronization process will be described in more detail with reference to FIGS. 6A, 6B, and 7. FIG.

An Example of Image Parallel Rendering Process

4 is a flowchart illustrating an image parallel rendering process performed by an image processing apparatus according to an embodiment of the present invention. Hereinafter, an image parallel rendering process according to the present invention will be described with reference to FIG.

First, the image processing apparatus 200 according to the present invention compares and matches the source image transmitted from the image source 300 with the display screen of the multi-display device 100, Pixel ratio is determined (S400).

Once the pixel ratio between the source image and the multi-display device 100 is determined, the image processing apparatus 200 determines whether the graphics adapter 210a-210d displays each pixel based on the pixel ratio and the sub- A divided image area of the source image is determined (S402). In addition, the image processing apparatus 200 allocates a video memory area to be stored in the video memory 242 according to the graphics adapter (S404).

Such a process as described above can be performed and stored only once at the initial setting, and thereafter, it is possible to process the image dispersion rendering according to the stored setting information. Of course, when any of the configurations of the image source 300, the image processing apparatus 200, and the multi-display apparatus 100 is changed, the above-described process must be performed again.

Next, the image processing apparatus 200 receives the source image transmitted from the image source 300 and decodes the received source image into raw image data (S406). As described above, a source video stream may be transmitted from the video source 300, in which case the video processing apparatus 200 receives each of the source video frames of the source video stream continuously transmitted from the video source 300 And decode it. At this time, the image processing apparatus 200 may assign a data time stamp to each of the received source image frames for the purpose of frame-by-frame synchronization of the divided image data.

Next, the image processing apparatus 200 stores each divided image data of the decoded image row data in a video memory area allocated to the corresponding graphic adapter (S408). On the other hand, as described above, a plurality of divided image data of a plurality of frame images can be stored in the video memory area allocated to the graphic adapter. For example, in the first memory area corresponding to the first graphic adapter 210a, the first divided image data of the first frame image, the first divided image data of the second frame image, the first divided image data of the third frame image, Video data can be stored, and the like. Accordingly, the image processing apparatus 200 adds the frame attribute information including the data time stamp to each divided image data to distinguish the divided image data of the plurality of frames corresponding to the same divided area by frame, It can be stored in the video memory area allocated to the adapter. The frame attribute information is used to frame-synchronize the segmented image data stored in the video memory 242 by the image processing apparatus 200 thereafter and transmit the frame attribute information to the corresponding graphic adapters 210a to 210d in parallel at the same time do.

Next, the image processing apparatus 200 checks the frame attribute information of the divided image data stored in the video memory area of the graphics adapter (S410). Based on the frame attribute information of the divided image data, the image processing apparatus 200 determines the display frame and the display time (S412).

When the display frame and the display time are determined, the image processing apparatus 200 extracts the divided image data of the frame determined as the display frame among the plurality of divided image data stored in the video memory area of the graphics adapter, (210a to 210d) (S414).

5 is a flowchart illustrating a display area comparison / matching process performed by the image processing apparatus according to an exemplary embodiment of the present invention. Hereinafter, a display area comparison / matching process according to the present invention will be described with reference to FIG.

First, the image processing apparatus 200 extracts a list of embedded graphics adapters connected to the multi-display device 100 (S500).

When the list of graphic adapters is extracted, the image processing apparatus 200 extracts or receives display information for each graphic adapter (S502). That is, as described above, when the display information for each graphics adapter including the sub-area setting information for each graphic adapter is inputted and stored by the user, the image processing apparatus 200 extracts the display information for each stored graphics adapter If not stored, the image processing apparatus 200 provides an input interface to the user and receives display information for each graphics adapter.

Once the display information for each graphic adapter is obtained, the image processing apparatus 200 groups the plurality of display devices 110a to 110l constituting the multi-display device 100 according to graphic adapters (S504). That is, the display area of the grouped display devices may correspond to the sub-area of the corresponding graphic adapter.

Then, as described above, the image processing apparatus 200 compares and matches the source image transmitted from the image source 300 with the display area of the multi-display device 100 (S506) A divided image area of the source image is determined (S508).

FIG. 6A is a diagram conceptually showing the acquisition time of each divided image. FIG. 6B is a diagram conceptually illustrating display times of divided images reproduced in synchronization according to the present invention.

6A, in the image processing apparatus 200 according to an embodiment of the present invention, each divided image data (for example, the first divided image data 320a through the fourth divided image data 320a through 320f) 320d) are time-varying. Therefore, if each divided image data is displayed in such a state, there arises a problem that synchronization of displayed images is out of synchronization.

Accordingly, in order to solve the synchronization problem that occurs because the acquisition time of each divided image data is different, the image processing apparatus 200 according to the present invention has a problem in that, as shown in FIG. 6B, To the corresponding graphics adapters 210a to 210d in parallel. Hereinafter, this synchronization process will be described in detail.

7 is a flowchart illustrating a process of synchronizing a reproduction time and a reproduction frame performed by an image processing apparatus according to an embodiment of the present invention. More specifically, FIG. 7 illustrates a synchronization process performed between a main handler 248 and a plurality of distribution handlers 205a-d according to the present invention.

Once the main handler 248 dynamically creates a plurality of distribution handlers 205a-d and sends processing request information for the graphics adapter to each of the distribution handlers 205a-d, 205a to 250d are completed.

First, the main handler 248 performs time synchronization with a plurality of distribution handlers 205a to 250d by transmitting the system time of the image processing apparatus 200 as a reference time to a plurality of distribution handlers 205a to 250d (S700).

With the time synchronization in progress, each of the distribution handlers 250a-d receives frame attribute information for the segmented image data and the segmented image data from the video memory area allocated to the corresponding graphics adapter 210a-210d, (S702, S706, S710, S714), and transmits the image reception information including the frame attribute information added to the divided image data to the main handler 248 (S704, S708, S712, S716). The image reception information transmitted from the distribution handlers 205a to 250d to the main handler 248 includes frame attribute information (input reception time, DTS, FPS, sequence number, etc.), distribution handler ID, . On the other hand, in another embodiment, each of the distribution handlers 250a to 250d may be configured not to receive the divided image data itself, but to receive the video memory address where the divided image data is stored. Further, in another embodiment, each of the distribution handlers 250a-d monitors in real time the video memory area allocated to the corresponding graphics adapter 210a-210d, and the new split image data is stored in the corresponding video memory area And if so, extracts corresponding frame attribute information, generates image reception information, and transmits the image reception information to the main handler 248.

It should be noted here that the receiving (or reading or extracting) time point and the image receiving information transmitting time point of the divided image data shown in the drawings are merely exemplary and the receiving (or reading or extracting) of the divided image data and The transmission of the video reception information can take place in an order different from that shown in the figure. That is, for example, the second distribution handler 250b may acquire the partition image data at a later time than the third distribution handler 250c, so that the second distribution handler 250b may acquire the partition image data from the third distribution handler 250c The image reception information may be transmitted at a later time point than the image reception information.

The main handler 248 receives the image reception information transmitted from each of the plurality of distribution handlers 250a to 250d, determines a reproduction start time and a frame to be reproduced based on the received image reception information (S718) (I.e., information capable of identifying a frame to be reproduced) of the reproduction start time and the reproduction start time and the reproduction start time and the reproduction start time and the start time and the reproduction target division image data, (S720).

The plurality of distribution handlers 250a to 250d receive the image reproduction information transmitted from the main handler 248 and output the divided image data corresponding to the frame designated at the reproduction start time designated in the image reproduction information to the corresponding graphics adapters 210a to 210d simultaneously (S722).

Accordingly, by the above-described synchronization process, the image processing apparatus 200 according to the present invention can display the divided image data of the frame synchronized at the exactly synchronized time point with corresponding graphics adapters 210a- 210d to display on the multi-display device 100. FIG.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as a hard disk, a floppy disk, and a magnetic tape; optical media such as CD-ROM and DVD; magnetic recording media such as a floppy disk; Includes hardware devices specifically configured to store and perform program instructions such as megneto-optical media and ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

Claims (16)

A video processing method performed by an image processing apparatus for performing an image parallel rendering processing on a multi-display apparatus, the image processing apparatus including a plurality of graphic adapters for controlling display of a predetermined sub- The method comprising:
(a) comparing and matching display regions between an image source and the multi-display device, and based on display region matching results and sub-region setting information for each graphics adapter, dividing a source image to be displayed by each of the plurality of graphics adapters Determining a video area and allocating a video memory area in which divided video data for each graphics adapter is to be stored;
(b) receiving a source image frame from the image source and decoding the received source image frame into raw image data;
(c) storing corresponding divided image data of the decoded raw image data based on the determined divided image area for each graphic adapter in a video memory area allocated to the corresponding graphic adapter; And
(d) synchronizing the respective divided image data stored in the video memory area allocated for each of the graphic adapters on a frame-by-frame basis, and simultaneously transmitting the image data in parallel to the corresponding graphics adapter.
The method according to claim 1,
The step (a)
(a1) extracting a list of the plurality of graphics adapters connected to the multi-display device;
(a2) extracting or receiving display information for each of the plurality of graphic adapters, the display information for each of the plurality of graphic adapters including the sub-area setting information for each of the plurality of graphic adapters;
(a3) comparing the resolution of the source image from the image source with the full resolution of the multi-display device to determine a pixel ratio between the source image and the multi-display device; And
(a4) determining a divided image area of the source image to be displayed by each of the plurality of graphic adapters, based on the determined pixel ratio and the sub-area setting information for each of the graphics adapters, And allocating a video memory area in which data is to be stored.
The method according to claim 1,
Wherein the step (b) further comprises the step of providing a data time stamp (DTS) to the received source image frame based on the reception time of the source image frame.
The method of claim 3,
Wherein the step (c) stores each divided image data together with frame attribute information including the data time stamp in a video memory area assigned to a corresponding graphics adapter.
The method of claim 4,
Wherein the frame attribute information includes the data time stamp, a frame per second (FPS), and a sequence number.
The method of claim 4,
Wherein the step (d) comprises: receiving each divided image data and the frame attribute information stored in the video memory area allocated for each graphic adapter, and synchronizing the received divided image data on a frame-by-frame basis, And simultaneously transmitting in parallel to a corresponding graphics adapter.
The method of claim 6,
The step (d)
(d1) temporally synchronizing a main handler and a plurality of distributor handlers corresponding to each of the plurality of graphics adapters with each other;
(d2) receiving the divided image data and the frame attribute information from a video memory area allocated to a corresponding graphic adapter of each of the plurality of distribution handlers;
(d3) transmitting, by the plurality of distribution handlers, image reception information including the frame attribute information to the main handler;
(d4) receiving, by the main handler, the image reception information transmitted from each of the plurality of distribution handlers, generating image reproduction information based on the received image reception information, and transmitting the generated image reproduction information to each of the plurality of distribution handlers, The video playback information includes a playback start time, a data time stamp of the playback subject division video data, and a sequence number; And
(d5) receiving the image reproduction information transmitted from the main handler by the plurality of distribution handlers, and dividing the divided image data corresponding to the data time stamp and the sequence number at the reproduction start time on the basis of the received image reproduction information At the same time, in parallel to a corresponding graphics adapter.
The method of claim 7,
Wherein the plurality of distribution handlers are dynamically generated by the main handler.
An image processing apparatus for performing image parallel rendering processing on a multi-display apparatus,
A plurality of graphics adapters each controlling display of a predetermined sub-area of the multi-display device;
A decoding module that receives the source image frame from the image source and decodes the received source image frame into raw image data; And
A display area between the image source and the multi-display device is compared and matched to determine a divided image area of a source image to be displayed by each of the plurality of graphic adapters, A memory area allocated to each of the graphics adapters, and stores corresponding divided image data of the decoded raw image data based on the determined divided image area for each graphics adapter in a video memory area assigned to the corresponding graphic adapter, And an image distribution rendering module that synchronizes the respective divided image data stored in the memory area on a frame-by-frame basis and simultaneously transmits the image data in parallel to a corresponding graphics adapter.
The method of claim 9,
Wherein the image distribution rendering module comprises:
Determining a divided image area of the source image to be displayed by each of the plurality of graphic adapters based on a pixel ratio between the source image and the multi-display device and sub-area setting information per graphic adapter, A display area processing unit for allocating a video memory area in which the divided image data is to be stored;
A video memory in which a plurality of pieces of divided image data of decoded row image data are stored in respective video memory areas assigned to corresponding graphic adapters; And
Storing the corresponding divided image data of the decoded raw image data based on the determined divided image area of each graphic adapter in a video memory area allocated to the corresponding graphic adapter, And an image division processing unit for synchronizing the divided image data on a frame-by-frame basis and simultaneously transmitting them in parallel to a corresponding graphics adapter.
The method of claim 10,
Wherein the image segmentation processing unit gives a data time stamp (DTS) to the received source image frame based on a reception time of the source image frame.
The method of claim 11,
Wherein the image segmentation processing unit stores each divided image data together with frame attribute information including the data time stamp in a video memory area assigned to a corresponding graphics adapter.
The method of claim 12,
Wherein the frame attribute information includes the data time stamp, a frame per second (FPS), and a sequence number.
The method of claim 12,
Wherein the image division processing unit receives the divided image data and the frame attribute information stored in the video memory area allocated for each graphic adapter and synchronizes the received divided image data on a frame-by-frame basis based on the frame attribute information And at the same time to the corresponding graphics adapter in parallel.
15. The method of claim 14,
Wherein the image division processing unit comprises:
A plurality of distribution handlers corresponding to each of the plurality of graphics adapters, wherein each of the plurality of distribution handlers receives the divided image data and the frame attribute information from a video memory area assigned to a corresponding graphics adapter, Receiving the video reproduction information transmitted from the main handler, and transmitting the data time stamp and the sequence number at the reproduction start time based on the received video reproduction information to the main handler The plurality of distribution handlers transmitting the corresponding divided image data in parallel to a corresponding graphics adapter at the same time; And
Receiving image transmission information transmitted from each of the plurality of distribution handlers, generating image reproduction information including a reproduction start time, a data time stamp of the reproduction target division image data, and a sequence number based on the received image reception information, And a main handler for transmitting the main handler to each of the plurality of distribution handlers.
16. The method of claim 15,
Wherein the plurality of distribution handlers are dynamically generated by the main handler.

Images