KR101417712B1 - Rendering correction method for 3D image - Google Patents

Abstract

The present invention relates to a method for correcting rendering of a 3D image, and more specifically, to a method for correcting rendering of a 3D image, which is capable of automatically searching and correcting an error generated in the other frame rendered in a process of rendering each frame of a 2D image to synthesize a 3D image.

Description

[0001] Rendering correction method for 3D image [0002]

The present invention relates to a 3D image rendering and correction method, and more particularly, to a 3D image rendering method and a 3D image rendering method, which are capable of automatically detecting and correcting errors generated in a rendered frame in a process of rendering each frame of a 2D image, And more particularly, to a rendering correction method for an image.

With the development of digital technology and the diversification of broadcasting media by the convergence of broadcasting and communication, additional services related to broadcasting using the characteristics of digital technology are newly introduced. Currently, the development direction of video services such as TV is going to be high-definition and large-screen, but since 2D image contents are still provided, viewers can not feel stereoscopic effect through current image contents.

As a result, the necessity of stereoscopic image is increasing gradually, but the contents of the stereoscopic image have not been enough yet. Stereoscopic image processing technology is a core technology in the next generation information and communication service field, and it is a cutting-edge technology in which competition for technology development is intense along with development in the information industry society. Such stereoscopic image processing technology is an essential element for providing high quality image service in multimedia applications. Today, the application fields such as broadcasting, medical treatment, education, military, game and virtual reality are diversified as well as the information communication field have.

Accordingly, in the past, techniques for converting 2D images or images into 3D images or images and applying them to various fields have been proposed. For example, Patent Publication No. 2011-013142 (published on December 14, 2011) proposes a technique for a stereoscopic image conversion method using depth map information, and patent registration No. 1291071 (registered on March 23, 2013) A method and an apparatus for improving a stereoscopic image error are proposed in which the stereoscopic effect is not caused by an error of the depth map information.

However, in the conventional technique, a process of converting a 2D image into a 3D image is performed by converting a parallax according to a time difference between frames into a binocular parallax to give a stereoscopic effect. In the case of a horizontally moving frame, This advantage, however, does not provide enough information for stereoscopic effect formation for a stationary background or object.

In particular, conventionally, one side frame is transformed into a depth map in a 2D image, and then another side frame in which a viewpoint is shifted in one side frame is rendered, and the one side frame and the other side frame are synthesized and converted into a 3D image. Conventionally, however, in the rendering process of one frame as described above, since a pixel representing an object included in one frame is moved and broken, a hole A '(see FIG. Therefore, in the related art, it is necessary to correct the hole generated in the other frame generated by rendering the one frame. However, conventionally, the operator has to check the hole as the naked eye and manually correct the hole.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a method and apparatus for automatically detecting and correcting errors generated in a second frame rendered by rendering one frame of a 2D image, And a rendering correction method of the 3D image.

The present invention includes the following embodiments in order to achieve the above object.

A preferred embodiment of the 3D image rendering correction method according to the present invention is a one-frame output step of outputting one frame in a 2D image; A depth map output step of generating and outputting a depth map representing a distance between each object in the one frame; A right frame output step of rendering the depth map and the one frame and generating and outputting the other frame; A channel image output step of generating and outputting a channel image in which an object of the other frame is compared with one frame to represent the same object as black and white; A checking step of outputting a detection signal of an error area when an object not included in the one frame is identified in the channel image; And an error correction step of, if the error area is identified in the verification step, setting the boundary line of the error area as a reference line and symmetrically copying the area including the same object to the error area, A baseline setting step of setting a boundary between black and white as a reference line in the channel image; An error area calculating step of calculating an area of the error area; A copy area setting step of setting a copy area including an object representing the same object at a position symmetrical with the error area around the baseline; A copying step of mirroring the copy area set in the copy area setting step and attaching the copied area to the error area; And a storing step of storing the other frame after the copying step.

delete

delete

delete

In another embodiment of the present invention, the copying step may include: a comparison step of comparing areas of the error area and the copy area; And repeatedly copying and pasting the copy area if it is determined that the area of the copy area is small after the comparing step.

The present invention is capable of automatically checking and correcting an error occurring in the process of rendering each frame of a 2D image, thereby shortening the working time and improving the convenience of the operator.

1 is a diagram for explaining an outline of a rendering correction method for a 3D image according to the present invention,
FIG. 2 is a block diagram illustrating a rendering correction system of a 3D image according to the present invention.
3 is a flowchart illustrating a rendering correction method of a 3D image according to the present invention.
4 is a flowchart showing an error correction step in a 3D image rendering correction method according to the present invention;
5 is a diagram showing a left frame extracted from a 2D image,
6 is a photograph showing the depth map of the left frame,
7A is a photograph showing the right frame,
Figure 7b is a partial enlarged view of Figure 7a,
8 is a photograph showing the channel image of the right frame,
9 is a view showing a correction area of a right frame.

Hereinafter, a preferred embodiment of a 3D image rendering correction method according to the present invention will be described with reference to the accompanying drawings.

1 is a diagram for explaining an outline of a 3D image rendering and correction method according to the present invention.

1, a 3D image rendering correction method according to the present invention includes extracting one frame as an image in a 2D image, moving pixels of the object included in the one frame, generating and displaying another frame, Synthesized and converted into a 3D image.

At this time, one frame of the 2D image is subjected to a rendering process, and another frame shown at the other view is generated. Here, the one frame and the other frame are named as a left frame and a right frame according to the moving direction of the pixels of the frame extracted from the 2D image after the rendering.

Hereinafter, a frame before rendering is referred to as a left frame, and a frame after rendering is referred to as a right frame.

The left frame is transformed into a right frame through rendering by referring to a depth map expressing a difference in distance between the objects by the lightness difference. At this time, the rendering generates the right frame by moving each pixel constituting the left frame in one direction according to the difference in brightness by the depth map. Here, each object of the right frame is damaged in the rendering process, resulting in a hole A '(see FIG. 7B).

Accordingly, in order to automatically check and correct an error area in which the hole A 'is generated, the present invention compares the left frame and the right frame so as to set an error area as to whether or not an object that is not matched is included, If an area including the same object exists at a position symmetrical with the area, the area is mirrored and pasted to the error area. Here, the present invention is characterized in that the verification and symmetry of the error region are automatically performed.

The detailed configuration will be described with reference to Fig. 2 is a block diagram illustrating a rendering correction system of a 3D image according to the present invention.

Referring to FIG. 2, a rendering correction system for a 3D image according to the present invention includes an input unit 11 for outputting an operation command, a rendering unit 12 for rendering a 2D image frame, A correcting unit 13 for checking whether an error has occurred in the right frame of the frame, a correcting unit 14 for symmetrically correcting the error region detected by the accepting unit 13, data for storing a left frame and a right frame, And controls the rendering unit 12 according to an operation command of the input unit 11 to generate a right frame with reference to the left frame, A control unit 16 for controlling the display unit 14, and a display 17.

The input unit 11 applies a command to the control unit 16 by a combination of keys operated by a user. Here, the input unit 11 selects a left frame and a depth map calculated from the 2D image stored in the database 15, and outputs a driving instruction of the rendering unit 12.

The rendering unit 12 includes a shift module 121 for generating a right frame, a channel module 122 for generating a channel image, and a depth module 123 for generating a depth map of the left frame.

The depth module 123 converts a selected frame of the 2D image into a depth map representing a distance as a black and white image.

The shift module 121 generates an image representing the right frame by moving each object included in the left frame to the right. Here, the right frame is completed by moving a pixel of the left frame referring to the distance difference represented by the depth map.

The channel module 122 compares the objects of the right frame and the left frame, and generates a channel image that represents the difference object in black and white. At this time, the channel image is represented by white when the left frame and the right frame are compared with each other, and when the different areas are included between the two frames, the channel image is expressed in black. That is, the channel image compares the left frame and the right frame with each other to see whether they contain the same data (e.g., pixels), and displays the difference in black.

However, the channel image may be black in the same object included in both the left frame and the right frame as opposed to the above, and the object in which the difference is generated may be expressed in white. The setting of black and white according to the choice of designer and operator corresponds to any one of various embodiments.

The acceptance unit 13 confirms whether there is a black or white region in the channel image output from the channel module 122 and applies the same to the controller 16.

Here, the channel image is shifted in one direction in a frame of a 2D image, and the object is compared between the images of the generated frame, and the difference is expressed as black and white. Therefore, the acceptance unit 13 senses that a region displayed in black is generated in the channel image, and applies the sensed region to the controller 16.

Under the control of the control unit 16, the correcting unit 14 corrects an error by confirming an area indicated by black in the channel image in the right frame, and by symmetrically correcting pixels in the corresponding area. At this time, the correcting unit 14 confirms the area of the area where the hole A 'is generated and its area, and a region where no error has occurred at a position symmetric with the error area with respect to the set reference line is referred to as a copy area A2, see Fig. 9), and corrects the error by symmetrically copying the pixels of the copy area. That is, the correction unit corrects the area of the symmetric position of the error area by copying (symmetric copying) and pasting the area of the symmetric position of the error area to the error area A1 (see FIG. 9) where the hole A 'is generated.

The control unit 16 selects a left frame calculated from the 2D image stored in the database 15 and outputs the selected left frame to the display 17 when the selection command of the input unit 11 is applied, And controls the rendering unit 12 to generate a right frame and a channel image, respectively. The control unit 16 controls the correcting unit 14 to correct the error area in which the hole A 'is generated when the acceptance signal of the accepting unit 13 is received.

In addition, the controller 16 controls the display 17 to output the left frame, the right frame, the depth map, and the channel image, respectively.

The present invention includes the above-described configuration. Hereinafter, a method of rendering and correcting a 3D image according to the present invention will be described in detail with reference to FIG. 3 to FIG.

3 is a flowchart illustrating a rendering correction method of a 3D image according to the present invention.

3, a left frame output step S11 for outputting a 2D image frame as an image, a depth map output step S12 for outputting a depth map of a left frame of the 2D image, A right frame output step (S13) of rendering a 2D frame and outputting a right frame, a channel image output step (S14) of calculating a channel image of the right frame, comparing the channel image with a right frame, (S15) for inspecting the error region (A1) (see FIG. 9) where the error region is generated, and an error correction step (S16) for symmetrically correcting the error region, ).

The left frame output step S11 is a step of outputting the left frame extracted from the 2D image. The input unit 11 applies a selection signal of a left frame obtained by converting a frame of a 2D image into an image by a user's operation. Accordingly, the control unit 16 receives the control signal of the input unit 11 and outputs the left frame stored in the database 15 to the display 17. The left frame is as shown in Fig.

In the depth map output step S12, the control unit 16 controls the rendering unit 12 to convert the left frame into a depth map. The depth map is as shown in FIG. 6 as an image expressing the distance difference by light and dark by converting the left frame into a monochrome image. Accordingly, when the command of the input unit 11 is applied, the controller 16 controls the rendering unit 12 to generate a depth map of the left frame. The rendering unit 12 drives the depth module 123 under the control of the control unit 16, converts the left frame into a depth map, and outputs the depth map to the display 17.

As shown in FIG. 6, the depth map is a black-and-white image in which each object included in the left frame of FIG. 5 is expressed in black and white, and the distances between the objects are represented by a contrast.

In the right frame output step S13, the controller 16 controls the rendering unit 12 to render the depth map and the left frame to generate a right frame. The rendering unit 12 generates the right frame by moving the pixel representing the object of the left frame to the right by driving the shift module 121 under the control of the controller 16. At this time, the shift module 121 generates right frames as shown in FIGS. 7A and 7B by moving each pixel to the right while maintaining a difference in distance between objects according to the difference in brightness of the depth map. The control unit 16 outputs the right frame generated by the rendering unit 12 to the display 17. [

In the channel image output step S14, the controller 16 controls the rendering unit 12 to compare the objects of the right frame and the left frame to determine whether the objects not included in the left frame are either black or white As shown in FIG. The channel module 122 converts the right frame into a monochrome image under the control of the controller 16, the same object as the left frame is expressed in white, and the object having a difference from the left frame (for example, A new object or an object with a difference in shape) is displayed in black. The channel image is as shown in FIGS. 8 and 9. FIG.

The verification step S15 is a step in which the verification unit 13 confirms a region indicated by black in the channel image generated in the channel image output step under the control of the control unit 16. [ The channel image is set to represent the same object between the left frame and the right frame as white, and the different objects as black. Accordingly, the channel image displays the same objects in the left frame and the right frame in white, and displays different objects in black. Such a channel image is shown in FIG. 8 as an example. The acceptance unit 13 confirms whether there is an object represented by black in the channel image and its area.

An example of the hole A 'has been introduced in Figs. 7A and 7B. That is, an error occurs in the process of moving each pixel in the rendering step and the left frame appears to be stretched like the enlarged area of FIG. 7B. The hole A 'is displayed in black in the channel image since it is an object that was not in the left frame or added to the right frame in the rendering process.

The error correcting step S16 is a step of correcting the error area A1 by controlling the correcting part 14 when the control part 16 detects that the hole A 'is generated in the right frame in the verification step S15 . The error correction step S16 will be described below with reference to FIG. 4 and FIG. Here, FIG. 9 shows an example of the repair process of the area 'A' in FIGS. 7A, 7B and 8 as an example.

4 is a flowchart illustrating an error correction step in a 3D image rendering correction method according to the present invention.

Referring to FIG. 4, the error correction step may include a reference line setting step S161 of setting a boundary line between black and white in the channel image as a reference line C (see FIG. 9), an error An error area calculating step (S162) of calculating an area of the area (A1); a copy area setting step (S163) of setting a copy area (A2) having the same area as the area calculated in the error area calculating step (S162) A copying step S164 of copying the copy area A2 set in the copy area setting step S163 and attaching the copy area A2 to the error area A1; and a copying step S164 of storing the right frame corrected after the copying step S164 Step S165.

The reference line setting step S161 is a step of setting the reference line C in the correcting unit 14. As shown in FIG. 9, the correcting unit 14 sets the boundary line between the area represented by black and the area represented by white in the channel image as the reference line C.

The error region calculation step S162 is a step of calculating the area of the area represented by black in the channel image by the correction unit 14. [ The error area A1 corresponds to the area of the entire area where the error of the pixel data occurs.

The copying area setting step S163 is a step of setting the copying area A2 having the same area as the area calculated in the error area calculating step S162 in the correcting part 14. The copy area A2 is a region having the same pixel data as the error area A1 (the area having the same object in the left frame) around the reference line C as the center. More specifically, the copy area A2 is set in the right frame or the left frame among the same objects as the error area A1.

The copying step S164 is a step of copying the copy area A2 and attaching the copy area A2 to the error area A1. The correcting unit 14 sets a copy area A2 among the areas symmetric with the error area A1 with respect to the reference line C to express the same object as the error area A1, (A2) is copied and pasted to the error area (A1).

If the area of the object represented by the same pixel as the error area A1 is smaller than the error area A1, it is preferable to repeat the copying step S164.

That is, if the area of the copy area A2 is smaller than the error area A1, the copying step S164 repeatedly copies and pastes the pixels of the copy area into the error area.

To this end, the copying step S164 includes a comparison step (not shown) for comparing the area of the error area A1 with the area of the copy area A2, and an area of the copy area after the comparison step (not shown) It is preferable to further include a pasting step (not shown) for performing repetitive pasting.

In the comparing step, the correcting unit 14 refers to the left frame around the reference line C to calculate an area of the copying area A2 having the same pixel at a position symmetrical to the error area A1 , And comparing the area of the copy area A2 with the area of the error area A1.

If the area of the copy area A2 is greater than or equal to the error area A1 in the comparison step, the copying and pasting is performed as described above, or if the entire area of the copy area A2 is the error area (A1), it is repeatedly performed until the error area A1 is filled up.

In the storing step S165, when the correcting unit 14 has completed the pasting of the copying area A2 with the error area A1 in the copying step S164, . In addition, the controller 16 outputs the execution result of the error correction step S16 of the correction unit 14 to the display 17.

As described above, according to the present invention, since an error generated in the 2D image rendering process is automatically checked and restored, the work time of the operator can be reduced, and the degree of difficulty of the work can be reduced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

11: input unit 12: rendering unit
13: acceptance unit 14:
15: Database 16:
17: Display 121: Shift module
122: channel module 123: depth module
A1: Error area A2: Copy area
C: Baseline

Claims (5)

delete delete A one-frame output step of outputting one frame from the 2D image;
A depth map output step of generating and outputting a depth map representing a distance between each object in the one frame;
A right frame output step of rendering the depth map and the one frame and generating and outputting the other frame;
A channel image output step of generating and outputting a channel image in which an object of the other frame is compared with one frame to represent the same object as black and white;
A checking step of outputting a detection signal of an error area when an object not included in the one frame is identified in the channel image; And
And an error correction step of, if the error area is identified in the verification step, setting the boundary line of the error area as a reference line, and mirroring the area including the same object and attaching the area to the error area,
The error correction step
A reference line setting step of setting a boundary between black and white as a reference line in the channel image;
An error area calculating step of calculating an area of the error area;
A copy area setting step of setting a copy area including an object representing the same object at a position symmetrical with the error area around the baseline;
A copying step of mirroring the copy area set in the copy area setting step and attaching the copied area to the error area; And
And storing the other frame after the copying step.
delete 4. The method of claim 3, wherein the copying step
A comparison step of comparing the area of the error area with the area of the copy area; And
Further comprising the step of repeatedly copying and pasting the copy area if it is determined that the area of the copy area is small after the comparing step.

Images