KR20100060330A - Apparatus and method for creating multi-view image for stereo image - Google Patents

Abstract

PURPOSE: An apparatus and a method for creating a multi-view image for a stereo image are provided to generate multi-view video in consideration of left and right video thereby improving picture of output multi-view video. CONSTITUTION: A multi-view image generating device(100) comprises a time difference information extractor(101), a view location determination unit(102) and a multi-view image generator(103). The view location determination unit determines view location at the multi-viewpoint. The left and right video of the stereo video is outputted at each view point. The multi-view image generator generates the multi-view image from the stereo video based on the position of the viewpoint.

Description

Apparatus and method for generating multiview image for stereo image {APPARATUS AND METHOD FOR CREATING MULTI-VIEW IMAGE FOR STEREO IMAGE}

One embodiment of the present invention relates to the field of image processing, 3D display for generating a multi-view viewpoint image.

Recently, multi-view displays have been developed to express three-dimensional objects. However, there are not many multiview 3D images generated for multiview display. Most 3D images are in stereo form so that they can be optimally represented on a stereo type display. As a result, when a multi-view display is gradually generalized, an image processing technique capable of representing a stereo image on a multi-view display may be required.

In this case, since the stereo image is not optimized for multi-view display, deterioration of image quality may occur when the multi-view image is generated from the stereo image. For example, multi-view displays such as LCDs, lenticular sheets, or parallax barriers have constraints that can represent images. Due to this limitation, the depth of the image can be reduced without causing visual fatigue, and 3D representation may be impossible due to severe image distortion.

Accordingly, there is a need for a technique of generating a multiview image from a stereo image in consideration of the physical characteristics of the multiview display.

The multi-view image generating apparatus according to an embodiment of the present invention is based on a view position determiner and a view position determiner for determining a view position to output each of the left image and the right image of the stereo image in the multiview image; It may include a multi-view image generating unit for generating at least one multi-view image from the stereo image.

According to an aspect of the present invention, the multi-view image generating apparatus may further include a disparity information extraction unit for extracting disparity information for each pixel of the left image and the right image of the stereo image.

According to an aspect of the present invention, the view position determiner may determine the view position in the multiview image by using the first maximum disparity of the stereo image and the second maximum disparity that can be expressed by the multiview display.

According to an aspect of the present invention, the multi-view image generating unit is a first view image generating unit for generating at least one first view image corresponding to the left view position of the view position of the left image, and the view position of the left image and A second view image generator which generates at least one second view image corresponding to view positions between the view positions of the right image, and at least one third view image corresponding to the right view position of the view position of the right image It may include a third view image generating unit for generating a.

The multi-view image generating method according to an embodiment of the present invention comprises the steps of determining the view (view) position to be output each of the left image and the right image of the stereo image in the multi-view image from the stereo image based on the view position The method may include generating at least one multi-view image.

The apparatus for generating a multiview image according to an embodiment of the present invention may improve the image quality of the output multiview image by generating a multiview image in consideration of both the left image and the right image of the stereo image.

The multi-view image generating apparatus according to an embodiment of the present invention determines the view position of the stereo image by considering the parallax information of the stereo image and the parallax information of the multi-view display representing the multi-view image, thereby generating a natural 3D multi-view image. Can be generated.

Hereinafter, with reference to the contents described in the accompanying drawings will be described in detail an embodiment according to the present invention. However, the present invention is not limited to or limited by the embodiments. Like reference numerals in the drawings denote like elements. The multi-view image generating method according to an embodiment of the present invention may be performed by the multi-view image generating apparatus.

1 is a block diagram showing the overall configuration of a multi-view image generating apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the apparatus 100 for generating a multiview image 100 may include a parallax information extractor 101, a view point determiner 102, and a multiview image generator 103.

The parallax information extractor 101 may extract parallax information for each pixel of the left image and the right image of the stereo image. In this case, the disparity information may mean a disparity value for each pixel of each of the left image and the right image. For example, the parallax information extractor 101 may extract parallax information by using a disparity map of each of the left image and the right image. Here, the disparity diagram may mean the position difference information generated according to the disparity of the left image and the right image with respect to the same object.

As shown in FIG. 1, when the disparity diagrams for the left and right images of the stereo image are input, the disparity information does not need to be separately extracted.

The view position determiner 102 may determine a view position at which each of the left and right images of the stereo image is output from the multiview image. As an example, the viewpoint position determiner may determine the viewpoint position in the multiview image using parallax information of one of the left image and the right image.

In this case, the viewpoint location determiner 102 may determine the viewpoint position in the multiview image in consideration of the maximum parallax of the stereo image and the maximum parallax that can be expressed by the multiview display. In FIG. 1, NL may mean a viewpoint position of a left image and NR may mean a viewpoint position of a right image. The apparatus for generating a multiview image according to an embodiment of the present invention may generate a multiview image by using both images of a stereo image and considering them as physical features of a multiview display representing a multiview image.

The multi-view image generator 103 may generate a multi-view image from the stereo image based on the viewpoint positions of the left image and the right image. In detail, the multiview image generator 103 may generate a view image corresponding to a view position emptied through the left image and the right image in consideration of the view positions of the left image and the right image in the multiview image.

A detailed operation of the multiview image generator 103 will be described with reference to FIGS. 4 to 7.

2 is a diagram for describing an operation of a disparity information extracting unit of a multiview image generating apparatus according to an exemplary embodiment.

Referring to FIG. 2, a stereo image composed of a left image 201 and a right image 202 may be input to the disparity information extractor 101. At this time, it can be seen that a position difference occurs with respect to the position of the star in the left image 201 and the right image 202. Here, referring to the left image 201, the star is positioned at a distance D1 from the center to the right, and referring to the right image 202, the star is positioned at a distance D2 from the center to the left. D1 and D2 differ only in direction and may be the same size.

The parallax information extractor 101 generates a parallax for each of the left image 201 and the right image 202, and extracts parallax information 203 and 204 for each of the left image 201 and the right image. have. For example, the disparity information extractor 101 may extract a disparity diagram for each of the left image 201 and the right image 202 using a stereo matching method from the stereo image. Alternatively, the disparity diagram of the stereo image may be input to the disparity information extractor 101 from the beginning. In this case, the disparity information may mean disparity value of the stereo image.

Referring to FIG. 2, the parallax information extracting unit 101 may extract parallax D1, which is parallax information of the left image 201, and extract parallax D2, which is parallax information of the right image 202.

3 is a diagram for describing an operation of a viewpoint positioning unit of a multiview image generating apparatus according to an exemplary embodiment of the present invention.

The viewpoint position determiner 102 may determine a viewpoint position at which each of the left image 301 and the right image 302 of the stereo image is output from the multiview image. In this case, the viewpoint position determiner 102 may determine the viewpoint position in the multiview image by using parallax information of one of the left image 301 and the right image 302.

For example, the viewpoint position determiner 102 uses the first maximum parallax of the stereo image and the second maximum parallax that can be represented by the multiview display, respectively, to the left image 301 and the right image 302 in the multiview image. The viewpoint position with respect to

Referring to FIG. 3, the parallax for the left image 301 and the parallax for the right image 302 may be input to the viewpoint location determiner 102.

For example, the viewpoint location determiner 102 may determine the first maximum parallax of the stereo image through parallax with respect to the left image 301 or parallax with respect to the right image 302. In this case, the first maximum parallax may refer to a maximum parallax represented by a pixel interval in the stereo image.

Then, the viewpoint position determiner 102 may calculate the number of viewpoints existing between the left image and the right image by comparing the first maximum parallax of the stereo image with the second maximum parallax that can be expressed by the multiview display. For example, the viewpoint location determiner 102 may calculate the number of viewpoints according to Equation 1 below.

는 제1 최대 시차를 의미하고,

는 제2 최대 시차를 의미한다.Here, NoV means the number of viewpoints between the left image and the right image. And,

Means the first maximum parallax,

Denotes a second maximum parallax.

For example, the viewpoint position determiner 102 may determine the viewpoint position NL of the left image and the viewpoint position NR of the right image according to Equation 2 below.

는 다시점 영상(303)에 대한 최대 시점 개수를 의미한다. 예 를 들어,

가 4이고,

가 10인 경우, NL은 3이고, NR은 7으로 결정된다. 즉, 10개의 시점을 표현할 수 있는 다시점 영상(303)에서 왼쪽 영상은 3번 시점 위치로 결정되고, 오른쪽 영상은 7번 시점 위치로 결정된다. 이 때,

가

보다 큰 경우, NL=1 또는 NR=

로 결정될 수 있다. 본 발명의 일실시예는

가

보다 작은 경우를 위주로 설명된다.here,

Denotes the maximum number of viewpoints for the multiview image 303. E.g,

Is 4,

Is 10, NL is 3 and NR is determined to be 7. That is, in the multiview image 303 capable of expressing ten viewpoints, the left image is determined to be the third viewpoint position, and the right image is determined to be the seventh viewpoint position. At this time,

end

If greater than NL = 1 or NR =

Can be determined. One embodiment of the present invention

end

The smaller case is explained mainly.

The viewpoint position determiner 102 may set an output viewpoint position for stereo images input for each frame or use a fixed value for each program by using Equations 1 and 2 above. However, when the output view position of the input stereo images changes for each frame, the viewing distance may suddenly change. When the output view position of the stereo images is fixed for each program, the stereoscopic sense of the image may be reduced.

The viewpoint position determiner 102 naturally changes the output viewpoint position in consideration of the depth characteristic change for each frame, thereby more effectively expressing a three-dimensional effect in which an object pops out or enters backward.

For example, the viewpoint location determiner 102 may determine the viewpoint positions of the stereo images using the maximum disparity for each scene. In this case, the viewpoint position determiner 102 may determine the viewpoint position such that the stereo image input for the specific scene is output at the same viewpoint position.

For example, when a new scene is input, the viewpoint position determiner 102 sets the Input_Dmax value, which is the parallax of the input frame of the input scene, to the Scene_Max value, which is the maximum parallax for each scene, and then uses the Scene_Max value to set the viewpoint position. You can decide. At this time, when the Input_Dmax value of the next frame is larger than the stored Scene_Max value, the viewpoint positioning unit 102 may determine the new viewpoint position by resetting the Input_Dmax value of the next frame to the Scene_Max value of the scene.

If the process is continuously performed until the scene changes, the Scene_Max value of the input scene may converge to a specific value. As a result, the Scene_Max value is fixed to a specific value, and according to an embodiment of the present invention, it is possible to provide a stereoscopic feeling optimized for scene characteristics without changing the viewpoint position during a specific scene. That is, the viewpoint position determiner 201 may set the Scene_Max value of the converged scene to Input_Dmax of Equation 1 to determine the viewpoint position of the stereo images.

For example, after setting reference blocks at fixed positions in an image, a change in a scene may be determined according to color differences of color reference blocks between frames. At this time, when the color value difference is larger than the preset threshold value, the viewpoint position determiner 201 may determine that the scene has changed. If it is determined that the scene has changed, the viewpoint position determiner 201 may reset the Scene_Max value.

As another example, the view position determiner 201 may determine the view position of the current frame by using view position information of a previous frame and a subsequent frame with respect to the current frame.

For example, it is assumed that the frame for which the viewpoint position is to be determined as the current frame f. In addition, it is assumed that the previous frame f-n and the subsequent frame f + n to be referred to when determining the start position of the current frame. Then, the viewpoint position determiner 201 may determine the viewpoint position of the current frame by comparing Df-n, Df, and Df + n which are the previous frame, the current frame, and the next frame maximum parallax Input_Dmax. For example, the viewpoint position determiner 201 may determine the viewpoint position of the current frame according to the following process. The following process is only an example and may be changed according to the configuration of the system.

A. When Df is smaller than Df-n and Df + n, the viewpoint positioning unit 201 may determine the viewpoint position of the previous frame as the viewpoint position of the current frame.

B. When Df is greater than Df-n and Df + n, the viewpoint positioning unit 201 may determine the viewpoint position of the previous frame as the viewpoint position of the current frame. In this case, crosstalk may be caused when the parallax per view exceeds the maximum value. Even if crosstalk is induced, in order to maximize the stereoscopic effect of the output image, the view position determiner 201 may determine the view position of the previous frame as the view position of the current frame.

C. When Df is larger than Df-n and smaller than Df + n, or when Df is smaller than Df-n and larger than Df + n, the viewpoint positioning unit 201 uses the viewpoint position of the previous frame and the subsequent frame. Instead, the viewpoint position may be determined according to Equation 1.

In this case, when determining the start position of the current frame, n may be any value for the previous frame f-n to be used and the f + n subsequent frame.

4 is a block diagram illustrating a detailed configuration of a multiview image generating unit of a multiview image generating apparatus according to an embodiment of the present invention.

Referring to FIG. 4, the multiview image generator 103 includes a first view image generator 401, a second view image generator 402, and a third view image generator 403.

The first view image generator 401 may generate at least one first view image corresponding to the left view position of the view position of the left image. Referring to the example of FIG. 3, when the view position of the left image is determined to be the position 3 point of view, the first view image generator 401 may view the first view point position 2 and the left view point position n <NL of the left image. A first viewpoint image corresponding to the first viewpoint position may be generated. In this case, the first viewpoint image generator 401 may generate the first viewpoint image from the left image by using extrapolation.

The second view image generator 402 may generate at least one second view image corresponding to the view position between the view position of the left image and the view position of the right image. Referring to the example of FIG. 3, when the view position of the left image is determined to be the view 3 point and the view position of the right image is determined to be the view 7 point, the second view image generator 402 may display the left image and the right. The second view image corresponding to the 4th, 5th, and 6th view positions between NL <n <NR may be generated. In this case, the second viewpoint image generator 402 may generate a second viewpoint image from the left image and the right image by using interpolation.

The third view image generator 403 may generate at least one third view image corresponding to the right view position of the view position of the right image. Referring to the example of FIG. 3, when the view position of the right image is determined to be the position 7 point, the third view image generator 403 may use the 8th, 9th and 9th points of the right view position (n> NR) of the right image. A third view image corresponding to the view point 10 may be generated. In this case, the third view image generator 403 may generate a third view image from the right image by using extrapolation.

5 is a diagram for describing a process of generating a first view image from a left image of a stereo image according to an embodiment of the present invention.

The first view image generator 401 may move the pixel value of the left image according to the parallax information of the left image (S501). In operation S502, the first view image generator 401 may generate a first view image while moving the pixel value of the left image. For example, the first view image generator 401 may generate a first view image according to Equation 3 below.

는 생성되는 제1 시점 영상의 각 픽셀값을 의미하고,

는 왼쪽 영상의 각 픽셀값을 의미한다.

은 픽셀값의 이동 정도를 의미하며,

은 시차가 발생한 오브젝트를 구성하는 픽셀의 시차(D1)을 나타낸다. 그리고, 본 발명에서 왼쪽으로 이동하는 시차를 (+)로 정의한다면, 수학식 3에서

은 -1을 나타낸다. here,

Denotes each pixel value of the generated first view image,

Denotes each pixel value of the left image.

Is the amount of shift in pixel values,

Denotes a parallax D1 of pixels constituting an object in which parallax has occurred. And, if the parallax moving to the left in the present invention is defined as (+),

Represents -1.

That is, the pixels constituting the parallax in the left image are moved to the right. If the left image is the view point 3, the first view image corresponding to the view point 1 and the view 2 position is generated, and according to Equation 3, the degree of movement of the pixel value is the view point and the left image, respectively. Is determined by the time difference. As a result, the first viewpoint image generator 401 may generate at least one first viewpoint image from the left image through extrapolation.

When the pixel value is shifted according to the parallax information, an area without image information may exist in the generated first view image. Then, the first viewpoint image generator 401 may fill an area without image information (S403). In this case, the first view image generator 401 may fill the region by using the surrounding information (eg, adjacent background information) of the region without the image information.

6 is a diagram for describing a process of generating a second view image from a left image and a right image of a stereo image according to an embodiment of the present invention.

The second view image generator 402 may move the pixel value of the left image according to the parallax information between the left image and the left image (S601). In addition, the second view image generator 402 may move the pixel value of the right image according to the disparity information between the right image and the right image (S601).

Then, the second view image generator 402 may generate a left view image with respect to the left image (S602). In operation S603, the second view image generator 402 may generate a right view image of the right image. The second view image generator 402 may generate a second view image corresponding to a view position between the left image and the right image through the left view image and the right view image (S604). For example, the second view image generator 402 may generate a second view image according to Equation 4 below.

는 왼쪽 영상을 의미하고,

는 왼쪽 영상에 대한 왼쪽 시점 영상을 의미한다. 그리고,

는 오른쪽 영상을 의미하고,

는 오른쪽 시점 영상을 의미한다.

과

은 각각 왼쪽 영상과 오른쪽 영상에서의 픽셀 이동 정도를 의미한다. 또한,

과

은 각각 왼쪽 영상과 오른쪽 영상에 대한 시차 정보(시차)를 의미한다.here,

Means the left image,

Denotes a left view image for the left image. And,

Means the right image,

Means the right view image.

and

Denotes the degree of pixel movement in the left image and the right image, respectively. Also,

and

Denotes parallax information (disparity) for the left image and the right image, respectively.

Referring to Equation 4, the second viewpoint image generator 402 compares image values with respect to pixel positions of the generated left viewpoint image and the right viewpoint image, so that both the left viewpoint image and the right viewpoint image have image values. If so, the second view image may be generated by averaging the image values. In addition, when only one of the two images has an image value, the second view image generator 402 may determine the image having the image value as the second view image. As a result, the second view image generator 402 may generate a second view image from the left image and the right image through interpolation.

In the case of generating the second view image, since both the left image and the right image are considered, a region without image information is not required, and thus a process of filling the region may not be necessary.

7 is a diagram for describing a process of generating a third view image from a right image of a stereo image according to an embodiment of the present invention.

The third view image generator 403 may move the pixel value of the right image according to the parallax information of the right image (S701). The third view image generator 401 may generate a third view image while moving the pixel value of the right image (S702). For example, the first view image generator 401 may generate a first view image according to Equation 5 below.

는 생성되는 제3 시점 영상의 각 픽셀값을 의미하고,

는 오른쪽 영상의 각 픽셀값을 의미한다.

은 픽셀 값의 이동 정도를 의미하며,

은 시차가 발생한 오브젝트를 구성하는 픽셀의 시차(D2)을 나타낸다. 그리고, 본 발명에서 왼쪽으로 이동하는 시차를 (+)로 정의한다면, 수학식 5에서

은 +1을 나타낸다. here,

Means each pixel value of the generated third view image,

Denotes each pixel value of the right image.

Is the amount of shift in pixel values,

Denotes a parallax D2 of pixels constituting an object in which parallax has occurred. In the present invention, if the parallax shifting to the left is defined as (+),

Represents +1.

That is, the pixels constituting the parallax in the right image are moved to the left. If the image on the right is the view point 7, the third view image corresponding to the positions 8, 9, and 10 is generated. According to Equation 5, the degree of movement of the pixel value is the position of the view point and the right. It is determined by the parallax of the image. As a result, the third view image generator 403 may generate at least one third view image from the right image through the extrapolation method.

When the pixel value is shifted according to the parallax information, there may exist a region without image information in the generated third [1] view image. Then, like the first viewpoint image generator 401, the third viewpoint image generator 403 may fill an area without image information (S703). In this case, the third view image generator 403 may fill the region using the surrounding information (eg, adjacent background information) of the region without the image information.

8 is a diagram illustrating an entire process of generating a multiview image through a multiview image generating apparatus according to an embodiment of the present invention.

Referring to FIG. 8, the viewpoint location determiner 102 may determine viewpoint positions NL and NR of the left image 803 and the right image 804 from the multiview image 801. In this case, the viewpoint position determiner 102 may determine the viewpoint position using the parallax information of the left image 803 or the right image 804. In addition, the viewpoint position determiner 102 may determine the viewpoint position in consideration of disparity information of the display representing the multiview image 801.

Then, the multi-view image generator 103 may generate an image for the remaining view positions except for the view positions of the left image 803 and the right image 804 in the multi-view image 801. In detail, the multi-view image generator 103 may generate at least one first view image (including the view image 802) corresponding to the left view position of the left image 801. That is, the multi-view image generator 103 may generate at least one first view image from the left image through left view extrapolation. In this case, the multi-view image generator 103 may fill the image information through the surrounding information on the region in which the image information does not exist in the first view image.

The multi-view image generator 103 may generate at least one second view image corresponding to the view position between the view position of the left image 801 and the right image 803. In detail, the multiview image generator 103 may generate a left view image for the left image and a right view image for the right image, and then generate a second view image using the generated view image. That is, the multi-view image generator 103 may generate at least one second view image by using the left view image and the right view image through view interpolation.

Also, the multiview image generator 103 may generate at least one third view image (including the view image 805) corresponding to the right view position of the right image 804. That is, the multi-view image generator 103 may generate at least one third view image from the right image through right view extrapolation. In this case, the multi-view image generator 103 may fill the image information through the surrounding information on the region where the image information does not exist in the third view image.

9 is a flowchart illustrating an entire process of a method of generating a multiview image according to an embodiment of the present invention.

In operation S901, the apparatus for generating a multiview image may extract parallax information for each pixel of the left image and the right image of the stereo image. In this case, the parallax information may be extracted through a parallax of each of the left image and the right image. For example, the multi-view image generating apparatus may extract a parallax from a stereo image through a stereo matching method.

In operation S902, the apparatus for generating a multiview image may determine a viewpoint position at which each of the left image and the right image of the stereo image is output from the multiview image. That is, the apparatus for generating a multiview image may determine at what point in time the left image and the right image are output from the multiview image.

For example, the apparatus for generating a multiview image may determine a viewpoint position in the multiview image by using parallax information of one of the left image and the right image. In detail, the apparatus for generating a multiview image may determine the view position by using the first maximum parallax of the stereo image and the second maximum parallax that can be expressed by the multiview display. In this case, the multi-view image generating apparatus may calculate the number of viewpoints between the left image and the right image by comparing the first maximum parallax and the second maximum parallax. Then, the apparatus for generating a multiview image may determine the view positions of the left image and the right image with respect to the multiview image using the number of viewpoints and the second maximum parallax.

For example, the apparatus for generating a multiview image may determine the viewpoint position for each scene in the multiview image by using a first maximum parallax that is the maximum parallax for each scene of the stereo image and a second maximum parallax that may be expressed by the multiview display.

In this case, the multi-view image generating apparatus may detect a change in a scene for each frame based on a difference in color values between frames of the stereo image. For example, the apparatus for generating a multiview image may detect a change in a scene based on a difference in color values of reference blocks at fixed positions between frames. For example, when the color value difference of the reference block is larger than a preset threshold, the multi-view image generating apparatus may detect a scene change.

When the scene change occurs, the multi-view image generating apparatus may determine the first maximum parallax using the maximum parallax of the first frame in which the scene change occurs. Here, the multi-view image generating apparatus may set the maximum parallax of the first frame as the first maximum parallax. If the maximum parallax of the next frame is greater than the first maximum parallax, the first maximum parallax may be reestablished to the maximum parallax of the next frame. This process is repeated over and over to the frame where the scene change occurs. As a result, the first maximum parallax, which is the maximum parallax for each scene, converges to a specific value, so that a fixed view position for a specific scene may be determined.

Then, the multi-view image generating apparatus may calculate the number of viewpoints between the left image and the right image by comparing the first maximum parallax and the second maximum parallax. In addition, the apparatus for generating a multiview image may determine the view positions of the left image and the right image of the multiview image using the calculated number of views and the second maximum parallax.

For example, the apparatus for generating a multiview image may determine a viewpoint position of a current frame with reference to a previous frame and a subsequent frame with respect to the current frame for which the viewpoint position is to be determined. At this time, the viewpoint position of the current frame may be determined based on a result of comparing the maximum parallax of each of a frame before, after, and the current frame of the multiview image.

In operation S903, the apparatus for generating a multiview image may generate a multiview image from the stereo image based on the viewpoint position. For example, the multi-view image generating apparatus may generate at least one first view image corresponding to the left view position of the view position of the left image. That is, the multiview image generating apparatus may generate at least one first view image from the left image through left view extrapolation. In this case, an area without image information in the first view image may be filled with surrounding information.

The multi-view image generating apparatus may generate at least one second view image corresponding to the view position between the view position of the left image and the right image. In this case, the apparatus for generating a multiview image may determine at least one left view image for the left image and a right image view for the right image, and then generate at least one second view image using the left view image and the right view image. . That is, the multi-view image generating apparatus may generate at least one second view image from the left image and the right image through interpolation.

The multi-view image generating apparatus may generate at least one third view image corresponding to the right view position of the view position of the right image. That is, the multi-view image generating apparatus may generate at least one third view image from the right image through right view extrapolation. In this case, an area without image information in the third view image may be filled with surrounding information.

Then, the apparatus for generating a multiview image may generate a final multiview image by combining the generated at least one first view image, the second view image, and the third view image according to the characteristics of the multiview display.

Parts not described in FIG. 9 may refer to descriptions of FIGS. 1 to 8.

In addition, the multi-view image generating method according to an embodiment of the present invention includes a computer readable medium including program instructions for performing operations implemented by various computers. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The media may be program instructions that are specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Although one embodiment of the present invention as described above has been described by way of limited embodiments and drawings, it is not limited to the embodiment of the present invention, which is one of ordinary skill in the art to which one embodiment of the present invention belongs. Various modifications and variations are possible from the substrate. Therefore, the spirit of the present invention should be grasped only by the claims set out below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

1 is a block diagram showing the overall configuration of a multi-view image generating apparatus according to an embodiment of the present invention.

2 is a diagram for describing an operation of a disparity information extracting unit of a multiview image generating apparatus according to an exemplary embodiment.

3 is a diagram for describing an operation of a viewpoint positioning unit of a multiview image generating apparatus according to an exemplary embodiment of the present invention.

4 is a block diagram illustrating a detailed configuration of a multiview image generating unit of a multiview image generating apparatus according to an embodiment of the present invention.

5 is a diagram for describing a process of generating a first view image from a left image of a stereo image according to an embodiment of the present invention.

6 is a diagram for describing a process of generating a second view image from a left image and a right image of a stereo image according to an embodiment of the present invention.

7 is a diagram for describing a process of generating a third view image from a right image of a stereo image according to an embodiment of the present invention.

8 is a diagram illustrating an entire process of generating a multiview image through a multiview image generating apparatus according to an embodiment of the present invention.

9 is a flowchart illustrating an entire process of a method of generating a multiview image according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

100: multi-view video generator

101: parallax information extraction unit

102: viewpoint positioning unit

103: multi-view image generator

Claims (20)

A viewpoint position determiner which determines a view position at which a left image and a right image of the stereo image are output in the multiview image; And A multiview image generation unit generating a multiview image from the stereo image based on the viewpoint position A multi-view image generating device comprising a. The method of claim 1, A parallax information extraction unit for extracting disparity information for each pixel of the left and right images of the stereo image More, The viewpoint positioning unit, And a viewpoint position in the multiview image using the parallax information of one of the left image and the right image. The method of claim 1, The viewpoint positioning unit, And a viewpoint position in the multiview image using the first maximum parallax of the stereo image and a second maximum parallax that can be expressed by the multiview display. The method of claim 3, The viewpoint positioning unit, And determining the viewpoint position of each scene in the multiview image by using a first maximum parallax that is the maximum parallax for each scene of the stereo image and a second maximum parallax that can be expressed by a multiview display. The method of claim 4, wherein The viewpoint positioning unit, Detects a change in a scene for each frame based on a difference in color values between frames of the stereo image, And when the change of the scene occurs, determining the first maximum disparity using the maximum disparity of the first frame where the change of the scene occurs. The method of claim 5, The viewpoint positioning unit, Set the maximum parallax of the first frame to the first maximum parallax, And reconstructing the first maximum parallax by comparing the maximum parallax for each frame from the first frame to the frame where the scene change occurred. The method of claim 3, The viewpoint positioning unit, The number of viewpoints between the left image and the right image is calculated by comparing the first maximum parallax and the second maximum parallax, and the left image with respect to the multiview image using the number of viewpoints and the second maximum parallax. A multi-view image generating device for determining a view point of each of the right image. The method of claim 1, The viewpoint positioning unit, And a viewpoint position of the current frame with reference to a previous frame and a subsequent frame with respect to the current frame for which the viewpoint position is to be determined. The method of claim 8, The viewpoint positioning unit, And a viewpoint position of the current frame is determined according to a result of comparing the maximum parallax of each of the previous frame, the next frame, and the current frame. The method of claim 1, The multi-view image generator, A first viewpoint image generator configured to generate at least one first viewpoint image corresponding to a left viewpoint position of a viewpoint position of the left image; A second viewpoint image generator configured to generate at least one second viewpoint image corresponding to the viewpoint position between the viewpoint position of the left image and the viewpoint position of the right image; And A third viewpoint image generator configured to generate at least one third viewpoint image corresponding to the right viewpoint position of the viewpoint position of the right image A multi-view image generating device comprising a. Determining a viewpoint position at which a left image and a right image of the stereo image are respectively output from the multiview image; And Generating a multiview image from the stereo image based on the viewpoint position Including a multi-view image generation method. The method of claim 11, Extracting disparity information for each pixel of each of the left and right images of the stereo image; More, The determining of the viewpoint position may include: And determining a viewpoint position in the multiview image by using parallax information of one of the left image and the right image. The method of claim 11, The determining of the viewpoint position may include: And determining a viewpoint position in the multiview image by using a first maximum parallax of the stereo image and a second maximum parallax that can be expressed by a multiview display. The method of claim 13, The determining of the viewpoint position may include: And determining a viewpoint position per scene in the multiview image by using a first maximum parallax that is the maximum parallax for each scene of the stereo image and a second maximum parallax that can be expressed by a multiview display. The method of claim 14, The determining of the viewpoint position may include: Detecting a change in a scene for each frame based on a difference in color values between frames of the stereo image; And When the change of the scene occurs, determining the first maximum disparity using the maximum disparity of the first frame where the change of the scene occurs; Including a multi-view image generation method. The method of claim 15, The determining of the first maximum parallax may include: Setting a maximum parallax of the first frame to the first maximum parallax; And Resetting the first maximum parallax by comparing the maximum parallax for each frame from the first frame to the frame where the scene change occurred Including a multi-view image generation method. The method of claim 13, The determining of the viewpoint position may include: Calculating the number of viewpoints between the left image and the right image by comparing the first maximum parallax with the second maximum parallax; And Determining a viewpoint position of each of the left and right images with respect to a multiview image using the number of viewpoints and the second maximum parallax; Including a multi-view image generation method. The method of claim 11, The determining of the viewpoint position may include: And determining a viewpoint position of the current frame with reference to a previous frame and a subsequent frame with respect to the current frame for which the viewpoint position is to be determined. The method of claim 18, The determining of the viewpoint position may include: And determining a viewpoint position of the current frame according to a result of comparing the maximum parallax of each of the previous frame, the next frame, and the current frame. The method of claim 11, The generating of the at least one multi-view image may include: Generating at least one first view image corresponding to a left view position of a view position of the left image; Generating at least one second view image corresponding to the view position between the view position of the left image and the view position of the right image; And Generating at least one third view image corresponding to a right view position of the view position of the right image; Including a multi-view image generation method.

Images