TW201320002A - Ulti-view stereoscopic image generating method and multi-view stereoscopic image generating apparatus applying the same method - Google Patents

Abstract

A multi-view stereoscopic image generating method comprises the following steps: providing a first two-dimensional (2D) image and a first depth map corresponding to the first 2D image, wherein the first 2D image has a first visual angle; generating a second 2D image according to the first 2D image and the first depth map, wherein the second 2D image has a second visual angle, and the second visual angle and the first visual angle are adjacent to each other; generating a second depth map according to the second 2D image; and generating a third 2D image according to the second 2D image and the second depth map, wherein the third 2D image has a third visual angle, and the third visual angle and the second visual angle are adjacent to each other. In addition, a multi-view stereoscopic image generating apparatus applying the same method is also provided.

Description

Multi-view stereo image generation method and multi-view stereo image generation device using the same

The present invention relates to the technical field of multi-view stereoscopic images, and more particularly to a multi-view stereoscopic image generating method and a multi-view stereoscopic image generating device using the same.

In recent years, the technology of image processing has been continuously improved, representing the continuous pursuit of a more realistic visual experience. In addition to the general processing techniques of image and color, the processing of multi-view stereoscopic images further increases the visual experience of the three-dimensional space.

The current multi-view stereoscopic image technology utilizes a two-dimensional image and a depth map to generate a plurality of two-dimensional images with other viewing angles. However, this technique can cause serious image distortion problems, as illustrated in Figure 1.

FIG. 1 is a schematic diagram of a conventional multi-view stereo image generation technique. Referring to FIG. 1 , a depth image based rendering (DIBR) 100 receives a two-dimensional image view 3 of a first viewing angle and a depth image DM3 corresponding to the two-dimensional image view 3 , and according to the two To generate four 2D images with other perspectives, labeled as view1, view2, view4, and view5. In the four two-dimensional images generated, the two-dimensional images view2 and view4 are both adjacent to the view angle of the two-dimensional image view3, and the two-dimensional images view1 and view5 are respectively viewed from The 2D image view2 is adjacent to the view of view4.

Since the two-dimensional image view 2 re-projects the pixels in the two-dimensional image view 3, the two-dimensional image view 3 is displaced to the left by the preset displacement amount Δshift, so that the two-dimensional image view 2 is generated due to the difference between the shadowing effect and the viewing angle. Image gaps. Similarly, the 2D image view4 re-projects the pixels in the 2D image view3, so that the 2D image view3 is displaced to the right by the preset displacement amount Δshift, so the 2D image view4 will also be different due to the shadowing effect and the viewing angle. Produce image holes.

Because the above-mentioned two-dimensional image view1 re-projects the pixels in the two-dimensional image view3, so that the two-dimensional image view3 is displaced to the left by a preset displacement amount Δshift, the two-dimensional image view1 will have a shadowing effect and a viewing angle. The difference is that there are twice the size of the image holes. Similarly, since the above-mentioned two-dimensional image view 5 re-projects the pixels in the two-dimensional image view 3, so that the two-dimensional image view 3 is shifted to the right by a preset displacement amount Δ shift, the two-dimensional image view 5 is also shielded. The effect is different from the angle of view, resulting in twice the size of the image holes. As a result, the problem of severe distortion of multi-view stereoscopic images is caused.

As can be seen from the foregoing description, the more the angle of view of the generated two-dimensional image, the more severe the distortion of the multi-view stereo image.

The invention provides a multi-view stereo image generation method, and the multi-view stereo image generated according to the method does not have serious distortion.

The present invention provides a multi-view stereoscopic image generating device that adopts the multi-view stereoscopic image generating method described above.

The present invention provides a multi-view stereoscopic image generating method, including the steps of: providing a first two-dimensional image and a first depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image has a first viewing angle; The second two-dimensional image and the first depth map are used to generate a second two-dimensional image, wherein the second two-dimensional image has a second angle of view, and the second angle of view is adjacent to the first angle of view; generating a correspondence according to the second two-dimensional image a second depth map; and generating a third two-dimensional image according to the second two-dimensional image and the second depth image, wherein the third two-dimensional image has a third viewing angle, and the third viewing angle and the second viewing angle are adjacent viewing angles .

The present invention further provides a multi-view stereoscopic image generating device, including: a first depth image drawing unit, a first depth map generating unit, and a second depth image drawing unit. The first depth image drawing unit is configured to receive the first two-dimensional image and the first depth map corresponding to the first two-dimensional image, and generate the second two-dimensional image according to the first two-dimensional image and the first depth image, The first two-dimensional image has a first viewing angle, the second two-dimensional image has a second viewing angle, and the second viewing angle and the first viewing angle are adjacent viewing angles. The first depth map generating unit is configured to receive the second two-dimensional image, and accordingly generate a corresponding second depth map. The second depth image drawing unit is configured to receive the second two-dimensional image and the second depth image, and generate a third two-dimensional image according to the second two-dimensional image and the second depth image, wherein the third two-dimensional image The image has a third viewing angle, and the third viewing angle and the second viewing angle are adjacent viewing angles.

The method for solving the foregoing problem is to first generate a two-dimensional image having an adjacent viewing angle according to a two-dimensional image and a depth map corresponding to the two-dimensional image, and generate a corresponding image according to the generated two-dimensional image. The depth map is then further generated according to the generated two-dimensional image and the depth map corresponding to the two-dimensional images to further generate a two-dimensional image with adjacent viewing angles. Since each newly generated two-dimensional image is generated by a two-dimensional image having an adjacent viewing angle and its corresponding depth map, each newly generated two-dimensional image has a displacement amount that is only twice the preset displacement amount, and There is no preset multiple amount of displacement (for example, twice). Therefore, the multi-view stereo image generated in accordance with the method of the present invention does not have a serious image distortion problem.

The above and other objects, features and advantages of the present invention will become more <RTIgt;

2 is a schematic diagram of a multi-view stereoscopic image generating device according to an embodiment of the invention. Referring to FIG. 2 , the multi-view stereoscopic image generating device 20 includes a depth image drawing unit 200 , a depth map generating unit 210 , a depth image drawing unit 220 , a depth map generating unit 230 , and a depth image drawing unit 240 . The depth image drawing unit 200 is configured to receive a two-dimensional image view 3 having a first viewing angle and a depth map DM3 corresponding to the two-dimensional image view 3, and generate two-dimensional images view 2 and view 4 according to the two. The above two-dimensional images view3, view2 and view4 each have a viewing angle, and the viewing angles of the two-dimensional images view2 and view4 are adjacent to the viewing angle of the two-dimensional image view3, but the two-dimensional images view2 and view4 are two. The perspective of the person is not an adjacent perspective.

The depth map generating unit 210 is configured to receive the two-dimensional image view 2 having the second viewing angle, and accordingly generate a depth map DM2 corresponding to the two-dimensional image view 2 . In addition, the depth map generating unit 210 also transmits the two-dimensional image view 2 and the depth map DM2 to the depth image drawing unit 220. The depth image drawing unit 220 is configured to receive the two-dimensional image view 2 and the depth map DM2, and generate a two-dimensional image view 1 having a third viewing angle according to the two. The two-dimensional image view1 also has a viewing angle, and the viewing angle of the two-dimensional image view1 and the viewing angle of the two-dimensional image view2 are also adjacent viewing angles. Similarly, the depth map generating unit 230 is configured to receive the two-dimensional image view 4 having the fourth viewing angle, and accordingly generate a depth map DM4 corresponding to the two-dimensional image view 4. In addition, the depth map generation unit 230 also transmits the two-dimensional image view 4 and the depth map DM4 to the depth image drawing unit 240. The depth image drawing unit 240 is configured to receive the two-dimensional image view 4 and the depth map DM4, and generate a two-dimensional image view 5 having a fifth viewing angle according to the two. The two-dimensional image view 5 also has a viewing angle, and the viewing angle of the two-dimensional image view 5 and the viewing angle of the two-dimensional image view 4 are also adjacent viewing angles.

As can be seen from the above description, since each newly generated two-dimensional image is generated by a two-dimensional image having an adjacent viewing angle and its corresponding depth map, the displacement of each newly generated two-dimensional image is only doubled. The preset displacement amount Δshift does not occur several times (for example, twice) of the preset displacement amount Δshift. Therefore, the multi-view stereo image generated in accordance with the method of the present invention does not have a serious image distortion problem.

It is worth mentioning that, in this example, the depth image drawing units 200, 220, and 240 all use depth-image-based rendering (DIBR) according to the received two-dimensional image and depth map. A new two-dimensional image with adjacent viewing angles is produced, which is not intended to limit the invention. In addition, although the above example is exemplified by generating five two-dimensional images having different viewing angles, it is not intended to limit the present invention. For example, if only two two-dimensional images having different viewing angles are needed, only the depth image drawing unit 200, the depth image generating unit 210, and the depth image drawing unit 220 are used to derive the two-dimensional image view2 according to the two-dimensional image view3. For example, the depth image drawing unit 200, the depth map generating unit 230, and the depth image drawing unit 240 may be used to derive the two-dimensional image view4 and view5 from the two-dimensional image view3. Of course, if six 2D images with different viewing angles are needed, or even more 2D images with different viewing angles, a new depth map generating unit and a depth image drawing unit may be added to at least according to the 2D images view1 and view5. One of them is to derive other two-dimensional images with different perspectives.

Through the description of the above implementations, those skilled in the art can summarize some basic operational steps of the above multi-view stereoscopic image generating device, as shown in FIG. 3 is a flow chart of a method for generating a multi-view stereoscopic image according to an embodiment of the invention. Referring to FIG. 3, the multi-view stereoscopic image generating method includes the following steps: first, providing a first two-dimensional image and a first depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image has a first viewing angle (eg, Step S31). And generating, according to the first two-dimensional image and the first depth map, the second two-dimensional image, wherein the second two-dimensional image has a second viewing angle, and the second viewing angle and the first viewing angle are adjacent viewing angles (step S32) Shown). Then, a corresponding second depth map is generated according to the second two-dimensional image (as shown in step S33). And generating a third two-dimensional image according to the second two-dimensional image and the second depth image, wherein the third two-dimensional image has a third viewing angle, and the third viewing angle and the second viewing angle are adjacent viewing angles (such as steps S34)).

In the above example, the depth image drawing unit 200 directly receives the two-dimensional image view 3 having the first viewing angle and the depth image DM3 corresponding to the two-dimensional image view 3, but is known in the art as a general knowledge. Before the depth image drawing unit 200 of 2, a depth map generating unit, a depth map generating unit 410 as shown in FIG. 4, may also be disposed. In FIG. 4, the same reference numerals as those of FIG. 2 are denoted as the same object. The depth map generating unit 410 shown in FIG. 4 can be used to receive the two-dimensional image view 3 having the first viewing angle, and accordingly generate a depth map DM3 corresponding to the two-dimensional image view3. Then, the depth map generating unit 410 transmits the two-dimensional image view 3 and the depth map DM3 to the depth image drawing unit 200.

Of course, those skilled in the art also know that before the depth image drawing unit 200 of FIG. 2, two depth images generating units can also receive two two-dimensional images with different viewing angles (which can be regarded as receiving a stereo image). A depth map generation unit 510 as shown in FIG. In FIG. 5, the same reference numerals as those of FIG. 2 are denoted as the same object. The multi-view stereoscopic image generating device shown in FIG. 5 is different from the multi-view stereoscopic image generating device 20 shown in FIG. 2 in that the depth map generating unit 510 is configured to receive a two-dimensional image view3- having a first viewing angle. 1 and a 2D image view 3-2 having a second viewing angle, and accordingly generating a depth map DM3-1 corresponding to the 2D image view 3-1, or generating a depth map DM3 corresponding to the 2D image view 3-2 -2. Then, the depth map generating unit 510 transmits the two-dimensional image view 3-1 and the depth map DM3-1, or transmits the two-dimensional image view 3-2 and the depth map DM3-2 to the depth image drawing unit 200.

In summary, the present invention solves the foregoing problems by first generating a two-dimensional image having an adjacent viewing angle according to a two-dimensional image and a depth map corresponding to the two-dimensional image, and based on the generated two-dimensional image. The image is used to generate a corresponding depth map, and then the two-dimensional image with the adjacent viewing angle is further generated according to the generated two-dimensional image and the depth map corresponding to the two-dimensional image. Since each newly generated two-dimensional image is generated by a two-dimensional image having an adjacent viewing angle and its corresponding depth map, each newly generated two-dimensional image has a displacement amount that is only twice the preset displacement amount, and There is no preset multiple amount of displacement (for example, twice). Therefore, the multi-view stereo image generated in accordance with the method of the present invention does not have a serious image distortion problem.

While the present invention has been described in its preferred embodiments, the present invention is not intended to limit the invention, and the present invention may be modified and modified without departing from the spirit and scope of the invention. The scope of protection is subject to the definition of the scope of the patent application.

20. . . Multi-view stereo image generating device

100, 200, 220, 240. . . Depth image drawing unit

210, 230, 410, 510. . . Depth map generation unit

View1, view2, view3, view4, view5, view3-1, view3-2. . . Two-dimensional image

DM2, DM3, DM4, DM3-1, DM3-2. . . Depth map

Δshift. . . Preset displacement

S31, S32, S33, S34. . . step

FIG. 1 is a schematic diagram of a conventional multi-view stereo image generation technique.

2 is a schematic diagram of a multi-view stereoscopic image generating device according to an embodiment of the invention.

3 is a flow chart of a method for generating a stereoscopic image of a view according to an embodiment of the invention.

4 is a schematic diagram of a multi-view stereoscopic image generating device according to another embodiment of the present invention.

FIG. 5 is a schematic diagram of a multi-view stereoscopic image generating apparatus according to another embodiment of the present invention.

S31, S32, S33, S34. . . step

Claims (12)

A method for generating a multi-view stereoscopic image, comprising: providing a first two-dimensional image and a first depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image has a first viewing angle; The second two-dimensional image and the first depth map are used to generate a second two-dimensional image, wherein the second two-dimensional image has a second viewing angle, and the second viewing angle and the first viewing angle are adjacent viewing angles; Generating a second two-dimensional image according to the second two-dimensional image; generating a third two-dimensional image according to the second two-dimensional image and the second depth image, wherein the third two-dimensional image has a third viewing angle, and The third viewing angle and the second viewing angle are adjacent viewing angles. The multi-view stereoscopic image generating method according to claim 1, wherein the second two-dimensional image is generated according to the first two-dimensional image and the first depth image by a depth image mapping method, and The depth image mapping method generates the third two-dimensional image according to the second two-dimensional image and the second depth map. The multi-view stereoscopic image generating method of claim 1, further comprising generating a fourth two-dimensional image according to the first two-dimensional image and the first depth image, wherein the fourth two-dimensional image has a fourth viewing angle, and the fourth viewing angle and the first viewing angle are also adjacent viewing angles. The multi-view stereoscopic image generating method according to claim 3, wherein the fourth two-dimensional image is generated according to the first two-dimensional image and the first depth map by a depth image mapping method. The method for generating a multi-view stereo image according to claim 3, further comprising: generating a corresponding third depth map according to the fourth two-dimensional image, and according to the fourth two-dimensional image and the third depth The figure generates a fifth two-dimensional image, wherein the fifth two-dimensional image has a fifth angle of view, and the fifth angle of view and the fourth angle of view are adjacent angles of view. The multi-view stereoscopic image generating method according to claim 5, wherein the fifth two-dimensional image is generated according to the fourth two-dimensional image and the third depth map by a depth image mapping method. A multi-view stereoscopic image generating device, comprising: a first depth image drawing unit, configured to receive a first two-dimensional image and a first depth map corresponding to the first two-dimensional image, and according to the first two-dimensional image The image and the first depth map are used to generate a second two-dimensional image, wherein the first two-dimensional image has a first viewing angle, the second two-dimensional image has a second viewing angle, and the second viewing angle and the first The viewing angle is an adjacent viewing angle; a first depth map generating unit is configured to receive the second two-dimensional image, and accordingly generate a corresponding one of the second depth maps; and a second depth image drawing unit to receive the Generating a second two-dimensional image and the second depth image, and generating a third two-dimensional image according to the second two-dimensional image and the second depth image, wherein the third two-dimensional image has a third angle of view, and The third perspective and the second perspective are adjacent perspectives. The multi-view stereoscopic image generating device of claim 7, wherein the first depth image drawing unit generates the first image by using a depth image mapping method according to the first two-dimensional image and the first depth image. The second two-dimensional image is generated by the second depth image drawing unit according to the second two-dimensional image and the second depth image. The multi-view stereoscopic image generating device of claim 7, wherein the first depth image drawing unit further generates a fourth two-dimensional image according to the first two-dimensional image and the first depth image, wherein the The fourth two-dimensional image has a fourth viewing angle, and the fourth viewing angle and the first viewing angle are also adjacent viewing angles. The multi-view stereoscopic image generating device of claim 9, wherein the first depth image drawing unit generates the first image by using a depth image mapping method according to the first two-dimensional image and the first depth image. Four two-dimensional images. The multi-view stereoscopic image generating device of claim 9, further comprising: a second depth map generating unit configured to generate a corresponding one of the third depth maps according to the fourth two-dimensional image; a third depth image drawing unit, configured to generate a fifth two-dimensional image according to the fourth two-dimensional image and the third depth image, wherein the fifth two-dimensional image has a fifth angle of view, and the fifth angle of view The fourth viewing angle is an adjacent viewing angle. The multi-view stereoscopic image generating device of claim 11, wherein the third depth image drawing unit generates the first image by using a depth image mapping method according to the fourth two-dimensional image and the third depth image. Five two-dimensional images.