JP2013066025A - Video display device and video display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video display device capable of displaying multi-viewpoint video having reduced breakdown.SOLUTION: A video display device includes: interpolation units 1-3 each for generating an interpolation video signal having a viewpoint positioned between a plurality of reference viewpoints; extrapolation units 4 and 5 each for generating an extrapolation video signal having a viewpoint positioned out of the reference viewpoints; pseudo virtual viewpoint video generation units 6 and 7, as an extrapolation unit, each for converting a video signal having one viewpoint generated based on the video signals having the reference viewpoints, to a stereoscopic video signal including left and right video signals in a pseudo manner, and setting one of the left and right video signals to an extrapolation video signal having a virtual viewpoint positioned out of the reference viewpoints; and projectors 11-19 each for projecting the video signal having each viewpoint.

Description

  The present invention relates to a video display device and a video display method for realizing stereoscopic viewing with the naked eye by presenting images from a plurality of viewpoints.

  In recent years, stereoscopic video (3D video) has become widespread. In order to view 3D images, liquid crystal shutters or dedicated glasses using polarized light are mainly used. On the other hand, development of autostereoscopic display that realizes stereoscopic vision with the naked eye without using dedicated glasses has been actively conducted. In the autostereoscopic display device, the viewing position is generally limited in order to perceive a 3D image.

  There are a plurality of methods for the autostereoscopic display device, and as a method in which the viewing position is not relatively limited, as disclosed in Patent Document 1, a multi-view that presents a large number of images according to the position of the viewpoint There is a method called an expression. In a multi-eye type autostereoscopic display device, images taken from many viewpoints are required. The 3D video that is widely used is two-viewpoint stereo content, and the two-viewpoint stereo content cannot be directly displayed on a multi-view autostereoscopic display device. Therefore, it is necessary to convert the stereo contents of two viewpoints into a multi-view video having three or more viewpoints.

JP 2010-81440 A JP 9-27969 A JP 2009-124308 A JP 2005-151534 A

  In order to convert two viewpoints of stereo content into three or more viewpoints of multi-viewpoint images, as described in Patent Documents 2 and 3, an intermediate viewpoint image is inserted between the left and right images of stereo contents. As described in Patent Document 3, it is necessary to generate and place a virtual viewpoint video outside the left and right videos outside the left and right videos of the stereo content, as described in Patent Document 3. . Inserting the interpolated video between the left and right videos of the stereo content is called interpolation, and placing the interpolated video outside the left and right videos is called extrapolation.

  However, it is difficult to generate a high-quality image of a virtual viewpoint outside the left and right images of stereo content. In particular, the video of the outside viewpoint that is generated tends to break down as it goes outward.

  In view of the above problems, the present invention displays a multi-view video with few failures when generating and displaying a multi-view video signal having a larger number of viewpoints based on video signals of a plurality of viewpoints. An object of the present invention is to provide a video display device and a video display method capable of performing the above.

  In order to solve the above-described problems of the related art, the present invention provides an interpolation unit (1-3) for generating video signals of viewpoints positioned between a plurality of reference viewpoints, and an outside of the reference viewpoint. An extrapolation unit (4-7, 61, 62, 71, 72) for generating a video signal of the viewpoint located at the position, and the video signal of the reference viewpoint, which is output from the interpolation unit and the extrapolation unit Projection units (11 to 19) for projecting video signals, respectively, and as the extrapolation unit, one of the reference viewpoint video signals or the reference viewpoint video signal among the reference viewpoint video signals. A virtual signal that is generated based on one viewpoint is converted into a stereoscopic video signal composed of left and right video signals in a pseudo manner, and one of the left and right video signals is positioned outside the reference viewpoint. Pseudo virtual viewpoint video generation for a video signal of various viewpoints To provide a video display apparatus characterized by the use of (6,7,61,62,71,72).

  In the video display device, a video signal of a viewpoint located on a side closer to the reference viewpoint among a plurality of viewpoints positioned outside the reference viewpoint is included in the video signal of the reference viewpoint. A first extrapolation unit (4, 5) generated by shifting the video signal of one viewpoint of the first viewpoint, and far from the reference viewpoint among a plurality of viewpoints located outside the reference viewpoint A video signal of a viewpoint located on the side can be configured to include a second extrapolation unit (6, 7) that is generated by the pseudo virtual viewpoint video generation unit.

  In the video display device described above, all the video signals of a plurality of viewpoints positioned outside the reference viewpoint can be generated by the pseudo virtual viewpoint video generation unit.

  In addition, in order to solve the above-described problems of the related art, the present invention generates an interpolated video signal of viewpoints positioned between a plurality of reference viewpoints, and selects one of the reference viewpoint video signals. One viewpoint video signal generated based on one viewpoint video signal or the reference viewpoint video signal is converted into a stereoscopic video signal consisting of left and right video signals in a pseudo manner, Is generated as an extrapolated video signal of a virtual viewpoint located outside the reference viewpoint, and the video signal of the reference viewpoint, the interpolated video signal, and the extrapolated video signal are respectively projected. Thus, a video display method characterized by presenting a stereoscopic video is provided.

  In the above video display method, an extrapolated video signal of a viewpoint located on the side closer to the reference viewpoint among a plurality of viewpoints located outside the reference viewpoint is converted into a video signal of the reference viewpoint. An extrapolated video signal of a viewpoint that is generated by shifting a video signal of one of the viewpoints and that is located on the far side of the reference viewpoint among a plurality of viewpoints that are located outside the reference viewpoint Can be generated as an extrapolated video signal of the virtual viewpoint.

  In the video display method described above, all extrapolated video signals of a plurality of viewpoints located outside the reference viewpoint can be generated as extrapolated video signals of the virtual viewpoint.

  According to the video display device and the video display method of the present invention, when generating and displaying a multi-view video signal that is a larger number of viewpoints based on the video signals of a plurality of viewpoints, a multi-view video with few failures is displayed. Can be displayed.

It is a block diagram which shows the video display apparatus of 1st Embodiment. It is a figure which shows each viewpoint of a multiview video. It is a block diagram which shows the specific structural example of the insertion parts 1-3 in FIG.1 and FIG.7. It is a block diagram which shows the specific structural example of the extrapolation parts 4 and 5 in FIG. It is a block diagram which shows the specific structural example of the pseudo virtual viewpoint image | video production | generation part 6, 7, 61, 62, 31-38 in FIG.1, FIG.7, FIG.8. It is a characteristic view which shows the relationship between the position of the viewpoint by 1st Embodiment, and the utilization efficiency of video information. It is a block diagram which shows the video display apparatus of 2nd Embodiment. It is a block diagram which shows the video display apparatus of a modification. It is a figure for demonstrating the parallax of a stereo pair. It is a figure for demonstrating the relationship between the position of a viewpoint, and interpolation and extrapolation. It is a figure for demonstrating the utilization efficiency of video information. It is a characteristic view which shows the relationship between the position of a general viewpoint, and the utilization efficiency of video information. FIG. 13 is a characteristic diagram showing the relationship between the viewpoint position and the use efficiency of video information when the horizontal parallax is 10% in FIG. 12.

  Embodiments of a video display apparatus and method according to the present invention will be described below with reference to the accompanying drawings. Before describing the configuration and operation of each embodiment, the reason why a virtual viewpoint video extrapolated to the left and right videos of stereo content tends to break down as it goes outwards will be described. Here, for the sake of simplification, as shown in FIG. 9, the left and right images are assumed to be planes, and description will be made using a stereo pair in which a constant horizontal parallax exists in the entire left and right images.

  In FIG. 9, the left video ImL and the right video ImR are shifted as horizontal parallax by n% when the horizontal width of the left video ImL and the right video ImR is 100%. In FIG. 10, the xy coordinate represents the video plane, and the b coordinate represents the position of the viewpoint. The left image ImL is located at the b coordinate 0, and the right image ImR is located at the b coordinate 100. In the range where the b coordinate exceeds 0 and less than 100, an interpolated image of the intermediate viewpoint is generated by interpolation, and in the range where the b coordinate is less than 0 and exceeds 100, an interpolated image of the outer viewpoint is generated by extrapolation. .

  In FIG. 10, a portion surrounded by a thick broken line indicates a common video portion for the left video ImL and the right video ImR. In the interpolation and extrapolation, it is assumed that the video of the left video ImL and the right video ImR is completely correctly collated in the common video portion to generate an interpolated video. Since the left video ImL and the right video ImR have n% horizontal parallax, the (100−n)% portions of the left video ImL and the right video ImR are common. In the interpolation, an interpolated video can be generated using all of the video portions that are common to (100−n)%. This is defined as video information being used with a usage efficiency of (100−n)%.

  The use efficiency of video information will be described with reference to FIG. FIG. 11 is a view in which the b-axis is horizontal and the x-axis is vertical when FIG. 10 is viewed from above the y-axis. The left image ImL is located at the b coordinate 0, and the right image ImR is located at the b coordinate 100. The thick solid line portion of the left image ImL and the right image ImR is a common image portion. Here, the use efficiency of the video information is (100−n)%. Extrapolation is performed at positions where the b coordinates are b1 and b2. At the position where the b coordinate is b1, the portion b1x outside the video width W cannot use the video information of the common video portion that can be collated. Similarly, at the position where the b coordinate is b2, the portion b2x outside the video width W cannot use the video information of the common video portion that can be collated.

  The use efficiency of the video information at the position where the b coordinate is b1 is −n × (b1-100) / 100 + 100−n by geometric calculation, and the use efficiency of the video information at the position where the b coordinate is b2 is calculated by geometric calculation. n × b2 / 100 + 100-n.

  When the usage efficiency of the video information is graphed as E, it is as shown in FIG. As shown in FIG. 12, the utilization efficiency E is a straight line having a characteristic represented by n × b / 100 + 100−n when the b coordinate is negative, and −n × (b -100) / 100 + 100-n. When the horizontal parallax n is 10%, the utilization efficiency E has the characteristics shown in FIG. As shown in FIG. 13, the utilization efficiency E is a straight line having a characteristic represented by 0.1 × b + 90 when the b coordinate is negative, and is represented by −0.1 × b + 90 when the b coordinate is 100 or more. It becomes a straight line of the characteristic. When the b coordinate is −100 and 200, the utilization efficiency E is 80%.

  FIG. 13 shows an ideal state. Actually, the utilization efficiency E deteriorates more rapidly as it goes outward. As a result of verification by the present inventor, it has been clarified that a severe failure does not occur from 0 to −50, from 100 to 150, and the utilization efficiency E is about 85%. However, when a multi-viewpoint video is generated and displayed based on two-viewpoint stereo content, it is required to display a virtual viewpoint video even in an area where the b coordinate exceeds −50 and 150. . Therefore, even when the viewpoint video is extrapolated to an area where the b coordinate exceeds −50 and 150, it is necessary to generate a viewpoint video with less failure.

  Hereinafter, a specific configuration and operation of each embodiment corresponding to such a request will be described.

<First Embodiment>
The video display device according to the first embodiment shown in FIG. 1 is a multi-view video display device as in Patent Document 1. In FIG. 1, the left channel signal SL of the stereo video is input to the interpolation units 1 to 3, the extrapolation units 4 and 5, and the projector 13. The right channel signal SR of the stereo video is input to the interpolation units 1 to 3, the extrapolation units 4 and 5, and the projector 17. The projector 13 projects the left channel signal SL and presents an image of the left channel signal SL itself to the viewpoint PVC shown in FIG. The projector 17 projects the right channel signal SR and presents an image of the right channel signal SR itself at the viewpoint Pvg shown in FIG.

  The interpolation units 1 to 3 generate interpolation video signals Spvd to Spvf based on the left channel signal SL or the right channel signal SR, respectively, and supply them to the projectors 14 to 16. The projectors 14 to 16 project the input interpolated video signals Spvd to Spvf, and present the video based on the interpolated video signals Spvd to Spvf to the viewpoints Pvd to Pvf shown in FIG. As a method for generating the interpolated video signals Spvd to Spvf in the interpolating units 1 to 3, the method described in Patent Document 3 may be used.

  Specifically, the interpolation units 1 to 3 are configured as shown in FIG. In FIG. 3, the horizontal parallax calculation unit 21 receives a right channel signal SR and a left channel signal SL. The horizontal parallax calculation unit 21 calculates the horizontal parallax based on the right channel signal SR and the left channel signal SL. Information indicating the horizontal parallax calculated by the horizontal parallax calculation unit 21 is input to the video shift unit 22. The viewpoint position information Sivp is also input to the video shift unit 22. The video shift unit 22 generates the interpolated video signals Spvd to Spvf by shifting the right channel signal SR or the left channel signal SL by a pixel multiplied by a value corresponding to the viewpoint position information Sivp.

  For example, when a part of the video of the left channel signal SL has a parallax d with respect to the corresponding part of the right channel signal SR and the interpolated video signal Spvd at the viewpoint Pvd is generated, the viewpoint Pvc and the viewpoint Pvd Is a quarter of the distance between the viewpoint Pvc and the viewpoint Pvg, and therefore, it is only necessary to shift a part of the video of the left channel signal SL by ¼ × d. When generating the interpolated video signal Spve at the viewpoint Pve, since the distance between the viewpoint Pvc and the viewpoint Pve is ½ of the distance between the viewpoint Pvc and the viewpoint Pvg, a part of the video of the signal SL of the left channel May be shifted by ½ × d.

  In the case of generating the interpolated video signal Spvf at the viewpoint Pvf, the image quality is better when it is generated based on the signal SR of the right channel having a closer physical distance. Accordingly, a parallax d 'between a part of the video of the right channel signal SR and a corresponding part of the left channel signal SL is obtained. Since the distance between the viewpoint Pvg and the viewpoint Pvf is 1/4 of the distance between the viewpoint Pvg and the viewpoint Pvc, it is only necessary to shift a part of the video of the right channel signal SR by ¼ × d ′. Since the viewpoint Pve is located at the center between the viewpoint Pvc and the viewpoint Pvg, when generating the interpolated video signal Spve at the viewpoint Pve, the viewpoint Pve may be generated based on the right channel signal SR.

  The extrapolating units 4 and 5 generate extrapolated video signals Spvb and Spvh based on the left channel signal SL or the right channel signal SR, respectively, and supply them to the projectors 12 and 18. The projectors 12 and 18 project the input extrapolated video signals Spvb and Spvh, and present the video based on the extrapolated video signals Spvb and Spvh to the viewpoints Pvb and Pvh shown in FIG.

  The specific configuration of the extrapolating units 4 and 5 is the same as that of the interpolating units 1 to 3, and includes a horizontal parallax calculating unit 23 and a video shifting unit 24 as shown in FIG.

  For example, when a part of the video of the left channel signal SL has a parallax d with respect to the corresponding part of the right channel signal SR and the extrapolated video signal Spvb at the viewpoint Pvb is generated, the viewpoint Pvc and the viewpoint Pvb Is a quarter of the distance between the viewpoint Pvc and the viewpoint Pvg, and is opposite to the viewpoint between the viewpoints Pvc and Pvg with respect to the viewpoint Pvc, so that a part of the image of the signal SL of the left channel is obtained. It is sufficient to shift by −1 / 4 × d.

  When the extrapolated video signal Spvh at the viewpoint Pvh is generated, the image quality is better when it is generated based on the signal SR of the right channel that is closer to the physical distance. Accordingly, a parallax d 'between a part of the video of the right channel signal SR and a corresponding part of the left channel signal SL is obtained. The distance between the viewpoint Pvg and the viewpoint Pvh is 1/4 of the distance between the viewpoint Pvg and the viewpoint Pvc, and is opposite to the viewpoint between the viewpoints Pvc and Pvg with respect to the viewpoint Pvg. It is only necessary to shift a part of the video of −1 / 4 × d ′.

  The viewpoint Pvb is preferably -50 to 0 in the b coordinate. The viewpoint Pvh is preferably 100 to 150 in the b coordinate.

  An extrapolated video signal Spvb at the viewpoint Pvb output from the extrapolation unit 4 is input to the pseudo virtual viewpoint video generation unit 6 configured as an extrapolation unit. An extrapolated video signal Spvh at the viewpoint Pvh output from the extrapolation unit 5 is input to the pseudo virtual viewpoint video generation unit 7 configured as an extrapolation unit. The pseudo virtual viewpoint video generation units 6 and 7 generate pseudo virtual viewpoint video signals Spva and Spvi and supply them to the projectors 11 and 19. The projectors 11 and 19 project the input virtual viewpoint video signals Spva and Spvi, and present the video based on the virtual viewpoint video signals Spva and Spvi to the viewpoints Pva and Pvi shown in FIG.

  The pseudo virtual viewpoint video generation units 6 and 7 can be realized by using a pseudo-stereoscopic image creation apparatus as described in Patent Document 4 called so-called 2D3D conversion. The pseudo-stereoscopic image creating apparatus described in Patent Document 4 creates depth estimation data from a non-stereoscopic video even if it is a non-stereoscopic video for which depth information is not given explicitly or implicitly, A pseudo stereoscopic image can be created based on the depth estimation data.

  As shown in FIG. 5, the pseudo virtual viewpoint video generation units 6 and 7 include a depth estimation unit 25 and a video shift unit 26. The depth estimation unit 25 estimates the video depth based on the input extrapolated video signals Spvb and Spvh, and generates depth estimation data. At this time, the depth estimation unit 25 estimates the depth approximated by the curved surface, and generates depth estimation data. The video shift unit 26 generates virtual viewpoint video signals Spva and Spvi by shifting the extrapolated video signals Spvb and Spvh according to the curved surface approximated depth estimation data output from the depth estimation unit 25.

  According to the pseudo virtual viewpoint video generation units 6 and 7, the distance of the virtual viewpoint is increased compared to a simple shift in which video is collated and extrapolated as in the extrapolation units 4 and 5. However, it is possible to avoid a rapid decrease in the use efficiency of the video information. As a result, high-quality virtual viewpoint video signals Spva and Spvi can be generated.

  The operation of the pseudo virtual viewpoint video generation unit 6 will be described in more detail. Assuming that the position of the viewpoint Pvb is the position of the right eye, what is displayed in front of the screen is closer to the inside (nose side) of the person viewing the video. Accordingly, the video shift unit 26 of the pseudo virtual viewpoint video generation unit 6 moves the video of the corresponding part inward (rightward) by an amount corresponding to the depth indicated by the depth estimation data. What is displayed deeper than the screen is closer to the viewer who sees the image. Accordingly, the video shift unit 26 moves the video of the corresponding portion outward (leftward) by an amount corresponding to the depth indicated by the depth estimation data.

  The pseudo virtual viewpoint video generation unit 6 directly uses the extrapolated video signal Spvb as the right channel signal after 2D3D conversion, and uses the virtual viewpoint video signal Spva at the left viewpoint Spva as the left channel signal from the extrapolated video signal Spvb. Generate as Since it is not necessary to output the extrapolated video signal Spvb that is the right channel signal after 2D3D conversion, the pseudo virtual viewpoint video generation unit 6 outputs only the virtual viewpoint video signal Spva to the projector 11.

  The operation of the pseudo virtual viewpoint video generation unit 7 will be described in more detail. Assuming that the viewpoint Pvh position is the position of the left eye, what is displayed in front of the screen is closer to the inside (nose side) of the person viewing the video. Accordingly, the video shift unit 26 of the pseudo virtual viewpoint video generation unit 7 moves the video of the corresponding portion inward (leftward) by an amount corresponding to the depth indicated by the depth estimation data. What is displayed deeper than the screen is closer to the viewer who sees the image. Accordingly, the video shift unit 26 moves the video of the corresponding portion outward (rightward) by an amount corresponding to the depth indicated by the depth estimation data.

  The pseudo virtual viewpoint video generation unit 7 uses the extrapolated video signal Spvh as it is as the left channel signal after 2D3D conversion, and uses the virtual viewpoint video signal Spvi at the viewpoint Pvi from the right of the extrapolated video signal Spvh as the signal of the right channel. Generate. Since it is not necessary to output the extrapolated video signal Spvh that is the left channel signal after 2D3D conversion, the pseudo virtual viewpoint video generation unit 7 outputs only the virtual viewpoint video signal Spvi to the projector 19.

  Projectors 11 to 19 as projection units project the input video signals of the respective viewpoints. The light rays projected from the projectors 11 to 19 are collected by the lens 20. In FIG. 1, only light rays projected from the projectors 13, 15, and 17 are schematically shown for simplification. The observer views the 3D image projected from the projectors 11 to 19 and condensed by the lens 20 with the left and right eyes EL and ER.

  FIG. 6 shows the utilization efficiency E according to this embodiment. FIG. 6 also shows the characteristics where n is 10%. In the portion where the b coordinate is 0 to −50, a straight line having a characteristic represented by 0.1 × b + 90 is the same as in FIG. 13, but the decrease in the utilization efficiency E is suppressed in the portion where the b coordinate exceeds −50. I understand that. In addition, in the part where the b coordinate is 100 to 150, as in FIG. 13, it is a straight line having a characteristic represented by −0.1 × b + 90. However, in the part where the b coordinate exceeds 150, a decrease in the utilization efficiency E is suppressed. I understand that When the b coordinate is −100 and 200, the utilization efficiency E is greater than 80%.

  In the present embodiment, the number of viewpoints is not limited to nine viewpoints shown in FIG. The number of interpolation units, extrapolation units, and pseudo virtual viewpoint video generation units may be further increased. The interval between the viewpoints may not be equal. Note that the video shift direction (corresponding to convergence) and the amount (corresponding to the intensity of three-dimensionalization) may be appropriately adjusted so as to be close to the actual appearance of the video. In FIG. 2, viewpoints Pvc and Pvg are reference viewpoints. The reference viewpoint is not limited to two viewpoints.

  Interpolators 1 to 3 in FIG. 1 are interpolation units that generate viewpoint video signals located between reference viewpoints based on a plurality of reference viewpoint video signals. The extrapolation units 4 and 5 in FIG. 1 are extrapolation units that generate video signals of viewpoints located outside the reference viewpoint based on video signals of a plurality of viewpoints. The pseudo virtual viewpoint video generation units 6 and 7 are also extrapolation units that generate video signals of viewpoints located outside the reference viewpoint.

  The pseudo virtual viewpoint video generation units 6 and 7 generate one viewpoint video signals Spvb and Spvh generated based on the reference viewpoint video signals SR and SL, and a stereoscopic video signal consisting of pseudo left and right video signals. Thus, one of the left and right video signals is a video signal of a virtual viewpoint located outside the viewpoints Pvc and Pvg as a reference and further outside the viewpoints Pvb and Pvh.

  The first embodiment generates a plurality of extrapolation units that generate video signals of a plurality of viewpoints located on the left outer side of the reference viewpoint, and generates video signals of a plurality of viewpoints located on the right outer side of the reference viewpoint. And a plurality of extrapolation units. Among the plurality of extrapolation units, the extrapolation units 4 and 5 serve as a reference for a video signal of a viewpoint located on the side closer to the reference viewpoint among a plurality of viewpoints located outside the reference viewpoint. It is the 1st extrapolation part produced | generated by shifting the video signal of one viewpoint among the video signals of a viewpoint. Among the plurality of extrapolation units, the pseudo virtual viewpoint video generation units 6 and 7 generate video signals of viewpoints located on the side far from the reference viewpoint among the plurality of viewpoints located outside the reference viewpoint. It is the 2nd extrapolation part produced | generated using 2D3D conversion.

Second Embodiment
A video display apparatus according to the second embodiment will be described with reference to FIG. 7, the same parts as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted as appropriate. In the second embodiment shown in FIG. 7, the extrapolated video signals Spvb and Spvh are not generated by the extrapolating units 4 and 5 in FIG. 1, but the extrapolated video signal Spvb is generated by the pseudo virtual viewpoint video generating units 61 and 71. , Spvh is generated.

  In FIG. 7, the left channel signal SL is input to the pseudo virtual viewpoint video generation units 61 and 62, and the right channel signal SR is input to the pseudo virtual viewpoint video generation units 71 and 72. The specific configuration of the pseudo virtual viewpoint video generation units 61, 62, 71, 72 is the same as that shown in FIG.

  The pseudo virtual viewpoint video generation unit 61 generates a virtual viewpoint video signal Spvb2 to be presented to the viewpoint Pvb based on the left channel signal SL, and the pseudo virtual viewpoint video generation unit 71 generates a right channel signal SR. Based on, a virtual viewpoint video signal Spvh2 to be presented to the viewpoint Pvh is generated. The pseudo virtual viewpoint video generation unit 62 generates a virtual viewpoint video signal Spva2 for presentation to the viewpoint Pva based on the left channel signal SL, and the pseudo virtual viewpoint video generation unit 72 generates the right channel signal SR. Based on the above, a virtual viewpoint video signal Spvi2 to be presented to the viewpoint Pvi is generated.

  The pseudo virtual viewpoint video generation units 61, 62, 71, and 72 are also extrapolation units that generate video signals of viewpoints located outside the reference viewpoint. The pseudo virtual viewpoint video generation units 61, 62, 71, 72 convert one viewpoint video signal out of the reference viewpoint video signals SR, SL into a stereoscopic video signal consisting of pseudo left and right video signals. In addition, one of the left and right video signals is a video signal of a virtual viewpoint positioned outside the reference viewpoints Pvc and Pvg.

<Modification>
A video display device according to a modification will be described with reference to FIG. 8, the same parts as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted as appropriate. The modification shown in FIG. 8 generates video signals to be presented to all the viewpoints Pva to Pvi in FIG. 2 using the pseudo virtual viewpoint video generation units 31 to 38. The modification is a configuration when the input video signal is not a stereo video signal but a 2D video video signal. The specific configuration of the pseudo virtual viewpoint video generation units 31 to 38 is the same as that shown in FIG.

In FIG. 8, the 2D video signal S _2D is input to the pseudo virtual viewpoint video generation units 31 to 38. The pseudo virtual viewpoint video generation units 31 to 34 generate virtual viewpoint video signals Spva3 to Spvd3 for presentation to the viewpoints Pva to Pvd and output them to the projectors 11 to 14. 2D video signal S _2D is supplied to the projector 15 as a video signal to be presented to it viewpoint Pve. The pseudo virtual viewpoint video generation units 35 to 38 generate virtual viewpoint video signals Spvf3 to Spvi3 for presentation to the viewpoints Pvf to Pvi, and output them to the projectors 16 to 19.

In a variant, the congestion is constant for all of the video signal (2D image signal S _2D and the virtual viewpoint video signal Spva3~Spvi3). It is preferable to increase the strength of three-dimensionalization toward the end of the viewpoint in FIG.

  The pseudo virtual viewpoint video generation units 31 to 38 convert the 2D video signal into a stereoscopic video signal composed of left and right video signals based on the 2D video signal of one viewpoint serving as a reference, and the left and right video signals. One of the video signals is a video signal of a virtual viewpoint positioned outside one viewpoint as a reference.

  The present invention is not limited to the above-described embodiments (modifications), and various modifications can be made without departing from the scope of the present invention.

1-3 Interpolation unit 4,5 Extrapolation unit 6,7,61,62,71,72 Pseudo virtual viewpoint video generation unit (extrapolation unit)
11-19 Projector (projection unit)
20 lenses

Claims (6)

An interpolation unit that generates video signals of viewpoints positioned between a plurality of reference viewpoints;
An extrapolation unit that generates a video signal of a viewpoint located outside the reference viewpoint;
A projection unit for projecting the video signal of the reference viewpoint and the video signal output from the interpolation unit and the extrapolation unit, respectively;
With
As the extrapolation unit, one viewpoint video signal generated from the viewpoint video signal of the reference viewpoint or one viewpoint video signal generated based on the reference viewpoint video signal is simulated as left and right A pseudo virtual viewpoint video generation unit that converts the video signal into a stereoscopic video signal and uses one of the left and right video signals as a video signal of a virtual viewpoint located outside the reference viewpoint; A video display device characterized by the above. Among the plurality of viewpoints located outside the reference viewpoint, a viewpoint video signal located on the side closer to the reference viewpoint is selected as one of the viewpoint video signals. A first extrapolation generated by shifting
A second extrapolation unit that generates, from the plurality of viewpoints located outside the reference viewpoint, a video signal of a viewpoint located farther from the reference viewpoint by the pseudo virtual viewpoint video generation unit; When,
The video display device according to claim 1, further comprising:   The video display apparatus according to claim 1, wherein the pseudo virtual viewpoint video generation unit generates all video signals of a plurality of viewpoints located outside the reference viewpoint. Generate interpolated video signals of viewpoints located between multiple reference viewpoints,
A stereoscopic video composed of left and right video signals in a pseudo manner from a video signal of one viewpoint among the video signals of the reference viewpoint or a video signal of one viewpoint generated based on the video signal of the reference viewpoint. Converted into a signal, one of the left and right video signals is generated as an extrapolated video signal of a virtual viewpoint located outside the reference viewpoint,
A video display method characterized by presenting a stereoscopic video by projecting the video signal of the reference viewpoint, the interpolated video signal, and the extrapolated video signal, respectively. An extrapolated video signal of a viewpoint located on the side closer to the reference viewpoint among a plurality of viewpoints located outside the reference viewpoint is selected from one of the viewpoint video signals of the reference viewpoint. Generated by shifting the video signal,
Generating an extrapolated video signal of a viewpoint located farther from the reference viewpoint among a plurality of viewpoints located outside the reference viewpoint as the extrapolated video signal of the virtual viewpoint. The video display method according to claim 4, wherein:   5. The video display method according to claim 4, wherein all the extrapolated video signals of a plurality of viewpoints located outside the reference viewpoint are generated as extrapolated video signals of the virtual viewpoint.

Images