JP4631812B2 - Multi-view video display device, multi-view video imaging device, and multi-view video generation device - Google Patents

Description

The present invention relates to a multi-view video display device that displays images from different viewpoints according to the viewpoint of the observer, a multi-view video imaging device that captures a multi-view video displayed on the multi-view video display device, and a multi-view The present invention relates to a multi-view video generation apparatus that generates video .

  Active research has been conducted on glasses-free 3D displays that can observe 3D images without wearing special glasses such as anaglyph glasses, polarized glasses, and liquid crystal shutter glasses. Since this method can display an image obtained by photographing an object from multiple directions with a camera, it has a feature that a stereoscopic image can be viewed from different viewpoints according to the movement of the observer's viewing position. As a result of the research, for example, the following Non-Patent Document 1 discloses “8-eye glasses-free 3D television” using the principle of the lenticular method.

<Principle of lenticular 3D image display method>
The principle of the lenticular method is that a plurality of lens portions arranged in the horizontal direction are each photographed from multiple directions on one focal plane of the plurality of lens portions using a lenticular screen having a semi-cylindrical shape. When the images are arranged in a stripe shape, the light from the stripe image is refracted by the directivity of the lenticular lens, and it is a mechanism that enables the naked-eye stereoscopic viewing when observed at an optimal position. FIG. 3 shows the principle of a conventional three-dimensional video display device using an eight-lens lenticular method for viewing a three-dimensional video from eight different viewpoints. In FIG. 3, a multi-view stripe element image display surface 22 is a multi-view stripe element image (pixels 23, 24, 25, 26) observed from eight viewpoints i, j, k, l, m, n, o, p. , 27, 28, 29, 30), and the lenticular screen 21 enlarges a part of the multi-eye stripe element image in each of the plurality of cylindrical lens portions. In addition, the pixels 23, 24, 25, 26, 27, 28, 29, and 30 are pixels constituting a multi-view stripe-shaped element image displayed on the multi-view stripe-shaped element image display surface 1, and are respectively viewed from an observer's viewpoint. It corresponds to i, j, k, l, m, n, o, p. In FIG. 3, the viewing distance is slightly less than twice the thickness of the lenticular screen 21 due to space limitations, but the viewing distance is actually sufficiently larger than the thickness of the lenticular screen 21. Shall.

  As shown in FIG. 2B, images were taken from eight directions at equal intervals in the horizontal direction at the positions of the pixels 23, 24, 25, 26, 27, 28, 29, and 30 within one pitch of the lenticular screen 21. Images are displayed in a vertical stripe pattern. FIG. 4 shows a pixel arrangement for forming an image taken from each direction with an eight-lens multi-view stereoscopic camera into a striped element image. Images of each television camera are arranged within one pitch of the lenticular screen 21 in units of 8 pixels among horizontal pixels. In addition, as shown in FIG. 2B, the eight-eye stereoscopic television camera horizontally arranges eight cameras (i ′, j ′, k ′, l ′, m ′, n ′, o ′, p ′). Take photos at regular intervals.

  A stripe image taken from eight directions by the action of the lenticular screen 21 is separated and enters the left and right eyes, and a three-dimensional image can be observed without glasses by observing images of different viewpoints with the left and right eyes. it can. Furthermore, if the viewpoint is moved in the left-right direction, a three-dimensional image viewed from a different viewpoint can be viewed. Further, when the viewpoint interval is designed wider than the width of the left and right eyes, the same viewpoint video is observed with the left and right eyes. That is, the multi-viewpoint video display apparatus can observe a two-dimensional video viewed from different viewpoints by moving the viewpoint.

The light emitted from the lenticular screen 21 is composed of a main lobe and the surrounding side lobes. If the observer moves from side to side during observation in the main lobe, the viewpoint that can be observed changes. When the viewpoint of one eye deviates from the main lobe, the side lobe is observed with the other eye, and the stereoscopic view is reversed, which is uncomfortable. The wider the main lobe angle, the wider the motion parallax and the natural viewpoint movement becomes possible. In order to widen the angle of the main lobe, it is necessary to widen the viewpoint interval or increase the number of viewpoints within one pitch of the lenticular screen 21. However, when the viewpoint interval is widened as shown in FIG. 5 (Iij <Iqr, Ijk <Irs, Ikl <Ist, Ilm <Itu, Imn <Iuv, Ino <Ivw, Iop <Iwx), the screen between adjacent viewpoint images is displayed. The upper shift becomes large, flipping occurs, and smooth viewpoint movement is impaired. Here, in order to widen the viewpoint interval, the curvature of each lens of the lenticular screen 31 may be increased (the radius of curvature is decreased), or a material having a higher refractive index may be used for the lenticular screen 31. Further, when the number of viewpoints within one pitch of the lenticular screens 21 and 31 is increased, the horizontal resolution is lowered. As the lenticular method becomes more multi-view, the range of stereoscopic viewing becomes wider, and 3D images can be viewed simultaneously by many people.
Haruno Ogino et al. “8-eye glasses without 3D television” Journal of the Institute of Television Engineers of Japan, Vol. 48, No. 10, pp. 1267-1275 (1994)

  However, in the conventional multi-view video display device, in order to widen the viewing angle of the main lobe without changing the resolution and the number of viewpoints, it is necessary to widen the viewpoint interval. As a result, the problem is that smooth motion parallax is impaired. In order to obtain smooth motion parallax without changing the resolution and the number of viewpoints, it is necessary to narrow the viewpoint interval. As a result, it has been a problem that the viewing zone angle is narrowed when the viewpoint interval is narrowed.

In view of the above-mentioned problems of the prior art, the present invention can smooth the motion parallax of the central part, which is particularly important when observing, without increasing the resolution and the number of viewpoints. a multi-view image table 示装 location which can be widened), the multi-view image display device multiview video imaging device for imaging a multi-view image to be displayed on, and the multi-view image generation apparatus for generating a multi-view image The purpose is to provide.

In order to achieve the above object, the present invention provides a multi-view video display device that displays N (N is a natural number of 4 or more) viewpoint images.
The pixels constituting the N picked-up images picked up by the N image pickup means provided so that the peripheral portion is rougher than the central portion are arranged at equal intervals in the horizontal and vertical directions. Display means for simultaneously displaying on a display screen composed of a plurality of pixel display elements, and N pixels located at substantially the same position in the N picked-up images are arranged in the order of arrangement of the image pickup means. Display means for displaying the N captured images at the same time on the display screen by executing a process of displaying on the pixel display element in a corresponding arrangement for each pixel of the N captured images;
For each of the N pixels of the N captured images simultaneously displayed on the display screen by the display unit, the emitted light from the display of the N pixels corresponds to the arrangement of the N imaging units. An optical deflecting means for deflecting so as to be at a predetermined position interval,
It is characterized by having.
In a multi-view video display device that displays N (N is a natural number of 4 or more) viewpoint images,
Generation of N images generated so that the arrangement interval is equivalent to N captured images captured by N imaging means provided virtually so that the peripheral portion is coarser than the central portion. Display means for simultaneously displaying each pixel constituting an image on a display screen composed of a plurality of pixel display elements arranged at equal intervals in the horizontal and vertical directions, and at substantially the same position in the N generated images A process of displaying N pixels located in the pixel display element in an array corresponding to the order of arrangement of the N imaging means provided virtually is performed on each pixel of the N generated images. Display means for simultaneously displaying the N generated images on the display screen,
For each of the N pixels of the N generated images simultaneously displayed on the display screen by the display unit, the light emitted from the display of the N pixels is output to the N pieces of the virtually provided N pieces. A light deflecting means for deflecting the light so as to have a position interval corresponding to the arrangement of the imaging means;
It is characterized by having.

In order to achieve the above object, the present invention is characterized in that the light deflection means includes a lens array that deflects horizontally, vertically, and obliquely, or a lenticular sheet that deflects horizontally.
Further, N images equivalent to those obtained by capturing N (N is a natural number of 4 or more) viewpoint images provided so that the arrangement interval is coarser than the central portion are arranged. In the multi-view video generation device to generate,
It is characterized by having N image pickup means for picking up images of different viewpoints.
Furthermore, in a multi-view video generation device that generates N images equivalent to those obtained by capturing N (N is a natural number of 4 or more) viewpoint images,
Equivalent to an image picked up by N image pickup means for picking up images of N viewpoints (N is a natural number of 4 or more) provided such that the arrangement interval is coarser than the center portion. And generating means for generating N images.

  According to the present invention, it is possible to smooth the motion parallax in the central portion without changing the resolution and the number of viewpoints by making the visual point interval dense in the central portion and coarse in the peripheral portion, and the main lobe. The effect that the viewing zone angle can be expanded can be obtained.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an explanatory diagram showing an embodiment of a multi-view video display device according to the present invention, and FIG. 2 is an explanatory diagram showing an arrangement method of a camera realizing the multi-view video display device of FIG. 1 and a conventional camera. is there.

  As an example, FIG. 1 shows three-dimensional images (pixels 3, 4, 5, 6, 7, 8, 9, 10) from eight different viewpoints a, b, c, d, e, f, g, and h. It is a figure explaining the principle of the three-dimensional video display apparatus by an 8-eye lenticular system. In FIG. 1, a multi-view stripe-shaped element image display surface 2 displays multi-view stripe-shaped element images observed from a plurality of viewpoints a, b, c, d, e, f, g, and h, and a lenticular screen 1 has many. A part of the eye stripe element image is enlarged. Further, the pixels 3, 4, 5, 6, 7, 8, 9, and 10 are pixels constituting a multi-eye stripe element image displayed on the multi-ocular stripe element image display surface 2, and are respectively viewed from the observer's viewpoint. It corresponds to a, b, c, d, e, f, g, h.

  Further, Ra, Rb, Rc, Rd, Re, Rf, Rg, and Rh are ranges in which images of the viewpoints a, b, c, d, e, f, g, and h can be observed, respectively, and Ra> Rb> Rc> Rd = Re <Rf <Rg <Rh. Iab is the interval between the viewpoint a and the viewpoint b, and Ibc, Icd, Ide, Ief, Ifg, and Igh are the viewpoints b and c, c and d, d and e, e and f, f and g, and g and h, respectively. Is the interval. Therefore, Iab> Ibc> Icd> Ide <Ief <Ifg <Igh. In FIG. 1, the viewing distance (distance from the lenticular screen 1 to the observer) is slightly less than twice the thickness of the lens lenticular screen 1 due to space limitations. On the other hand, the viewing distance is sufficiently large. In FIG. 1, the light beam emitted from the lenticular screen is condensed on the eyeball of the observer, but the light beam emitted from the lenticular screen includes an afocal beam. 3 and 5 of the prior art, the viewpoint intervals of the prior art are equal, whereas the viewpoint intervals Iab, Ibc, Icd, Ide, Ief, Ifg, and Igh according to the embodiment of the present invention are the central part. Is different and the periphery is sparse.

  The multi-eye stripe element image display surface 2 is a flat screen like a screen for diffusing and displaying an image projected by, for example, a projection type projector, and the like. 4, 5, 6, 7, 8, 9, 10 are equal in the horizontal interval. A direct-view type panel such as an LCD, PDP, or organic EL may be used. As in the prior art, the multi-view stripe-shaped element image display surface 2 has eight directions (viewpoints a) at the positions of the pixels 3, 4, 5, 6, 7, 8, 9, 10 within one pitch of the lenticular screen 1. , B, c, d, e, f, g, h) are arranged and displayed in the form of stripes of vertical stripes. Conventionally, there has been a method of photographing an image displayed on the multi-view stripe-shaped element image display surface 2. Different from technology.

  With respect to FIG. 3 (and FIG. 5) of the prior art, the signal displayed on the multi-view stripe-shaped element image display surface 22 is an image taken from an equally spaced viewpoint as shown in FIG. Then, as shown in FIG. 2 (a), eight cameras (a ′, b ′) corresponding to the viewpoints a, b, c, d, e, f, g, h of the observer of the 3D image display device. , C ′, d ′, e ′, f ′, g ′, h ′) are photographed with a close viewpoint interval in the central portion and a sparse viewpoint interval in the peripheral portion (Sab> Sbc> Scd> Sde <Sef). <Sfg <Sgh), the image is used. At this time, the ratio of the observer viewpoint intervals Iab, Ibc, Icd, Ide, Ief, Ifg, Igh and the ratio of the camera intervals Sab, Sbc, Scd, Sde, Sef, Sfg, Sgh are made equal. That is, Iab: Ibc: Icd: Ide: Ief: Ifg: Igh = Sab: Sbc: Scd: Sde: Sef: Sfg: Sgh. In addition, instead of a live-action image, a simulation was performed to render an image taken from a camera with a central viewpoint interval and a peripheral viewpoint interval sparsely arranged in accordance with the 3D image display apparatus of FIG. Computer graphics may be used.

  In the 3D image display apparatus according to the present invention shown in FIG. 1, the viewpoint interval is dense at the center and sparse at the periphery. Further, in the 3D video device of FIG. 5, the viewpoint interval is larger and the viewing range of the main lobe is wider than the 3D video device of FIG. 3. By increasing the curvature of the lenticular screen 31 of FIG. 5 (decreasing the radius of curvature) with respect to the lenticular screen 21 of FIG. 3, the inclination of the normal line of the lenticular screen 31 is increased and the inclination of the emission angle is increased. . However, the screen deviation between adjacent viewpoint images increases, causing flipping, and smooth viewpoint movement is impaired.

  In FIG. 1, the viewpoint interval Ide of the central viewpoints d and e is equal to the viewpoint interval Ilm of the viewpoints l and m in FIG. Further, the viewpoint interval Iab between the viewpoints a and b in the most peripheral part in FIG. 1 is equal to the viewpoint interval Iqr between the viewpoints q and r in FIG. In order to do this, the central portion of the lenticular screen 1 in FIG. 1 has the same curvature as the central portion of the lenticular screen 21 in FIG. 3, and the peripheral portion of the lenticular screen 1 in FIG. 1 is the peripheral portion of the lenticular screen 31 in FIG. The curvature or refractive index of a plurality of lens portions arranged at equal intervals in the horizontal direction may be small in the central portion and large in the peripheral portion.

  As described in the prior art, the stripe images photographed from eight directions by the action of the lenticular screen 1 are separated and enter the left and right eyes, and the left and right eyes respectively observe the images of different viewpoints. A three-dimensional image can be observed without any. Furthermore, if the viewpoint is moved in the left-right direction, a three-dimensional image viewed from a different viewpoint can be viewed. Further, when the viewpoint interval is designed wider than the width of the left and right eyes, the same viewpoint video is observed with the left and right eyes. That is, the multi-viewpoint video display apparatus can observe a two-dimensional video viewed from different viewpoints by moving the viewpoint.

  In addition, by designing the viewpoint interval narrower than the width of the left and right eyes in the central part where the viewpoint interval is dense, and designing the viewpoint interval wider than the width of the left and right eyes in the peripheral part where the viewpoint interval is sparse, When viewed from the center, a 3D image can be formed, and when viewed from the periphery, a display device can be configured to observe a 2D image.

  The super multi-view display is a system that displays parallax images at a sampling distance that is finer than the pupil diameter at a certain viewing distance. If at least two parallax images are presented in the monocular pupil, continuous motion parallax can be reproduced. It is known that the congestion control contradiction is solved as well as it is possible (communication / broadcasting mechanism, advanced 3D image remote display project final report, September 2002). Even in this method, the central part, that is, the part where the viewpoint interval is dense, has a viewpoint distance finer than the pupil diameter, and the peripheral part, that is, the part where the viewpoint interval is sparse, the viewpoint distance is coarser than the pupil sensation. When observing, it can also be configured as a display that can eliminate congestion adjustment contradiction.

  In the present embodiment, the lenticular method having a parallax only in the horizontal direction has been described. However, the present invention is applied to a multi-view video display apparatus that has a parallax in all directions including not only the horizontal but also the vertical and oblique directions and can be freely viewed from any viewpoint. You can also For example, “three-dimensional image engineering” (Takayoshi Ohkoshi, Asakura Shoten, 1991) introduces a three-dimensional image technique that can be viewed freely from any viewpoint. The IP (Integral Photography) system, which is a typical system, was proposed by Lippmann in 1908. A lens array called a fly-eye lens as shown in FIG. 6 is placed in front of the image display surface. The lenticular method of this embodiment is an IP that is simplified by removing the upper and lower parallax information and limiting the parallax in the horizontal direction. When applying the present invention to the integral photography system, the lens array is designed by capturing images so that the central part of the viewpoint interval other than the horizontal direction is dense and the peripheral part is sparse, not only in the horizontal direction. This can be achieved.

It is explanatory drawing which shows one Embodiment of the multiview video display apparatus which concerns on this invention. It is explanatory drawing which shows each arrangement | positioning method of the camera which implement | achieves the multiview video display apparatus of FIG. 1, and the conventional camera. It is explanatory drawing which shows the conventional multiview video display apparatus. It is explanatory drawing which shows the formation method of a striped element image. It is explanatory drawing which shows the other conventional multiview video display apparatus. It is explanatory drawing which shows the lens array used for an integral photography system.

Explanation of symbols

1, 21, 31 Lenticular screen 2, 22 Multi-eye stripe element image display surface

Claims (6)

In a multi-view video display device that displays images of N viewpoints (N is a natural number of 4 or more),
The pixels constituting the N picked-up images picked up by the N image pickup means provided so that the peripheral portion is rougher than the central portion are arranged at equal intervals in the horizontal and vertical directions. Display means for simultaneously displaying on a display screen composed of a plurality of pixel display elements, and N pixels located at substantially the same position in the N picked-up images are arranged in the order of arrangement of the image pickup means. Display means for displaying the N captured images at the same time on the display screen by executing a process of displaying on the pixel display element in a corresponding arrangement for each pixel of the N captured images;
For each of the N pixels of the N captured images simultaneously displayed on the display screen by the display unit, the emitted light from the display of the N pixels corresponds to the arrangement of the N imaging units. An optical deflecting means for deflecting so as to be at a predetermined position interval,
A multi-view video display device characterized by comprising. In a multi-view video display device that displays images of N viewpoints (N is a natural number of 4 or more),
Generation of N images generated so that the arrangement interval is equivalent to N captured images captured by N imaging means provided virtually so that the peripheral portion is coarser than the central portion. Display means for simultaneously displaying each pixel constituting an image on a display screen composed of a plurality of pixel display elements arranged at equal intervals in the horizontal and vertical directions, and at substantially the same position in the N generated images A process of displaying N pixels located in the pixel display element in an array corresponding to the order of arrangement of the N imaging means provided virtually is performed on each pixel of the N generated images. Display means for simultaneously displaying the N generated images on the display screen,
For each of the N pixels of the N generated images simultaneously displayed on the display screen by the display unit, the light emitted from the display of the N pixels is output to the N pieces of the virtually provided N pieces. A light deflecting means for deflecting the light so as to have a position interval corresponding to the arrangement of the imaging means;
  A multi-view video display device characterized by comprising. 3. The multi-viewpoint video display apparatus according to claim 1, wherein the light deflecting unit is a lens array that deflects the emitted light from the display of the N pixels in a horizontal, vertical, and oblique direction. 3. The multi-viewpoint video display apparatus according to claim 1, wherein the light deflecting unit is a lenticular sheet that deflects outgoing light generated by displaying the N pixels in a horizontal direction. It is characterized by having N image pickup means for picking up images of N (N is a natural number of 4 or more) viewpoints provided so that the arrangement interval is coarser than the center portion. Multi-view video imaging device. Equivalent to an image picked up by N image pickup means for picking up images of N viewpoints (N is a natural number of 4 or more) provided such that the arrangement interval is coarser than the center portion. A multi-viewpoint video generation apparatus comprising generation means for generating N images.

Images