JP7424786B2 - 3D image display device, 3D image display method, and 3D image generation and display system - Google Patents

Description

The present invention relates to a stereoscopic image display device, a stereoscopic image display method, and a stereoscopic image generation and display system.

2. Description of the Related Art Three-dimensional images are displayed using an integral photography method. For example, in Patent Document 1, in response to the problem that the integral photography method has a narrow viewing range (viewing zone) in which a three-dimensional image can be observed, in a three-dimensional image display device, the display is arranged on a flat surface, and the viewing direction is By tilting the direction of the group of light beams directed toward the viewing direction with respect to the perpendicular direction of the plane, in other words, it is possible to observe a stereoscopic image from a viewing area called a side lobe rather than from a viewing area called a main lobe. It is stated that.

Japanese Patent Application Publication No. 2003-43413

Patent Document 1 further describes that a multi-view image displayed on a display is corrected by monotonically stretching it in the viewing direction as the distance from the viewer increases, and that a liquid crystal blind is used to prevent a deformed stereoscopic image from being formed. It is described that the corrected stereoscopic image can be displayed on a time-sharing basis to show it to multiple people while doing so.

However, even if the technique described in Patent Document 1 is used, the viewing range of each of the plurality of people is limited.
The present invention was made in order to solve such problems, and an object of the present invention is to provide a stereoscopic image display device, a stereoscopic image display method, and a stereoscopic image generation/display system that can expand the viewing area of a stereoscopic image. shall be.

A stereoscopic image display device according to the present invention has a display unit that displays a plurality of elemental images, and a plurality of exit pupils arranged two-dimensionally on the display unit, and has a direction in which light rays of the displayed elemental images exit. a light ray control unit that controls the rays; a ray blocking unit that is placed above or below the light ray control unit and transmits or blocks the rays for each exit pupil; The element image arrangement section is provided with an element image arrangement section that is arranged in a corresponding manner, and a synchronization control section that synchronizes the display of the element image on the display section and the transmission or blocking of the light beam on the light beam blocking section. The element images are arranged in an area that is the sum of an area corresponding to the exit pupil through which light rays are transmitted and an area corresponding to the exit pupil through which the light rays are blocked by the light ray blocking section.

Further, the stereoscopic image display method according to the present invention includes a display unit that displays a plurality of elemental images, and a plurality of exit pupils arranged two-dimensionally on the display unit, and includes a display unit that displays a plurality of elemental images, and a plurality of exit pupils that are arranged two-dimensionally on the display unit. A 3D image display method for a 3D image display device comprising: a light beam control section that controls an exit direction; and a light beam blocking section that is disposed above or below the light beam control section and transmits or blocks light rays for each exit pupil. , a plurality of elemental images are generated from a group of multi-view images, and a region corresponding to the exit pupil through which the light beam is transmitted by the light beam blocking section and a region corresponding to the exit pupil through which the light beam is blocked by the light beam blocking section are added. and a step of synchronizing the display of the element image on the display section and the blocking or transmission of the light beam in the light beam blocking section.

According to the present invention, it is possible to provide a stereoscopic image display device, a stereoscopic image display method, and a stereoscopic image generation/display system that expand the viewing area of a stereoscopic image.

1 is a diagram for explaining a method for observing a stereoscopic image according to Embodiment 1. FIG. FIG. 3 is a diagram for explaining a viewing zone according to the first embodiment. FIG. 3 is another diagram for explaining the viewing zone according to the first embodiment. 1 is a diagram for explaining a schematic configuration of a stereoscopic image display device 1 according to a first embodiment. FIG. FIG. 2 is a diagram for explaining a light beam control section 20 according to the first embodiment. FIG. 3 is a diagram for explaining a light beam blocking section 30 according to the first embodiment. 3 is a diagram for explaining an element image E according to Embodiment 1. FIG. FIG. 3 is a diagram for explaining the correspondence between partial elemental images PE and stereoscopic image observation areas OA according to the first embodiment. It is a figure for demonstrating the light beam blocking part 30 based on a 1st modification. FIG. 7 is a diagram for explaining an element image E according to a first modification. FIG. 7 is a diagram for explaining a viewing zone according to a first modification. FIG. 7 is a diagram for explaining an element image E according to a second modification. FIG. 7 is a diagram for explaining a viewing zone according to a second modification. 7 is a diagram for explaining the concept of Embodiment 2. FIG. FIG. 3 is a diagram for explaining the principle of displaying a stereoscopic image by the stereoscopic image display device according to the present embodiment. FIG. 3 is a diagram for explaining the principle of displaying a stereoscopic image by the stereoscopic image display device according to the present embodiment. FIG. 3 is a diagram for explaining the principle of displaying a stereoscopic image by the stereoscopic image display device according to the present embodiment. FIG. 3 is a diagram for explaining the principle of displaying a stereoscopic image by the stereoscopic image display device according to the present embodiment. FIG. 7 is a diagram illustrating an example in which the light beam control unit according to the second embodiment is implemented by a pinhole array. 20 is an enlarged view of the pinhole array shown in FIG. 19. FIG. FIG. 7 is a diagram illustrating an example in which the light beam control unit according to the second embodiment is implemented by a microlens array. FIG. 7 is a diagram for explaining design conditions of a stereoscopic image display device according to a second embodiment. 7 is a diagram showing the relationship between lens pitch and pixel enlarged image according to Embodiment 2. FIG. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 7 is a diagram for explaining the operation of the active shutter according to the second embodiment. FIG. 2 is a diagram for explaining the principle of moire generation in a conventional stereoscopic image display device. FIG. 3 is a diagram showing a stereoscopic image display device according to a second embodiment. FIG. 3 is a diagram showing a stereoscopic image display device according to a second embodiment. FIG. 7 is a diagram for explaining the principle of reducing the visibility of moiré in the stereoscopic image display device according to the second embodiment. FIG. 7 is a diagram for explaining the relationship between multi-view images and element images according to Embodiment 2. FIG. FIG. 7 is a diagram for explaining the relationship between multi-view images and element images according to Embodiment 2. FIG. 7 is a diagram for explaining the relationship between a camera and elemental images according to Embodiment 2. FIG. 7 is a diagram for explaining the relationship between a camera and elemental images according to Embodiment 2. FIG.

(Embodiment 1)
First, Embodiment 1 will be described. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. In addition, in each drawing, in principle, the same elements are denoted by the same reference numerals, and redundant explanations are omitted as necessary.
Before explaining the stereoscopic image display device and the stereoscopic image display method according to the first embodiment, the stereoscopic image observation method according to the device or the method will be briefly explained.
FIG. 1 is a diagram for explaining a stereoscopic image observation method according to the first embodiment.
Users 6 and 7 look down at the three-dimensional image 2 of an apple displayed on the three-dimensional image display device 1 from within a viewing area (not shown) in a comfortable posture.
In this specification, the side closer to the user viewing the stereoscopic image than the object in the direction perpendicular to the surface of the stereoscopic image display device 1 is referred to as "above the object", and also referred to as the side closer to the user viewing the stereoscopic image than the object. The following explanation will be made by defining a vertical relationship such that the far side is "below the object".

The stereoscopic image 2 is constructed as a collection of light rays 3 from the stereoscopic image display device 1. Humans perceive depth through binocular parallax, which occurs when viewing objects using both eyes, that is, the difference in retinal images between the left and right eyes. Depth is also perceived by motion parallax, that is, changes in the retinal image caused by relative movement between the observer and the object. Therefore, the users 6 and 7 can observe the stereoscopic image 2.

When the users 6 and 7 move their heads up, down, left and right within the viewing area, they can observe different stereoscopic images 2 at each position. Although users 6 and 7 are shown facing each other in FIG. 1, since the users 6 and 7 are observing the stereoscopic image from different directions, they can each observe different stereoscopic images. For example, when mahjong tiles are displayed as a three-dimensional image, users 6 and 7 can each observe the picture side of their own tiles, but cannot observe the picture side of their opponent's tiles. Since the stereoscopic image display device 1 has four viewing zones as described below, four users can surround the stereoscopic image display device 1 and simultaneously observe the stereoscopic image from four directions.

FIG. 2 is a diagram for explaining the viewing zone according to the first embodiment. FIG. 2 is a diagram of the stereoscopic image display device 1 and the viewing area as viewed diagonally from above.
Further, FIG. 3 is another diagram for explaining the viewing zone according to the first embodiment. FIG. 3 is a diagram of the viewing area viewed from above.

A viewing zone ML1 called a main lobe is formed in the vertical direction (normal direction) of the stereoscopic image display device 1, and a viewing zone SL1 called a side lobe is formed around the main lobe ML1. SL2, SL3, and SL4 are formed. Note that side lobes are also formed in diagonal directions of the main lobe ML1, but illustration and description thereof are omitted here.

In the observation method according to the first embodiment, the user can select different images from four areas OA1, OA2, OA3, and OA4 (hereinafter referred to as "stereoscopic image observation areas") indicated by triangles in side lobes SL1 to SL4. , it is possible to observe stereoscopic images without distortion. Note that if the user attempts to observe a stereoscopic image in a region other than the stereoscopic image observation area in the side lobes SL1 to SL4, the user will observe a different stereoscopic image from what was originally intended.

In the stereoscopic image display device 1 and the stereoscopic image display method according to the first embodiment, the main lobe ML1 and the side lobes SL1 to SL4 are wider than the viewing area according to the prior art, and as a result, the stereoscopic image observation area OA1 ~OA4 can also be made wider.

Next, a stereoscopic image display device 1 and a stereoscopic image display method according to the first embodiment will be described below with reference to the drawings.
FIG. 4 is a diagram for explaining the schematic configuration of the stereoscopic image display device 1 according to the first embodiment.
The stereoscopic image display device 1 includes a display section 10, a light beam control section 20, a light beam blocking section 30, a control section 40, and the like.

The display unit 10 is, for example, an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diode), etc., and its display area has a rectangular shape, and a plurality of pixels 11 (display pixels), and displays a plurality of elemental images, which will be described later. The image displayed by the display unit 10 may be either a moving image or a still image.

The light beam control section 20 is a microlens array having a plurality of microlenses 21 arranged two-dimensionally on the display section 10, and controls the emission direction of the light beam of the element image displayed on the display section 10.

FIG. 5 is a diagram for explaining the light beam control section 20 according to the first embodiment. FIG. 5 is a diagram of the light beam control section 20 without the light beam blocking section 30 viewed from above. The light beam control unit 20 has a honeycomb structure in which regular hexagonal microlenses 21 are arranged without gaps. With such a structure, it is possible to increase the density of the microlenses 21 and improve the resolution of the displayed stereoscopic image. Furthermore, the radius of curvature and the pitch of the microlenses 21 are made approximately equal to increase the power of the microlenses 21, thereby widening the viewing range.

Note that the light beam control unit 20 may be a pinhole array having a plurality of pinholes, for example, a honeycomb structure, instead of the microlens array. The microlens array or pinhole array itself is small and can provide horizontal and vertical parallax. Further, the shape of the microlens and pinhole may be other than a regular hexagon, for example, a square. In this specification, one microlens or one pinhole may be referred to as an "exit pupil."

The light beam blocking section 30 is an active shutter using, for example, a liquid crystal material, disposed on the light beam controlling section 20, and transmits or blocks (shields) the light beam from the exit pupil in a time-division manner for each exit pupil.
FIG. 6 is a diagram for explaining the light beam blocking section 30 according to the first embodiment. FIG. 6 is a diagram of the light beam blocking section 30 disposed on the light beam control section 20 viewed from above. A hatched region 31 indicates a region where the light ray blocking section 30 blocks light rays, and a non-hatched region 32 indicates a region through which the light ray blocking section 30 transmits light.

Furthermore, the grid-like thick line A indicates that each region surrounded by the thick line A corresponds to an area of an elemental image corresponding to one microlens 21 through which the light beam is transmitted by the light beam blocking section 30. Such a thick line does not actually exist in the light beam blocking section 30.
In the first embodiment, by providing the light beam blocking section 30, the area of one elemental image is changed from the conventional area corresponding to one microlens 21 to the area corresponding to the microlens 21 and the area where the light beam is blocked. 31 (regions on the left and right of the microlens 21 in FIG. 6) and a region corresponding to a part of the microlens 21 can be expanded. As a result, in the stereoscopic image display device 1, the main lobe ML1 and the side lobes SL1 to SL4 can be expanded.

Further, the light beam blocking section 30 blocks (or transmits) the light beam from the display section 10 every other row of the microlenses 21 . The light beam blocking section 30 alternately transmits and blocks light beams for one microlens 21 . For example, transmitting a light beam for 1/60 second and then blocking the light beam for 1/60 second is repeated 30 times per second. The number of repetitions per second may be increased by further shortening the time during which the light beam is transmitted or blocked.

By synchronizing the display by the display unit 10 and the transmission or blocking by the light blocking unit 30 and driving them at high speed, the user cannot perceive a change in the blocking area, and the number of pixels of the stereoscopic image is reduced to the number of exit pupils. , without degrading the user's sense of resolution.
In addition, here, the pitch of the area|region which blocks the light beam of the light beam blocking part 30 is set below the pitch of an exit pupil. As a result, the area that blocks the light rays at the center of the display area of the display unit 10 roughly matches the area of the exit pupil, but as it approaches the edges of the display area, the area that blocks the light rays is more central to the display area than the area of the exit pupil. It is off to the side. Such a configuration allows stereoscopic images to be displayed more efficiently.

Further, the light beam blocking section 30 may be provided below the light beam controlling section 20, and a mechanical active shutter or the like may be used for the light beam blocking section 30 instead of an active shutter using a liquid crystal.
The control unit 40 controls the operation of the stereoscopic image display device 1, and includes an element image arrangement unit 41, a synchronization control unit 42, etc. in order to display a stereoscopic image.

The elemental image arrangement unit 41 generates elemental images from the multi-view image group, and arranges the elemental images in correspondence with the microlenses 21. At this time, as described above, the elemental image placement unit 41 places the elemental image corresponding to the microlens 21 through which the light rays are transmitted in the region corresponding to the microlens 21 and the adjacent microlens 21 through which the light rays are blocked. and the corresponding area. A method for arranging element images will be explained next.

As described above, the synchronization control unit 42 synchronizes the display of the element images on the display unit 10 and the transmission (or blocking) of the light beam for each exit pupil in the light beam blocking unit 30.
With such a configuration, the stereoscopic image display device 1 according to Embodiment 1 can expand the viewing area while being thin.

Note that each component implemented by the control unit 40 can be implemented by executing a program under the control of an arithmetic unit (not shown) included in the control unit 40, which is a computer, for example.
More specifically, the control unit 40 loads a program stored in a storage unit (not shown) into a main storage device (not shown), and executes the program under control of the arithmetic unit. Further, each component is not limited to being realized by software based on a program, but may be realized by a combination of hardware, firmware, and software.

The programs described above can be stored and provided to the controller 40 using various types of non-transitory computer readable media. Non-transitory computer-readable media includes various types of tangible storage media.

Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tape, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), and CD-ROMs. R, CD-R/W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)).

The program may also be provided to the controller 40 by various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer readable medium can supply the program to the control unit 40 via a wired communication path such as an electric wire or an optical fiber, or a wireless communication path.

Next, the operation of the stereoscopic image display device 1 according to the first embodiment, that is, the stereoscopic image display method will be explained.
When the stereoscopic image display device 1 starts operating, the elemental image placement unit 41 inputs a multi-view image group from a multi-view imaging device group. The multi-view imaging device group is installed such that each imaging device corresponds to each viewpoint in the stereoscopic image observation area. Then, the elemental image arrangement unit 41 generates a plurality of elemental images from the multi-view image group, and arranges the elemental images so as to correspond to the individual microlenses 21.

FIG. 7 is a diagram for explaining the element image E according to the first embodiment. Each of the regions surrounded by thick grid lines A is an elemental image E corresponding to one microlens 21 through which the light beam is transmitted by the light blocking section 30. That is, the region E surrounded by the grid-like thick line A shown in FIG. 7 corresponds to the region E surrounded by the grid-like thick line A shown in FIG. FIG. 7 shows element images E1 to E9, which are 3 horizontal images×3 vertical images=9 images in total.

The elemental image E according to the first embodiment is composed of 96 viewpoint pixels (12 horizontal pixels×8 vertical pixels) selected one pixel from each image of a multi-view image group of 96 viewpoints.
Note that the display unit 10 displays a number of element images E corresponding to each image standard. For example, when a multi-view image group is captured with a VGA camera, 640 element images horizontally by 480 element images vertically are displayed. That is, one elemental image E and one microlens 21 correspond to one pixel of the camera.

Furthermore, each elemental image E has four partial elemental images PE1 to PE4. Each partial element image PE is an image corresponding to the light rays forming each stereoscopic image when the stereoscopic image is observed from the four stereoscopic image observation areas OA1 to OA4 shown in FIGS. 2 and 3. That is, the elemental image placement unit 41 divides the elemental image E into four parts so that different three-dimensional images can be observed simultaneously from four directions. In addition, in FIGS. 2, 3, and 7, the same hatching is attached to the corresponding stereoscopic image observation area OA and partial element image PE. For example, the stereoscopic image observation area OA1 and the partial element image PE1 for observing the stereoscopic image from the stereoscopic image observation area OA1 are each hatched in a diagonal line pattern from the upper right to the lower left. This also applies to the hatching in FIGS. 10 and 11 and the hatching in FIGS. 12 and 13.

With such a configuration, it is not possible to observe a distortion-free stereoscopic image from areas other than the stereoscopic image observation areas OA1 to OA4 in the main lobe and side lobes, but the four stereoscopic image observation areas OA1 to OA4 A stereoscopic image can be observed using binocular parallax or motion parallax.
FIG. 8 is a diagram for explaining the correspondence between partial elemental images PE and stereoscopic image observation areas OA according to the first embodiment. FIG. 8 is a sectional view taken along the line VV' in FIG. 7 when the element images E1 to E9 shown in FIG. 7 are displayed on the display unit 10.

The light beam from the partial elemental image PE2 of the elemental image E4 passes through the microlens 21 on the adjacent elemental image E5 and is observed as a stereoscopic image by the user in the stereoscopic image observation area OA2. Furthermore, the light beam from the partial elemental image PE4 of the elemental image E6 also passes through the microlens 21 on the adjacent elemental image E5 and is observed as a stereoscopic image by the user in the stereoscopic image observation area OA4. That is, the user can view the stereoscopic image from a position slightly higher than the 45-degree angle.

Note that when a honeycomb structure as shown in FIG. 5 is used, the effective elemental image width in the horizontal direction is shorter than the elemental image width in the vertical direction. However, as shown in Figure 6, if you place light ray blocking areas on the left and right, you can double the horizontal element image width while maintaining the vertical element image width, and It can be made longer than the width in the direction. For this reason, as for the discrete arrangement of elemental images, the positions that span both (the four corners of the elemental image) are preferentially allocated in the vertical direction, that is, to the partial elemental images PE2 and PE4, as shown in FIG. .

As described above, the synchronization control unit 42 allows the light beam blocking unit 30 to transmit or block the light beam in a time-division manner, and changes the display area and display content of the elemental image or partial elemental image in the display unit 10. synchronize.
In this way, the stereoscopic image display device and the stereoscopic image display method according to the first embodiment can expand the viewing range of the stereoscopic image so that the user can observe the stereoscopic image in a comfortable posture.

Note that various changes and modifications can be made to the stereoscopic image display device or the stereoscopic image display method according to the first embodiment.
For example, in the first embodiment described above, the light beam blocking section 30 transmits or blocks the light beams from the microlenses 21 in every other row, but it may transmit or block the light beams in every other row, or it may transmit or block the light beams in every other row and It is also possible to transmit or block every other row.

FIG. 9 is a diagram for explaining the light beam blocking section 30 according to the first modification. FIG. 9 is also a diagram of the light beam blocking section 30 disposed on the light beam control section 20 viewed from above. A hatched region 31 indicates a region where the light ray blocking section 30 blocks light rays, and a non-hatched region 32 indicates a region through which the light ray blocking section 30 transmits light.

In the first modification, the light ray blocking section 30 transmits or blocks the light rays from the microlenses 21 in every other column and every other row, that is, the light rays from one microlens 21 among the four microlenses 21. , and blocks the light rays from the remaining three microlenses 21. Furthermore, the light beam blocking section 30 sequentially changes the microlenses 21 that transmit the light beams, and sequentially transmits the light beams from the four microlenses 21.
At this time, the element image arrangement unit 41 places an element image corresponding to one microlens 21 through which the light beam is transmitted, in a region corresponding to the microlens 21 and in a region corresponding to the microlens 21 around which the light beam is blocked. Place it in the area plus the area.

FIG. 10 is a diagram for explaining the element image E according to the first modification. Again, each region surrounded by thick grid lines A is an element image E corresponding to one microlens 21 through which the light beam is transmitted by the light beam blocking section 30. Also shown in FIG. 10 are element images E11 to E19, which are 3 horizontal images×3 vertical images=9 images in total. Furthermore, each elemental image E11 to E19 is composed of partial elemental images PE11 to PE14.

FIG. 11 is a diagram for explaining the viewing zone according to the first modification. FIG. 11 is also a diagram when viewing the viewing area from above. Four side lobes SL11 to SL14 are formed around the main lobe ML11, and four stereoscopic image observation areas OA11 to OA14 are further formed within the side lobes SL11 to SL14.

In the first modification, the user observes the partial elemental image PE14 of the elemental image E14 displayed on the display unit 10 from the stereoscopic image observation area OA12. Further, the user observes the partial element image PE12 of the element image E16 displayed on the display unit 10 from the stereoscopic image observation area OA14.
In this way, the stereoscopic image display device and the stereoscopic image display method according to the first modification can also expand the viewing range of the stereoscopic image so that the user can observe the stereoscopic image in a comfortable posture.

Further, in the first embodiment or the modified example described above, the shape of the partial element image is triangular, but it may be quadrilateral. Since many displays are horizontally long, the elemental image arrangement unit 41 arranges partial elemental images in accordance with the aspect ratio of the display.

FIG. 12 is a diagram for explaining the element image E according to the second modification. Again, each region surrounded by thick grid lines A is an element image E corresponding to one microlens 21 through which the light beam is transmitted by the light beam blocking section 30. The light beam blocking section 30 transmits or blocks the light beams from the microlens 21 every other column and every other row, similarly to the one shown in FIG. 9 . FIG. 12 also shows element images E21 to E29, which are 3 horizontal images×3 vertical images=9 images in total. Further, each elemental image E21 to E29 is composed of partial elemental images PE21 to PE24.

FIG. 13 is a diagram for explaining the viewing zone according to the second modification. FIG. 13 is also a diagram when viewing the viewing area from above. Four side lobes SL21 to SL24 are formed around the main lobe ML21, and four stereoscopic image observation areas OA21 to OA24 are further formed within the side lobes SL21 to SL24.

In the second modification, the user observes the partial element image PE22 of the element image E24 displayed on the display unit 10 from the stereoscopic image observation area OA22. Further, the user observes the partial element image PE24 of the element image E26 displayed on the display unit 10 from the stereoscopic image observation area OA24.
In this way, the stereoscopic image display device and the stereoscopic image display method according to the second modification can also expand the viewing range of the stereoscopic image so that the user can observe the stereoscopic image in a comfortable posture.

Further, in the first embodiment described above, the display area of the display unit 10 has a rectangular shape, and the user can view the 3D image from the 3D image observation areas in four directions provided in directions substantially orthogonal to each side of the display area. Although the user observes the image, the user may also observe the stereoscopic image from one to three stereoscopic image observation areas provided in a direction substantially perpendicular to the sides of the display area. In this case, the elemental image placement unit 41 arranges the elemental images or the plurality of partial elemental images so that a stereoscopic image without distortion can be observed from each stereoscopic image observation area.

Further, in the first embodiment described above, the shape of the display area of the display unit 10 is a quadrilateral, but the shape of the display area may be a polygon, a rectangle, or a circle. The arrangement unit 41 arranges elemental images or a plurality of partial elemental images so that an undistorted stereoscopic image can be observed from each stereoscopic image observation area.

Furthermore, in the first embodiment described above, the elemental image placement unit 41 places the elemental images or partial elemental images of the bird's-eye-view stereoscopic image through the microlenses next to (or around) the microlenses 21 that transmit the light rays from them. 21, the elemental image arrangement unit 41 may further arrange elemental images or partial elemental images of the bird's-eye view stereoscopic image below the microlens 21 that transmits light rays from them. For example, in the first embodiment described above, in the stereoscopic image observation area OA2 shown in FIG. A stereoscopic image formed by light beams from PE4 and the partial elemental image PE2 of the elemental image E5 may be observed.

Further, in the first embodiment described above, the elemental image arrangement section 41 is one of the components of the stereoscopic image display device 1, but the elemental image arrangement section 41 may be configured separately from the stereoscopic image display device 1. That is, the stereoscopic image display device 1 may input the elemental images shown in FIGS. 7, 10, and 12 from an external elemental image arrangement device or the like.
Further, the stereoscopic image display device 1 according to the first embodiment described above may be used in a stereoscopic image generation and display system including the stereoscopic image display device 1 and a plurality of imaging devices that capture a group of multi-view images, such as a TV conference system. It may also be configured as
Furthermore, the stereoscopic image display device 1 or the stereoscopic image generation and display system according to the first embodiment described above may be configured as a game device, an interior, or the like.

As described above, the stereoscopic image display device 1 according to the first embodiment includes a display section 10 that displays a plurality of element images E, and a plurality of exit pupils arranged two-dimensionally on the display section 10. (microlenses 21) and controls the exit direction of the light rays of the displayed elemental image E; A light ray blocking unit 30 that transmits or blocks light rays; an elemental image arrangement unit 41 that generates a plurality of elemental images E from a multi-view image group and arranges them corresponding to the exit pupils (microlenses 21); The element image arrangement section 41 includes a synchronization control section 42 that synchronizes the display of the elemental image E and the transmission or blocking of the light beam in the light beam blocking section 30. The element image E is arranged in an area that is the sum of the area corresponding to the lens 21) and the area corresponding to the exit pupil (microlens 21) whose light rays are blocked by the light ray blocking section 30.

(Embodiment 2)
Next, a second embodiment will be described. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. Further, in each drawing, the same elements are denoted by the same reference numerals, and redundant explanation will be omitted as necessary. Embodiment 2 is configured to suppress moire from being visually recognized by a user when viewing a stereoscopic image.

FIG. 14 is a diagram for explaining the concept of the second embodiment. When observing a stereoscopic image through the microlens 21 in the stereoscopic image display device 1, depending on the observation position, spots called moiré that obstruct observation may occur, as shown in (A). Moiré (3D moiré) is periodic brightness unevenness. In FIG. 14A, the central region R1 of the stereoscopic image display device 1 is dark, and the regions R1 and R3 at both ends are bright. This moiré is a pattern with a very large period, so it is easy to visually recognize it.

On the other hand, as shown in (B), the stereoscopic image display device 1 according to the second embodiment is configured so that the large periodic spots are dispersed into fine periods and visually recognized. This suppresses moiré from being visually recognized by the user when viewing the stereoscopic image. The details will be described later.

15 to 18 are diagrams for explaining the principle of displaying a stereoscopic image by the stereoscopic image display device 1 according to the present embodiment. Note that the principle explained using FIGS. 15 to 18 can also be applied to the first embodiment described above. Note that FIGS. 15 to 18 are examples of general cases in which the viewing area is not expanded by the light beam blocking section 30 (shutter).

As shown in FIG. 15, the stereoscopic image display device 1 includes a light beam control section 20 (microlens array) having a plurality of microlenses 21 (exit pupils) and a plurality of pixels 11 (pixels 11A, 11B, 11C). It has a display section 10. As shown in FIG. 15, by arranging the microlens 21 above the pixel 11, the light from the pixel 11 becomes directional. For example, when the distance between the pixel 11 and the principal point of the microlens 21 is the focal length of the microlens 21, the light from the pixel 11 passes through the microlens 21 and is emitted as parallel light.

In the example of FIG. 15, light from the pixel 11A located at the center under the microlens 21-1 is emitted upward, as shown by the solid line. Furthermore, light from the pixel 11B on the right side is emitted toward the upper left, as shown by the broken line. Furthermore, the light from the pixel 11C on the left side is emitted toward the upper right, as shown by the dashed line. In this way, each light from the pixel 11 under the microlens 21 can be expressed as light in a certain direction.

As shown in FIG. 15, since the light from the pixel 11 has directionality, one point with depth can be expressed as a set of light rays for each pixel 11, as shown in FIG. 16. For example, point PA is a point where light from pixels 11A under each microlens 21 is gathered. Point PB is the point where light from the pixels 11B under each microlens 21 is gathered. Note that the pixels 11A under each microlens 21 correspond to the same position of the photographed object, and that position is reproduced at the point PA. Similarly, the pixels 11B under each microlens 21 correspond to the same position of the photographed object, and that position is reproduced at point PB. In this way, the stereoscopic image display device 1 according to the present embodiment expresses a point with three-dimensional depth by light condensed at one point in space.

Here, the number of pixels 11 under each microlens 21 is finite. In the example shown in FIG. 17, for the sake of explanation, it is assumed that there are three pixels 11, pixels 11A, 11B, and 11C, under each microlens 21. For example, there are three pixels 11 below the microlens 21-1: pixels 11A-1, 11B-1, and 11C-1. Then, in the adjacent microlens 21, the light ray is expressed by the pixel 11 below the adjacent microlens 21. Therefore, in one microlens 21-1, a light ray is expressed in the range of the elemental image directly below the microlens 21-1. In other words, the range of points that can be expressed by this microlens 21-1 is the range of angles (viewing angle θ) determined by the principal point of this microlens 21-1 and the width of the elemental image. Outside the viewing angle θ, light from the elemental image under the adjacent microlens 21 appears. Therefore, the intended three-dimensional image cannot be expressed.

Therefore, the range where the display ranges displayed through the microlens 21 overlap from the elemental images becomes the observable range (visual range). In the example of FIG. 18, the hatched area Vr represents the viewing zone where the display range from one end of the screen of the display unit 10 to the other overlaps.

Here, a case where the light beam control unit 20 (exit pupil) according to the second embodiment is realized by a microlens array and a pinhole array will be described using the drawings.
FIG. 19 is a diagram showing an example in which the light beam control unit 20 according to the second embodiment is implemented by a pinhole array. Further, FIG. 20 is an enlarged view of the pinhole array shown in FIG. 19. Since the light beam control section 20 formed of a pinhole array covers the display section 10, the size of the pinhole array is substantially the same as the size of the display section 10.

As shown in FIG. 19, the light beam control unit 20 configured with a pinhole array has a large number of pinholes 22. As shown in FIGS. 19 and 20, the pinhole 22 is formed, for example, in a regular hexagonal shape. Further, as shown in FIG. 20, the pinholes 22 and the adjacent pinholes 22 are arranged at an angle of 60 degrees to each other. Further, the length of the pinhole 22 in the horizontal direction (lateral direction) is defined as the horizontal opening width Hw. Further, the length of the pinhole 22 in the vertical direction (longitudinal direction) is defined as the vertical opening width Hh. Further, the distance (center-to-center distance) between adjacent pinholes 22 in the horizontal direction (lateral direction) is defined as a horizontal pitch Hph. Further, the distance (center-to-center distance) between adjacent pinholes 22 in the vertical direction (longitudinal direction) is defined as a vertical pitch Hpv.

Here, the number of pinholes 22 in the horizontal direction (horizontal transmission numerical aperture) corresponds to the number of horizontal pixels (number of 3D pixels in the horizontal direction) when displaying a stereoscopic image. Similarly, the number of pinholes 22 in the vertical direction (vertical transmission numerical aperture) corresponds to the number of vertical pixels (number of 3D pixels in the vertical direction) when displaying a stereoscopic image. Further, each pitch is determined by the width of the element image in each direction and the angle of the viewing zone (viewing zone angle). The aperture width is a size through which the light of one pixel to be displayed is transmitted. In other words, the aperture width is the size of a triplet (one set of RGB subpixels) that displays subpixels. By doing so, disruption of the color distribution is suppressed, so the user is less likely to perceive parallax moiré.

FIG. 21 is a diagram showing an example in which the light beam control unit 20 according to the second embodiment is implemented by a microlens array. Since the light beam control section 20 composed of a microlens array covers the display section 10, the size of the microlens array is substantially the same as the size of the display section 10.

Similar to the one shown in FIG. 5, the light beam control section 20 composed of a microlens array has a honeycomb structure in which regular hexagonal microlenses 21 are arranged without gaps. Therefore, FIG. 21 shows the aperture shape of the honeycomb lens. With this structure, the regular hexagonal microlenses 21 are arranged alternately, so that the microlenses 21 can be arranged efficiently. Thereby, the apparent resolution of the stereoscopic image can be improved. Furthermore, by arranging the microlenses 21 in this manner, the efficiency of the lenses can be increased.

Further, in FIG. 21, the distance from the upper end to the lower end of the microlens 21 is the 3D pixel pitch Lpv in the vertical direction (vertical direction). Further, the distance between the centers of adjacent microlenses 21 in the horizontal direction (lateral direction) is the 3D pixel pitch Lph in the horizontal direction. Here, in the microlens array, the number of peaks of the microlenses 21 corresponds to the number of 3D pixels when displaying a stereoscopic image. As in the case of a pinhole array, the number of ridges of the microlenses 21 in the vertical direction corresponds to the number of vertical pixels (the number of 3D pixels in the vertical direction) when displaying a stereoscopic image. Further, the number of ridges of the microlenses 21 in the horizontal direction corresponds to the number of horizontal pixels (the number of 3D pixels in the horizontal direction) when displaying a stereoscopic image.

FIG. 22 is a diagram for explaining the design conditions of the stereoscopic image display device 1 according to the second embodiment. FIG. 22 shows a case where the light beam control section 20 is realized by a microlens array. First, a method for calculating the distance d between lens pixels will be explained. The elemental image width E _p can be expressed by the following equation 1 using the pixel pitch P _p (pixel width). Here, _nv is the number of parallaxes.
E _p =n _v ×P _p ...(1)

From FIG. 17, the lens pixel distance d can be expressed by the following equation 2 using the viewing angle θ.
d=tan(θ/2)×E _p /2 (2)
Here, formula 2 can be expressed as the following formula 3 from formula 1.
d=tan(θ/2)× _nv × _Pp /2...(3)

Next, a method for calculating the lens pitch _Lp will be explained. In order to optimize the viewing zone at the position of the distance (visual distance L) from the lens (microlens 21) to the user viewpoint Pv, the lens is designed to be a circle where the light of the elemental image at the viewing distance L is located. By designing in this way, the viewing area of the folded back can also be optimized at the position of the viewing distance L.

The lens pitch L _p is determined by _the following equation 4 using a geometric method from FIG _. You can ask for it.
L _p =E _p ×L/(L+d)...(4)
In addition, in the case of an overhead view display, the user often observes with the viewing distance L being an arm's length distance. That is, the viewing distance L can be determined in advance depending on the length of the arm and the like. Since the length of an adult's arm is approximately 600 mm, it is preferable to design the lens pitch L _p by setting L = approximately 600 mm, for example.

Next, a method for calculating the radius of curvature of the microlens 21 will be explained. FIG. 23 is a diagram showing the relationship between the lens pitch and the pixel enlarged image according to the second embodiment. In FIG. 23, an image of the pixel 11 magnified by the microlens 21 is displayed. The pixel 11 is a triplet composed of a red sub-pixel 11r, a green sub-pixel 11g, and a blue sub-pixel 11b. Colors can be displayed by displaying all RGB triplets as pixels. Moreover, by doing this, even when observed from a position other than the front, adjacent parallax pixels are sequentially observed, making it difficult to observe moiré. In this way, as shown in FIG. 23, the magnification of the lens is determined from the conditions for all triplets of pixels to be observed within the lens.

The magnification m is obtained from the lens pitch L _p and the pixel pitch P _p using the following equation 5.
m= _Lp / _Pp ...(5)

The focal length f can be expressed by the following equation 6 using the lens pixel distance d and setting the position of the image plane where the pixels are enlarged as d'.
f=d×d'/(d'+d)
f=d/(1+d/d')...(6)

Here, the magnification m can also be expressed by the following equation 7.
m=d'/d...(7)
By substituting this equation into the imaging equation of equation 6, the following equation 8 is obtained.
f=d/(1+1/m)
f=d/(1+P _p /L _p )...(8)

Further, if the microlens 21 is a plano-convex lens, the radius of curvature r is determined by the following equation 9, where n is the refractive index.
r=f×(n-1)...(9)
Here, by substituting equation 8 into equation 9, the following equation 10 is obtained.
r=d/(1+P _p /L _p )×(n-1) (10)

A microlens array can be designed in the manner described above. Note that when the light beam control section 20 is implemented by a pinhole array, the pinhole array can be designed in the same manner. In this case, instead of the radius of curvature, the aperture width may be designed to be the size of the RGB triplet.

Next, the operation of the active shutter (light beam blocking section 30) according to the second embodiment will be explained using the drawings. The field sequential method is realized by the operation of the light beam blocking section 30 according to the second embodiment.

24 to 30 are diagrams for explaining the operation of the active shutter according to the second embodiment. 24 to 27 show a state in which the light beam blocking section 30 is blocking the microlens 21 at a certain moment. Here, it is assumed that the blocking of the light beam blocking section 30 is switched at high speed (for example, at 240 Hz) in the order from FIG. 24 to FIG. 27. Further, FIG. 28 is a cross-sectional view taken along the line A-A' in FIGS. 24 and 26. Further, FIG. 29 is a cross-sectional view taken along the line B-B' in FIGS. 25 and 27.

At a certain timing (instant) time t1, the light beam blocking section 30 blocks the microlens 21 as shown in FIG. 24. As a result, the light beam is blocked in the area 31, and the light beam is transmitted through the microlens 21-1 in the area 32 marked with "1". Here, the microlens 21 adjacent to the microlens 21-1 is shielded. Therefore, the area of one elemental image transmitted through the microlens 21-1 is changed from the conventional area corresponding to one microlens 21 (area E1' in FIG. 28) to the area corresponding to the microlens 21 and the area corresponding to the microlens 21. The first region (region E1 in FIGS. 24 and 28) can be expanded by adding the region and a region corresponding to a part of the region 31 that is adjacent to each other and blocks light rays. This can be achieved by designing the microlens 21 so that the elemental image pitch (elemental image width) _Ep described above corresponds to the expanded width of the elemental image E1. This also applies to FIGS. 25 to 27, which will be described later.

At time t2, which is the next timing (instant) after time t1, the light beam blocking section 30 blocks the microlens 21 as shown in FIG. As a result, the light beam is blocked in the area 31, and the light beam is transmitted through the microlens 21-2 in the area 32 marked with "2". Here, the microlens 21 adjacent to the microlens 21-2 is shielded. Therefore, the area of one elemental image transmitted through the microlens 21-2 is changed from the area corresponding to one conventional microlens 21 (area E2' in FIG. 29) to the area corresponding to the microlens 21 and the area corresponding to the microlens 21. It can be expanded to a first region (region E2 in FIGS. 25 and 29) including the region and a region corresponding to a part of the region 31 that is adjacent to each other and blocks light rays.

At time t3, which is the next timing (instant) after time t2, the light beam blocking section 30 blocks the microlens 21 as shown in FIG. As a result, the light beam is blocked in the area 31, and the light beam is transmitted through the microlens 21-3 in the area 32 marked with "3". Here, the microlens 21 adjacent to the microlens 21-3 is shielded. Therefore, the area of one elemental image transmitted through the microlens 21-3 is changed from the area corresponding to one conventional microlens 21 (area E3' in FIG. 28) to the area corresponding to the microlens 21 and the area corresponding to the microlens 21. It can be expanded to a first region (region E3 in FIGS. 26 and 28) including the region and a region corresponding to a part of the region 31 that is adjacent to each other and blocks light rays.

At time t4, which is the next timing (instant) after time t3, the light beam blocking section 30 blocks the microlens 21 as shown in FIG. As a result, the light beam is blocked in the area 31, and the light beam is transmitted through the microlens 21-4 in the area 32 marked with "4". Here, the microlens 21 adjacent to the microlens 21-4 is shielded. Therefore, the area of one elemental image transmitted through the microlens 21-4 is changed from the area corresponding to one conventional microlens 21 (area E4' in FIG. 29) to the area corresponding to the microlens 21 and the area corresponding to the microlens 21. It can be expanded to a first region (region E4 in FIGS. 27 and 29) including the region and a region corresponding to a part of the region 31 that is adjacent to each other and blocks light rays.

Moreover, at the next timing (instant) after time t4, the light beam blocking section 30 blocks the microlens 21 again as shown in FIG. Thereafter, the light beam blocking section 30 repeats blocking the microlens 21 in the order shown in FIGS. 24 to 27. Note that, as is clear from FIGS. 24 to 29, the pixels 11 constituting the element images E1 to E4 at times t1 to t4 are not independent of each other, and one pixel 11 constitutes the element images E1 to E4 at times t1 to t4. Each can be configured.

FIG. 30 is a diagram showing an image perceived by a user due to the operation of the active shutter according to the second embodiment. In FIG. 30, the microlens 21 (element image) marked with "1" corresponds to the microlens 21-1 (element image E1) shown in FIG. Further, the microlens 21 (element image) marked with "2" corresponds to the microlens 21-2 (element image E2) shown in FIG. Further, the microlens 21 (element image) marked with "3" corresponds to the microlens 21-3 (element image E3) shown in FIG. Further, the microlens 21 (element image) marked with "4" corresponds to the microlens 21-4 (element image E4) shown in FIG.

As described above, by switching the shutter (light beam blocking section 30) in time series, light beams are transmitted through all the microlenses 21, and corresponding images are displayed. Here, if the light beam blocking section 30 and the pixel 11 can be driven at high speed (for example, at several hundred Hz such as 240 Hz), the switching cannot be perceived by human ability. Therefore, as shown in FIG. 30, the user perceives the image as light coming from all the microlenses 21 (an illusion). In other words, the user can observe a stereoscopic image by using the number of 3D pixels as the number of microlenses 21. In other words, the user perceives an image in which the transparent images in FIGS. 24 to 27 are superimposed. Therefore, with the limited number of pixels 11, it is possible for the user to perceive an elemental image with an expanded viewing range without feeling any discomfort.

FIG. 31 is a diagram for explaining the principle of moire generation in the conventional stereoscopic image display device 1. As shown in FIG. The conventional stereoscopic image display device 1 shown in FIG. 31 does not have a light beam blocking section 30. Further, for explanation, the conventional display unit 10 shown in FIG. 31 has six pixels 11A, 11B, 11C, 11D, 11E, and 11F in the horizontal direction (horizontal direction) in each element image E. Pixels 11 with the same hatching correspond to pixels at the same position in a multi-view imaging device group.

In the stereoscopic image display device 1 in which a part of the display section 10 is enlarged and given directivity by the light beam control section 20, the period of the pixels and the periodic light beam control section 20 (microlens array or pinhole array) are have a finger in the pie. In other words, the pixel pitch and lens pitch interfere. As a result, large periodic spots (moiré) may be observed due to a black matrix (BM) that is a non-display area between pixels.

I will explain in detail. At the point where the light rays of the pixels 11 given directivity by the microlens 21 intersect, repeated pixels 11 (light rays of a plurality of corresponding pixels 11 arranged periodically) overlap, resulting in light intensity due to the luminance distribution of the pixels. Distribution 12 appears. In the example of FIG. 31, repeating pixels 11B overlap at a point Pfb where the light rays of a plurality of periodically arranged pixels 11B intersect, and a light intensity distribution 12B resulting from the luminance distribution of the pixels appears. Similarly, at a point Pfc where the light rays of a plurality of periodically arranged pixels 11C intersect, the repeated pixels 11C overlap, and a light intensity distribution 12C resulting from the luminance distribution of the pixels appears. Further, at a point Pfd where the light rays of a plurality of periodically arranged pixels 11D intersect, the repeated pixels 11D overlap, and a light intensity distribution 12D resulting from the luminance distribution of the pixels appears. Here, if the screen is observed at the designed viewing distance L (the point where the light rays intersect), the screen can be observed uniformly. However, when the viewing distance deviates from the designed viewing distance L, the black matrix 13 of the light intensity distribution 12 is recognized as a spot on the screen.

For example, if the point Pfd where the light rays intersect is taken as a viewpoint (observation position), the color corresponding to the light intensity distribution 12D can be seen on the entire screen. On the other hand, if the viewpoint (observation position) is set at a position shifted in the direction of viewing distance L from the point Pfd where these light rays intersect, the black matrix 13 between the light intensity distribution 12D and the light intensity distribution 12C will be visible. . Here, referring to FIG. 22, when the viewing distance becomes longer than the designed viewing distance L, the apparent elemental image width E _p ' becomes shorter than the designed elemental image width _Ep due to the geometric relationship. Become. Further, when the viewing distance becomes shorter than the designed viewing distance L, the apparent elemental image width E _p ′ becomes longer than the designed elemental image width E _p due to a geometrical relationship. Such a deviation between the apparent elemental image width E _p ′ and the designed elemental image width E _p causes interference between the periodic pixel pitch and the periodic lens pitch, and moire is visually recognized.

Note that in stereoscopic image display devices that provide only horizontal parallax, in order to disperse pixel spots in the vertical direction, the lens has been made oblique or the pixel side has been made oblique. The stereoscopic image display device 1 according to the second embodiment uses the light blocking section 30 to reduce the visibility of moire, as described below. Thereby, the stereoscopic image display device 1 according to the second embodiment can reduce the visibility of moire without requiring a special pixel shape or a special structure such as an optical diffusion structure.

32 and 33 are diagrams showing a stereoscopic image display device 1 according to the second embodiment. For clarity of explanation, in FIGS. 32 and 33, the horizontal direction (lateral direction) of the stereoscopic image display device 1 will be considered. Note that the structures in FIGS. 32 and 33 may correspond to the vertical direction (vertical direction) of the stereoscopic image display device 1. Furthermore, for clarity of explanation, in FIGS. 32 and 33, the light beam blocking section 30 is connected to the microlenses 21 in odd-numbered rows (odd-numbered rows from the left or right) and even-numbered rows (left or right) of the microlens array. It is assumed that the microlenses 21 in the even-numbered columns are alternately shielded.

Furthermore, for the sake of explanation, four microlenses 21-1A, 21-2A, 21-1B, and 21-2B are shown in FIGS. 32 and 33. It is assumed that microlenses 21-1A and 21-1B are microlenses 21 in odd-numbered rows, and microlenses 21-2A and 21-2B are microlenses 21 in even-numbered rows.

As shown in FIGS. 32 and 33, the microlens 21-1A and the microlens 21-2A are adjacent to each other. Further, the microlens 21-2A and the microlens 21-1B are adjacent to each other. Further, the microlens 21-1B and the microlens 21-2B are adjacent to each other. In FIG. 32, the microlens 21-2A and the microlens 21-2B are shielded by the light beam blocking section 30. On the other hand, in FIG. 33, the microlens 21-1A and the microlens 21-1B are shielded by the light beam blocking section 30.

The display unit 10 also includes pixels 11Aa, 11Ba, 11Ca, 11Da, 11Ea, 11Fa, 11Ga, 11Ha, 11Ia, 11Ja, and 11Ka that are adjacent to each other. Furthermore, the display section 10 has pixels 11Ab, 11Bb, 11Cb, 11Db, 11Eb, 11Fb, 11Gb, 11Hb, 11Ib, 11Jb, and 11Kb that are adjacent to each other. Furthermore, the display section 10 includes pixels 11Ac, 11Bc, 11Cc, 11Dc, 11Ec, and 11Fc that are adjacent to each other. Note that the pixel 11Ka and the pixel 11Ab are adjacent to each other, and the pixel 11Kb and the pixel 11Ac are adjacent to each other.

Here, in the stereoscopic image display device 1 (display unit 10) according to the second embodiment, the pixel width covered by one microlens 21 is N+1/2 of the pixels 11 (N is an integer of 1 or more). It is structured to have a length of 1 minute. In the examples of FIGS. 32 and 33, the pixel width covered by one microlens 21 corresponds to the length of 5.5 pixels 11.

Here, as described above, in this embodiment, the area of one elemental image E is expanded to a part of the area corresponding to the microlens 21 blocked by the light beam blocking section 30. Therefore, with the above-described configuration, the width of the elemental image E transmitted by one microlens 21 is 2×(N+1/2)=(2N+1) pixels 11. In other words, the width of the elemental image E is equal to the length of an odd number of pixels 11 forming the display section 10. In the examples of FIGS. 32 and 33, the width of one elemental image corresponds to the length of 11 pixels 11. By arranging the pixels 11 in this manner, the elemental images are configured to be shifted by half the width of the elemental image for each adjacent microlens 21.

In the state of FIG. 32, the element image E1A corresponding to the microlens 21-1A that is not blocked by the light beam blocking section 30 includes pixels 11Aa, 11Ba, 11Ca, 11Da, 11Ea, 11Fa, 11Ga, 11Ha, 11Ia, 11Ja. , 11Ka. Furthermore, the element image E1B corresponding to the microlens 21-1B that is not blocked by the light beam blocking section 30 corresponds to the pixels 11Ab, 11Bb, 11Cb, 11Db, 11Eb, 11Fb, 11Gb, 11Hb, 11Ib, 11Jb, and 11Kb. Note that pixels 11 with the same hatching correspond to pixels at the same position in a multiview imaging device group (the same applies to FIGS. 37 and 38 described later).

On the other hand, in the state of FIG. 33, the element image E2A corresponding to the microlens 21-2A that is not blocked by the light beam blocking section 30 has pixels 11Ga, 11Ha, 11Ia, 11Ja, 11Ka, 11Ab, 11Bb, 11Cb, 11Db, 11Eb. , 11Fb. Furthermore, the elemental image E2B corresponding to the microlens 21-2B that is not blocked by the light beam blocking section 30 corresponds to the pixels 11Gb, 11Hb, 11Ib, 11Jb, 11Kb, 11Ac, 11Bc, 11Cc, 11Dc, 11Ec, and 11Fc.

Furthermore, as described above, at the point where the light rays of the pixels 11 given directivity by the microlens 21 intersect, the repeating pixels 11 (the light rays of a plurality of corresponding pixels 11 arranged periodically) overlap each other. A light intensity distribution 12 resulting from the luminance distribution appears. In the example of FIG. 32, at the point Pfg where the light rays of the pixels 11G (11Ga, 11Gb) in the elemental image E1 (E1A, E1B) intersect, the repeated pixels 11G overlap, and a light intensity distribution 12G resulting from the luminance distribution of the pixels appears. Furthermore, at the point Pff where the light rays of the pixels 11F (11Fa, 11Fb) in the elemental image E1 (E1A, E1B) intersect, the repeated pixels 11F overlap and a light intensity distribution 12F resulting from the luminance distribution of the pixels appears. Further, at a point Pfe where the light rays of the pixels 11E (11Ea, 11Eb) in the elemental image E1 (E1A, E1B) intersect, the repeated pixels 11E overlap and a light intensity distribution 12E resulting from the luminance distribution of the pixels appears.

In addition, in the example of FIG. 33, at the point Pfb where the light rays of the pixels 11B (11Bb, 11Bc) in the elemental image E2 (E2A, E2B) intersect, the repeated pixels 11B overlap and the light intensity distribution 12B due to the luminance distribution of the pixels is appear. Further, at a point Pfa where the light rays of the pixels 11A (11Ab, 11Ac) in the elemental image E2 (E2A, E2B) intersect, the repeated pixels 11A overlap and a light intensity distribution 12A resulting from the luminance distribution of the pixels appears. Further, at a point Pfk where the light rays of the pixels 11K (11Ka, 11Kb) in the elemental image E2 (E2A, E2B) intersect, the repeated pixels 11K overlap, and a light intensity distribution 12K resulting from the luminance distribution of the pixels appears.

Here, as described above, by configuring the width of the elemental image E to be equal to the length of an odd number of pixels 11 constituting the display section 10, when the shutter (light blocking section 30) is switched, the pixel The light intensity distribution 12 resulting from the luminance distribution of the pixel 11 that appears at the point where the 11 light rays intersect appears at positions shifted by half a phase from each other. As shown in FIGS. 32 and 33, the positions where the light intensity distribution group 12-1 composed of the light intensity distributions 12G, 12F, and 12E appearing in FIG. 32 appear are the light intensity distributions 12B, 12A, and 12K appearing in FIG. The position where the light intensity distribution group 12-2 consisting of 12-2 appears is shifted by half a phase from each other.

In other words, the center positions of the light intensity distributions 12G, 12F, and 12E that make up the light intensity distribution group 12-1 are the same as those of the black matrix 13 of the light intensity distributions 12B, 12A, and 12K that make up the light intensity distribution group 12-2. This will correspond to the location. For example, the center position of the light intensity distribution 12G corresponds to the position of the black matrix 13 between the light intensity distribution 12B and the light intensity distribution 12A. Further, the center position of the light intensity distribution 12F corresponds to the position of the black matrix 13 between the light intensity distribution 12A and the light intensity distribution 12K. Further, the center position of the light intensity distribution 12A corresponds to the position of the black matrix 13 between the light intensity distribution 12G and the light intensity distribution 12F. Further, the center position of the light intensity distribution 12K corresponds to the position of the black matrix 13 between the light intensity distribution 12F and the light intensity distribution 12E.

With this configuration, the position where the black matrix 13 appears will be shifted by half a phase between when the microlenses 21 in the odd rows are shielded and when the microlenses 21 in the even rows are shielded. In other words, the period of the pixel 11 and the periodic light control unit 20 (microlens array or pinhole array) differ depending on whether the microlenses 21 in odd-numbered rows are shielded or the microlenses 21 in even-numbered rows are shielded. The phase of interference between the two will be shifted by half a phase. Therefore, the phases of the intensity of moiré observed by the black matrix 13 are also shifted by half a phase. Thereby, as described below, the stereoscopic image display device 1 according to the second embodiment can reduce the visibility of moiré.

FIG. 34 is a diagram for explaining the principle of reducing the visibility of moiré in the stereoscopic image display device 1 according to the second embodiment. FIG. 34 is an example where the observation position is a position shifted in the direction of viewing distance L from the point Pfg where the light rays in FIGS. 32 and 33 intersect. In this example, the observation position is the same for FIGS. 32 and 33.

In the example of FIG. 32, the direction directly below the observation position corresponds to the center of the light intensity distribution 12G, and therefore appears bright. On the other hand, as it moves away from the direction directly below in the horizontal direction, it approaches both ends of the light intensity distribution 12G, so the influence of the black matrix 13 increases and it appears darker. Graph Gr1 shows the intensity of light with respect to the screen position in the example of FIG. 32. As shown in graph Gr1, in the example of FIG. 32, the intensity of light is high near the center of the screen, and the intensity of light decreases toward both ends. Therefore, as shown by the moire distribution Mo1, moire is visible that is bright near the center of the screen and becomes darker toward both ends.

In the example of FIG. 33, the direction directly below the observation position corresponds to the black matrix 13 between the light intensity distribution 12B and the light intensity distribution 12A, and therefore appears dark. On the other hand, when moving away from the direction directly below in the horizontal direction, the influence of the black matrix 13 becomes lower, so the image appears brighter. Graph Gr2 shows the intensity of light with respect to the screen position in the example of FIG. 33. As shown in graph Gr2, the intensity of light near the center of the screen is low, and the intensity of light increases toward both ends. Therefore, as shown by the moire distribution Mo2, moire is visually recognized as being dark near the center of the screen and becoming brighter toward both ends.

Here, as described above, in the stereoscopic image display device 1 according to the second embodiment, the light beam blocking section 30 blocks or transmits the light beam from the display section 10 every other row of the microlenses 21. Then, the light beam blocking section 30 alternately transmits and blocks the light beam for one microlens 21 . As a result, the user visually perceives moire in which the moire distribution Mo1 and the moire distribution Mo2 are superimposed. Therefore, the user perceives the spots to be dispersed, as shown by the moiré distribution Mo3. Graph Gr3 shows the intensity of light corresponding to moire distribution Mo3. In this way, the intensity of light is dispersed on the screen that is visually recognized by the user.

As described above, each time the adjacent microlenses 21 are alternately shielded, the position where the black matrix 13 appears shifts by half a phase. Therefore, the phase of the intensity of moiré observed by the black matrix 13 is also shifted by half a phase. As a result, when the speckles are superimposed according to the user's perception, the phases of their intensity cycles are shifted by half a phase, so that the light intensities cancel each other out. Therefore, in the user's perception, light with an intensity that is approximately averaged with respect to the screen position is observed. Note that even if there is a spot, it will be a spot with the finest spatial frequency that can be displayed through the microlens 21, making it difficult for humans to perceive the spot. Therefore, the stereoscopic image display device 1 according to the second embodiment can reduce the perception of moire without requiring a special pixel shape or a special structure such as an optical diffusion structure.

Further, as described above, in the stereoscopic image display device 1 according to the second embodiment, elemental images are displayed at different timings by the microlenses 21 in odd-numbered columns and the microlenses 21 in even-numbered columns. The width of the elemental image, which is composed of pixels corresponding to the microlenses 21 not shielded by the shutter (light beam blocking section 30) and pixels corresponding to the microlenses 21 shielded by the shutter, is always an odd number of pixels. That is, the length is an integer number (2N+1; 11 in the above example). This has the effect that multi-view images can be efficiently displayed as element images (displayed perfectly).

Next, the relationship between multi-view images and element images will be explained. 35 and 36 are diagrams for explaining the relationship between multi-view images and element images according to the second embodiment. Note that in the following explanation, in order to clarify the explanation, the horizontal direction will be considered.

FIG. 35 is a diagram for explaining a method of photographing a multi-view image. A multi-view image of the object 90 is captured by the number of cameras 50 corresponding to the number of pixels corresponding to the element image width. In the above example, the element image width is the length of 11 pixels, so multi-view images in 11 directions are required. Therefore, multi-view images of the object 90 from 11 directions are captured by the cameras 50A to 50K that capture images from 11 directions.

FIG. 36 is a diagram showing a state in which a stereoscopic image is reproduced using multi-view images. As described above, the elemental image arrangement unit 41 (FIG. 4) receives a group of multi-view images from the multi-view cameras 50A to 50K. The elemental image arrangement unit 41 arranges images from the multi-view cameras 50A to 50K into a plurality of elemental images E. As a result, the three-dimensional image 2 is reproduced by the light rays 3A to 3K emitted from the elemental image E and corresponding to the cameras 50A to 50K. Note that, not only in live-action photography but also in computer graphics, the same number of cameras 50 as the number of pixels corresponding to the element image width is required.

37 and 38 are diagrams for explaining the relationship between the camera 50 and the element image E according to the second embodiment. In FIG. 37, the microlenses 21-1, 21-3, and 21-5 in the odd-numbered rows are not shielded (that is, the microlenses 21-1, 21-3, and 21-5 in the odd-numbered rows transmit the light beam). Indicates the condition. FIG. 38 shows a state in which the even-numbered microlenses 21-2, 21-4 are not shielded (that is, the even-numbered microlenses 21-2, 21-4 transmit light).

Further, as shown in FIG. 37, the elemental image E1A transmitted by the microlens 21-1 corresponds to the pixels 11Aa to 11Ka. Elemental image E1B transmitted by microlens 21-3 corresponds to pixels 11Ab to 11Kb. The elemental image E1C transmitted by the microlens 21-5 corresponds to the pixels 11Ac to 11Kc.

Further, as shown in FIG. 38, the elemental image E2A transmitted by the microlens 21-2 corresponds to the pixels 11Fa to 11Ka and 11Ab to 11Eb. The element image E2B transmitted by the microlens 21-4 corresponds to pixels 11Fb to 11Kb and 11Ac to 11Ec.

Further, in FIG. 37 (when the microlenses 21 in odd rows are transmitted), cameras 50A to 50K correspond to pixels 11A to 11K, respectively. Further, in FIG. 38 (when the microlenses 21 in even-numbered rows are transmitted), cameras 50A to 50F and 50G to 50K correspond to pixels 11F to 11K and 11A to 11E, respectively. In this way, in the stereoscopic image display device 1 according to the second embodiment, each camera 50 corresponds to the case where the microlenses 21 in the odd numbered rows are transmitted and the case where the microlenses 21 in the even numbered rows are transmitted. Pixel 11 is different.

In the integral photography method, by matching the photographed environment with the displayed environment, it is possible to display a stereoscopic image without distortion. Therefore, the camera 50 corresponding to each pixel 11 is placed at the point where the light rays emitted from each pixel 11 constituting each element image E via each microlens 21 intersect.

Here, in the second embodiment, as can be seen from FIGS. 32 and 33, the point at which the light rays intersect depends on whether the microlenses 21 in the odd number rows transmit the light rays or the microlenses 21 in the even number rows transmit the light rays. It's starting to shift. For example, the position of the point Pfg where the light rays intersect is different from the position Pfb where the light rays intersect. Therefore, the camera 50 is Make the positions different. This increases the variation in the direction of the light rays, making it possible to improve the quality of stereoscopic image display.

Furthermore, in the second embodiment, the microlenses 21 in odd-numbered rows and the microlenses 21 in even-numbered rows are alternately shielded by the shutter (light beam blocking section 30). In other words, the light beam blocking section 30 blocks the exit pupil (microlens 21) through which the light beam is transmitted and the exit pupil (microlens 21) adjacent to the exit pupil (microlens 21). Therefore, different cameras 50 are used when photographing multi-view images (elementary images) corresponding to the microlenses 21 in odd-numbered rows and when photographing multi-viewpoint images (elementary images) corresponding to the microlenses 21 in even-numbered rows. There is no need to take pictures.

That is, as shown in FIG. 37, when capturing multi-view images (element images) corresponding to the odd-numbered rows of microlenses 21, the cameras 50A to 50K capture the multi-view images at positions Pca to Pck, respectively. Then, as shown in FIG. 38, when capturing multi-view images (element images) corresponding to the even-numbered microlenses 21, the positions of the cameras 50A to 50K are moved to positions Pca' to Pck', respectively. . Then, the cameras 50A to 50K take multi-view images at positions Pca' to Pck', respectively.

In this way, by switching the microlens 21 to be shielded using the shutter, many multi-view images (element images) can be captured without increasing the number of cameras 50. In other words, the number of cameras 50 can be reduced to capture the same number of multi-view images (element images).

Furthermore, in computer graphics, when many cameras are set to take pictures at the same time, the load for generating multi-view images becomes heavy. On the other hand, by photographing multi-view images using the same camera in each field as in the second embodiment, the load for generating multi-view images can be reduced.

In addition, in the example mentioned above, the arrangement of the light rays in the horizontal direction was considered. On the other hand, the arrangement of the light beams in the vertical direction can be determined by the shape of the light beam control section 20 (exit pupil). In other words, since the exit pupil is a regular hexagon, the exit pupil (microlens 21) should be designed with an optimized lens pitch in the horizontal direction, and the vertical rays should be arranged to match the horizontal lens pitch. Bye.

1 Stereo image display device 2 Stereo image 3 Light rays 6, 7 User 10 Display section 11 Pixel 12 Light intensity distribution 13 Black matrix 20 Light beam control section 21 Microlens (exit pupil)
30 Light beam blocking section 31 Region 32 that blocks the light beam from the microlens 21 Region 40 that transmits the light beam from the microlens 21 Control section 41 Element image arrangement section 42 Synchronization control section 50 Camera ML Main lobe SL Side lobe OA Stereo image observation Area E Element image PE Partial element image

Claims (13)

a display section that displays a plurality of element images;
a light ray control unit having a plurality of exit pupils arranged two-dimensionally on the display unit and controlling the exit direction of the light rays of the displayed elemental images;
a light beam blocking section that is disposed above or below the light beam control section and transmits or blocks the light beam for each exit pupil;
an elemental image arrangement unit that generates the plurality of elemental images from a multi-view image group and arranges them in correspondence with the exit pupil;
comprising a synchronization control unit that synchronizes display of the element image on the display unit and transmission or blocking of the light beam in the light beam blocking unit,
The elemental image arrangement section includes a first region including a region corresponding to an exit pupil through which the light beam is transmitted by the light beam blocking section and a region corresponding to the exit pupil through which the light beam is blocked by the light beam blocking section. Place the element image ,
The width in the horizontal direction of the element image arranged in the first area is the length of an odd number of pixels in the horizontal direction that constitute the display section.
Stereoscopic image display device. The three-dimensional image display device according to claim 1, wherein the light beam blocking section blocks the light beam in at least one of every other column and every other row of the exit pupil. The display unit has a display area that displays the plurality of element images,
The elemental image includes a plurality of partial elemental images,
The elemental image arrangement unit arranges the plurality of partial elemental images in correspondence with a plurality of three-dimensional image observation areas in which a three-dimensional image is viewed from a direction substantially orthogonal to a side of the display area. Item 2. The stereoscopic image display device according to item 2. The element image arrangement section is
A first partial element image corresponding to the long side of the display area among the plurality of partial element images is set such that the length of the first partial element image in the long side direction in the display area is the element image. Arrange it so that it has the maximum possible length,
A second partial element image corresponding to the short side of the display area among the plurality of partial element images is set such that the length of the short side of the second partial element image in the display area is the display area. arranged so that it corresponds to the length of the short side of and is shorter than the maximum possible length of the element image;
The stereoscopic image display device according to claim 3. The stereoscopic image display device according to claim 3 or 4, wherein the shape of the partial element image is a triangle or a rectangle. The stereoscopic image display device according to any one of claims 1 to 5, wherein the exit pupil is a regular hexagon, and the light beam control section has a honeycomb structure formed by a plurality of the exit pupils. The stereoscopic image display device according to any one of claims 1 to 6, wherein the radius of curvature of the microlenses of the light beam control section and the pitch of the microlenses are approximately equal. The solid according to any one of claims 1 to 7, wherein a pitch of a region that blocks the light ray for each exit pupil of the light ray blocking section is equal to or less than a pitch of the exit pupil. Image display device. The stereoscopic image according to any one of claims 1 to 8, wherein the exit pupil is designed so that all of the pixels are displayed on the exit pupil when observed from a predetermined viewing distance. Display device. The stereoscopic image display device according to any one of claims 1 to 9 , wherein the light beam blocking section blocks the exit pupil adjacent to the exit pupil through which the light beam is transmitted. The stereoscopic image display device according to any one of claims 1 to 10 , wherein the exit pupil is a regular hexagon, and the light beam control unit has a honeycomb structure formed by a plurality of the exit pupils. A stereoscopic image display device according to any one of claims 1 to 11 ,
A stereoscopic image generation and display system comprising: a plurality of imaging devices that capture the multi-view image group. a display section that displays a plurality of element images;
a light ray control unit having a plurality of exit pupils arranged two-dimensionally on the display unit and controlling the exit direction of the light rays of the displayed elemental images;
A 3D image display method for a 3D image display device, comprising a light ray blocking section disposed above or below the light ray control section and transmitting or blocking the light ray for each exit pupil,
The plurality of elemental images are generated from a multi-view image group, and a region corresponding to an exit pupil through which the light beam is transmitted by the light beam blocking section, and a region corresponding to the exit pupil through which the light beam is blocked by the light beam blocking section. arranging the element image in a first area including a first area including a second area ;
a step of synchronizing the display of the elemental image on the display section and the blocking or transmission of the light beam in the light beam blocking section ;
The width in the horizontal direction of the element image arranged in the first area is the length of an odd number of pixels in the horizontal direction constituting the display section.
3D image display method.

Images