JP2012088549A - Stereoscopic image pickup device, stereoscopic image display device, and stereoscopic image pickup and display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reproduce a high-definition stereoscopic image in an integral photography method.SOLUTION: This stereoscopic image pickup and display device includes: a beam splitter 120 splitting a light flux arriving from a subject O into two systems; an image pickup system lens array 130 for converting one separated light flux into a light flux of parallax image group; a parallax image group image-pickup element 140 capturing the light flux of the parallax image group to acquire the parallax image group; a plane image-pickup element 150 for capturing the other separated light flux to acquire a plane image; a parallax image group display element 240 for displaying the parallax image group; a display system lens array 250 for converting the light flux of the parallax image group into the light flux of a stereoscopic image; a plane image display element 260 for displaying the plane image; and a beam mixer 270 for emitting the light flux of the stereoscopic image in conjunction with the light flux of the plane image.

Description

  The present invention relates to a stereoscopic image imaging device, a stereoscopic image display device, and a stereoscopic image imaging display device.

  There is known an integral photography (Integral Photography) system in which a three-dimensional image is photographed and reproduced using a lens array in which a large number of element lenses are arranged in both a photographing device and a display device (for example, Patent Documents). 1). In this integral photography system, at the time of photographing, a subject image captured by the photographing device via the photographing system lens array is picked up. The captured image obtained through the imaging system lens array includes a large number of element images having parallax with each other. An element image is an image obtained through an element lens alone. At the time of display, a display device in which the display system lens array is arranged on the display surface side displays the captured image. Thus, the observer can stereoscopically observe a subject image formed by a large number of element images formed through the display system lens array.

In the integral photography system, the resolution of an image of a subject at a short distance from the photographing system lens array changes in proportion to the number (density) of element lenses in the photographing system lens array. That is, in order to increase the resolution of an image of a subject at a short distance from the photographing system lens array, it is necessary to use a photographing system lens array having a high arrangement density of element lenses.
Further, the resolution of an image of a subject at a long distance from the photographing lens array depends on the number of pixels constituting each element image (pixel density in the element image). That is, in order to increase the resolution of an image of a subject at a long distance from the photographing system lens array, it is necessary to use a high-definition image sensor that can capture images so that the pixel density of each element image increases.

Further, the resolution of an image formed at a short distance from the display system lens array changes in proportion to the number of element lenses of the display system lens array. That is, in order to increase the resolution of an image formed at a short distance from the display system lens array, it is necessary to use a display system lens array having a high arrangement density of element lenses.
Further, the resolution of an image formed at a position far away from the display system lens array depends on the number of pixels constituting each element image. That is, in order to increase the resolution of an image formed at a position far from the display system lens array, it is necessary to use a high-definition display element that can display so that the pixel density of each element image increases.

JP 2002-228974 A

Therefore, in the integral photography system, when attempting to increase the imaging resolution and the display resolution, it is possible to use the imaging system lens array and the display system lens array having a high arrangement density of element lenses and to increase the resolution of the element image. Necessary. However, in that case, there are no current imaging elements and display elements that can obtain an element image having a pixel density corresponding to the required imaging resolution and display resolution. In the conventional integral photography system, a high-definition stereoscopic image is obtained. Can not be reproduced.
Accordingly, the present invention has been made to solve the above-described problem, and a stereoscopic image imaging apparatus and a stereoscopic image display apparatus capable of capturing and displaying a high-definition stereoscopic image in the integral photography system. And it aims at providing a three-dimensional image pick-up display device.

[1] In order to solve the above-described problem, a stereoscopic image capturing apparatus according to an aspect of the present invention includes a beam splitter that separates a light beam coming from a subject into two systems, and one light beam separated by the beam splitter, A lens array that converts a parallax image group having a parallax into a parallax image group, a parallax image group imaging device that obtains a parallax image group by imaging a light beam of the parallax image group obtained through the lens array, and the beam splitter separated And a planar image pickup element that obtains a planar image by imaging the other light beam.
[2] In order to solve the above-described problem, the stereoscopic image display device according to one embodiment of the present invention is image data of the same subject, each of which is a parallax image group data having a parallax and a plane that is a two-dimensional plane image. An acquisition unit that acquires image data, a parallax image group display element that displays the parallax image group data acquired by the acquisition unit, and a light beam of the parallax image group that is displayed by the parallax image group display element. A lens array to be converted into a light beam; a planar image display element that displays the planar image data acquired by the acquisition unit; a light beam of the stereoscopic image obtained through the lens array; and a planar image displayed by the planar image display element. And a beam mixer that emits the light beam together.
[3] The stereoscopic image display device according to [2], further including a signal level changing unit that changes a weight between the level value of the parallax image group data and the level value of the planar image data.
[4] In order to solve the above-described problem, a stereoscopic image pickup and display device according to an aspect of the present invention includes a beam splitter that separates a light beam coming from a subject into two systems, and one light beam separated by the beam splitter. An imaging system lens array that converts parallax image groups having parallax into parallax image groups, a parallax image group imaging device that obtains parallax image groups by imaging the parallax image group beams obtained through the imaging system lens array, and A flat image pickup device that picks up the other light beam separated by the beam splitter to obtain a flat image, a parallax image group display device that displays the parallax image group obtained by the parallax image group image pickup device, and the parallax image group display A display system lens array for converting a light beam of a parallax image group displayed by the element into a light beam of a three-dimensional image, and a planar image display element for displaying the planar image obtained by the planar image imaging device. When, characterized in that and a beam mixer for emitting together with light flux of the planar images the light beam planar image display device of the three-dimensional image obtained through the display system lens array is displayed.

  In the integral photography system, a high-definition stereoscopic image can be taken and displayed.

It is a block diagram showing the function structure of the stereoscopic image pick-up display apparatus which is 1st Embodiment of this invention. It is the figure which represented typically the arrangement | sequence of an element lens when an imaging system lens array and a display system lens array are seen from the incident side of a light beam. It is a side view of the element lens by a refractive index distribution lens, and is a figure showing typically the optical path in an element lens. It is a block diagram showing the functional structure of the three-dimensional image pick-up display apparatus which is 2nd Embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a functional configuration of a stereoscopic image capturing and displaying apparatus according to the first embodiment of the present invention. As shown in the figure, the stereoscopic image capturing and displaying apparatus 1 includes a stereoscopic image capturing apparatus 10 and a stereoscopic image displaying apparatus 20.

  The stereoscopic image capturing apparatus 10 separates the light beam coming from the subject O into two systems (a first system light beam and a second system light beam), and captures the first system light beam. Is a device that records parallax image group data according to the above and also records planar image data obtained by imaging a light beam of the second system. The parallax image group data is frame image data having parallax image groups with parallax. The planar image data is frame image data having a single two-dimensional image (planar image).

The stereoscopic image capturing apparatus 10 is, for example, a digital video camera that captures and records moving images. The stereoscopic image capturing apparatus 10 captures a parallax image group and a planar image, respectively, at a desired frame rate of 60 frames / second or 30 frames / second, for example, and for each series of moving images, a group of parallax image group data and a group of parallax images. Plane image data is recorded.
Here, the group of parallax image group data refers to a plurality of time-series parallax image group data, and the group of plane image data refers to a plurality of time-series plane image data.

Note that the stereoscopic image capturing apparatus 10 can also function as, for example, a digital still camera that captures and records still images. In this case, the stereoscopic image capturing apparatus 10 records a pair of parallax image group data and planar image data for each still image.
In the present embodiment, an example in which the stereoscopic image capturing apparatus 10 captures and records a moving image will be specifically described.

  The stereoscopic image display device 20 reads the parallax image group data and the planar image data recorded in the stereoscopic image imaging device 10 respectively, converts the parallax image group data into a luminous flux of the parallax image group, and converts the planar image data into the planar image. This is a device that converts the luminous flux of the parallax image group and the luminous flux of the planar image into a single luminous flux. The redevelopment I reproduced by the light beam output from the stereoscopic image display device 20 is a composite image of a stereoscopic image and a two-dimensional planar image by the integral photography method, and is perceived as a stereoscopic image by the observer. . Perception is that an image formed on the retina through the eyeball of the observer is converted into an electrical signal, the signal reaches the brain through the optic nerve, and a stereoscopic image is recognized by the action of the brain.

The stereoscopic image display device 20 is a display device that reproduces and displays a stereoscopic moving image, for example. The stereoscopic image display device 20 acquires a group of parallax image group data and a group of plane image data for each series of moving images from the stereoscopic image imaging device 10, and reproduces them at the same frame rate as that at the time of imaging. indicate.
Here, a group of parallax image group data is a plurality of time-series parallax image group data, and a group of plane image data is a plurality of time-series plane image data.

Note that the stereoscopic image display device 20 can also function as a display device that displays a stereoscopic still image, for example. In this case, the stereoscopic image display device 20 acquires and displays a pair of parallax image group data and planar image data for each still image from the stereoscopic image imaging device 10.
In the present embodiment, an example in which the stereoscopic image display device 20 reproduces and displays a stereoscopic moving image will be specifically described.

Next, a detailed functional configuration of the stereoscopic image capturing apparatus 10 will be described.
The stereoscopic image imaging device 10 includes an imaging lens 110, a beam splitter 120, an imaging system lens array 130, a parallax image group imaging element 140, a planar image imaging element 150, a parallax image group recording control unit 160, and a planar image. A recording control unit 170 and a recording unit 180 are provided.
However, in FIG. 1, the imaging lens 110, the beam splitter 120, the imaging system lens array 130, the parallax image group imaging element 140, and the planar image imaging element 150 are schematically arranged mainly for explaining the optical path. It is a thing.

  The imaging lens 110 is a lens optical system that collects a light beam coming from the subject O. The imaging lens 110 includes an objective lens and a focusing lens. The light beam collected by the imaging lens 110 enters the beam splitter 120.

  The beam splitter 120 is a light beam separator that splits an incident light beam from the imaging lens 110 into two systems (a first system light beam and a second system light beam) and emits them. The beam splitter 120 in the present embodiment is a cube beam splitter formed by bonding two right-angle prisms into a cube shape. For example, a dielectric multilayer film is deposited on the joint surface 121 of the two right-angle prisms constituting the cube beam splitter. The beam splitter 120 emits a part of the incident light beam as the first system light beam through the joint surface 121 and reflects the other part of the incident light beam at the joint surface 121 to be emitted as the second system light beam. .

  In FIG. 1, an alternate long and short dash arrow that is input to the beam splitter 120 through the imaging lens 110 represents an incident light beam on the optical axis of the imaging lens 110. A broken-line arrow passing straight through the joint surface 121 of the beam splitter 120 represents a first-system light beam (a first-system light beam) among the incident light beams on the optical axis. Further, the direction of travel changes at the joint surface 121 of the beam splitter 120, and the solid line arrow that travels at right angles to the incident light beam on the optical axis indicates the second line of the incident light beams on the optical axis. It represents a light beam (the light beam of the second system light beam).

  Note that a half mirror or a polarizing beam splitter can be applied to the beam splitter 120.

  The imaging lens array 130 is a lens plate formed by arranging a large number of element lenses in a two-dimensional manner so that the optical axes of the element lenses are parallel to each other. The imaging system lens array 130 condenses the first system of light beams emitted from the beam splitter 120 by each element lens, and outputs the condensed light beams to the parallax image group imaging device 140.

  The parallax image group imaging element 140 is a photoelectric conversion element that has an imaging surface, images a light beam that forms an image on the imaging surface, and converts it into image data. Specifically, the parallax image group imaging element 140 is a solid-state imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The parallax image group image sensor 140 is provided at a position where the light beam condensed by each element lens of the imaging system lens array 130 forms an image on the imaging surface. An image formed on the imaging surface of the parallax image group image sensor 140 is a parallax image group. The parallax image group imaging device 140 images a plurality of light beams obtained through the imaging system lens array 130 and outputs parallax image group data.

  The planar image pickup element 150 is a photoelectric conversion element that has an image pickup surface, picks up a light beam that forms an image on the image pickup surface, and converts it into image data. Specifically, the planar image pickup element 150 is a solid-state image pickup element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The planar image pickup device 150 is provided at a position where the second light flux emitted from the beam splitter 120 forms an image on the imaging surface. The image formed on the imaging surface of the planar image imaging device 150 is a planar image. The planar image capturing element 150 captures the second light flux and outputs planar image data.

The parallax image group recording control unit 160 takes in the parallax image group data output from the parallax image group imaging device 140 and causes the recording unit 180 to record it.
The planar image recording control unit 170 takes in the planar image data output from the planar image imaging device 150 and causes the recording unit 180 to record it.
The parallax image group recording control unit 160 and the plane image recording control unit 170 cause the recording unit 180 to record a group of parallax image group data and a group of plane image data for each series of moving images. At this time, the parallax image group recording control unit 160 records the parallax image group data included in the moving image in the recording unit 180 as a group of frame image data with a time stamp, and the planar image recording control unit 170 The planar image data included in the image is recorded in the recording unit 180 as a group of frame image data with a time stamp. Although the pair of frame image data is associated with each other by a time stamp, they may be associated with each other by means other than the time stamp.

  The recording unit 180 is a recording device such as a magnetic hard disk device or a memory card that records parallax image group data and planar image data as described above.

In the stereoscopic imaging apparatus 10, and the distance L _a to the center plane of the imaging system lens array 130 from one point on the bonding surface 121 for separating the light beam, and the distance L _b to the imaging plane of the planar image imaging element 150 from one point above If the ratio is A = L _a / L _b , the stereoscopic image capturing apparatus 10 is configured such that A is “1”.

Next, a detailed functional configuration of the stereoscopic image display device 20 will be described.
The stereoscopic image display apparatus 20 includes an acquisition unit 210, a parallax image group reproduction unit 220, a planar image reproduction unit 230, a parallax image group display element 240, a display system lens array 250, a planar image display element 260, and a beam mixer. 270.
However, in FIG. 1, the parallax image group display element 240, the display system lens array 250, the planar image display element 260, and the beam mixer 270 are schematically arranged and represented mainly for explaining the optical path.

  The acquisition unit 210 reads and acquires a group of parallax image group data and a group of plane image data in a series of moving images from the recording unit 180 of the stereoscopic image capturing apparatus 10, and among them, the group of parallax image group data is parallaxed. In addition to being supplied to the image group reproduction unit 220, a group of plane image data is supplied to the plane image reproduction unit 230.

  Note that the acquisition unit 210 sequentially captures a pair of parallax image group data and plane image data from the recording unit 180 and temporarily stores (buffers) the pair of parallax image group data and plane image data. It may be read out in a FIFO (First-In First-Out) format and supplied to each of the parallax image group reproduction unit 220 and the planar image reproduction unit 230. In other words, the difference between the speed at which the acquisition unit 210 reads from the recording unit 180 and the speed at which the acquisition unit 210 supplies the parallax image group reproduction unit 220 and the planar image reproduction unit 230 is absorbed by the buffering of the acquisition unit 210. It may be.

The parallax image group reproduction unit 220 supplies the group of parallax image group data supplied from the acquisition unit 210 to the parallax image group display element 240 at a predetermined periodic timing for each parallax image group data. This predetermined periodic timing is a timing based on the same frame rate as the frame rate when the stereoscopic image capturing apparatus 10 captured (for example, 1/60 when the frame rate at the time of capturing is 60 frames / second). Every second).
The planar image reproduction unit 230 supplies the group of planar image data supplied from the acquisition unit 210 to the planar image display element 260 at the same timing as the predetermined periodic timing for each planar image data.
That is, the parallax image group reproduction unit 220 and the planar image reproduction unit 230 supply the parallax image group data to the parallax image group display element 240 while synchronizing with the predetermined periodic timing. At the same time, the planar image reproduction unit 230 supplies the planar image data to the planar image display element 260.

  The parallax image group display element 240 has a display surface, and displays the parallax image group of the parallax image group data supplied from the parallax image group reproduction unit 220 on the display surface. Specifically, the parallax image group display element 240 is, for example, a liquid crystal display element. As the parallax image group display element 240, a reflective display element or a transmissive display element can be applied.

  The display system lens array 250 is a lens plate formed by arranging a large number of element lenses in a two-dimensional manner so that the optical axes of the element lenses are parallel to each other. The display system lens array 250 condenses the light beams of the parallax image group displayed by the parallax image group display element 240 with each element lens and outputs the condensed light beams to the beam mixer 270.

  The flat image display element 260 has a display surface, and displays a flat image of the flat image data supplied from the flat image reproducing unit 230 on the display surface. Specifically, the flat image display element 260 is, for example, a liquid crystal display element. As the flat image display element 260, a reflective display element or a transmissive display element can be applied.

  The beam mixer 270 is a light beam mixer that emits the incident light beam of the element image group from the display system lens array 250 and the incident light beam of the planar image displayed by the flat image display element 260 in accordance with one system of light beams. The beam mixer 270 in this embodiment is a cube beam mixer formed by bonding two right-angle prisms into a cube shape. For example, a dielectric multilayer film is deposited on the joint surface 271 of the two right-angle prisms constituting the cube beam mixer. The beam mixer 270 reflects the first luminous flux that passes through the junction surface 271 among the incident luminous fluxes of the element image group from the display system lens array 250 and the incident luminous flux of the planar image displayed by the planar image display element 260 at the junction surface 271. The second luminous flux to be synthesized is emitted.

  In FIG. 1, a wavy arrow input from the display system lens array 250 to the beam mixer 270 represents an incident light beam on the central axis of the display system lens array 250. A solid line arrow input from the planar image display element 260 to the beam mixer 270 represents an incident light beam on the central axis of the display surface of the planar image display element 260. The one-dot chain line arrow proceeding from the contact point between the broken-line arrow and the solid-line arrow represents a light beam obtained by combining the first light beam (the light beam of the first light beam) and the second light beam (the light beam of the second light beam). ing.

  The beam mixer 270 can be a half mirror or a polarizing beam splitter.

In the stereoscopic image display device 20, the distance L _c from the center plane of the display system lens array 250 to one point on the joint surface 271 that combines the two light beams, and from the one point to the one point from the display surface of the planar image display element 260. When the ratio of the distance _{L d} between _{B =} L c / _{L d,} a stereoscopic image display device 20 so B is "1" is configured.

In the above description, the case where the ratio A in the stereoscopic image capturing apparatus 10 and the ratio B in the stereoscopic image display apparatus 20 are both “1” has been described, but the ratio A and the ratio B are not “1”. It suffices that the ratio A and the ratio B coincide with each other. In this way, by setting the ratio A and the ratio B to the same value, the redevelopment I is correctly reproduced.
When the ratio A = the ratio B> 1, each element image captured by the element image group image sensor 140 has a relatively longer distance from the subject O than the planar image captured by the planar image sensor 150. That is, when the ratio A = the ratio B> 1, the imaging position of each element image is relatively far from the subject O compared to the imaging position of the planar image. That is, when the ratio A = the ratio B> 1, the image of the redevelopment I displayed by the stereoscopic image display device 20, particularly the image positioned on the display system lens array 250, is positioned farther than the planar image. Therefore, when ratio A = ratio B> 1, the resolution on the near side in the depth direction viewed from the observer is high.
When the ratio A = the ratio B <1, each element image captured by the element image group image sensor 140 has a relatively short distance from the subject O compared to the planar image captured by the planar image sensor 150. Become. That is, when the ratio A = the ratio B <1, the imaging position of each element image is relatively closer to the subject O than the imaging position of the planar image. That is, when the ratio A = the ratio B <1, the image located on the display system lens array 250 of the redevelopment I displayed by the stereoscopic image display device 20 is located closer to the planar image. Therefore, when the ratio A = the ratio B <1, the resolution on the far side in the depth direction viewed from the observer is high.

Next, the imaging system lens array 130 and the display system lens array 250 will be described. In the imaging system lens array 130 and the display system lens array 250, element lenses are mainly arranged in a delta arrangement or a square lattice arrangement.
FIG. 2 is a diagram schematically showing the arrangement of the element lenses when the imaging system lens array 130 and the display system lens array 250 are viewed from the light incident side. FIG. 4A is a diagram showing a delta arrangement, and FIG. 4B is a diagram showing a square lattice arrangement. In the case of the delta arrangement of FIG. 6A, if the pitch of the element lens in the X-axis direction is R _x , the pitch R _y in the Y-axis direction orthogonal to the X-axis is (√3) × R _x / 2. . Further, in the case of the square lattice arrangement in FIG. 5B, the pitch _{Rx in} the X-axis direction and the pitch _{Ry in the} Y-axis direction of the element lenses are equal.

The element lens of the imaging system lens array 130 may be configured using a gradient index lens (GRIN Lens). By using a refractive index distribution lens as the element lens of the imaging system lens array 130, it is possible to eliminate the reverse viewing phenomenon in which the depth of the stereoscopic image that is a reproduced image is reversed with respect to the subject O.
As shown in the following formula (1), the gradient index lens is a radial type whose refractive index changes in the radial direction. The refractive index n of this gradient index lens has a characteristic that it becomes maximum at the center position of the lens and decreases toward the outside. Here, n ₀ is the refractive index on the central axis, A is a refractive index distribution constant determined by the material of the element lens, and r is the radial distance from the center.

The length in the optical axis direction of the gradient index lens is set to a length that allows an erect real image of an object at infinity to be formed on the end face of the gradient index lens. This length will be specifically described.
FIG. 3 is a side view of an element lens using a gradient index lens, and is a diagram schematically showing an optical path in the element lens. As shown in the figure, by making the length L in the optical axis direction of the element lens 3 by the refractive index distribution lens to be three quarters (0.75 pitch) of the meandering period T, the element lens 3 is A small upright real image I can be obtained on the exit end face of the subject O located sufficiently far away.

Therefore, when the imaging lens array 130 is configured with a radial type refractive index distribution lens, the imaging surface of the elemental image group imaging element 140 is disposed on the exit end face of the element lens group of the imaging lens array 130.
In the configuration in which the parallax image group imaging element 140 is disposed immediately after the imaging system lens array 130 as described above, a light shielding member is preferably provided in the gap between adjacent element lenses of the imaging system lens array 130. For example, a black resin or the like is filled between the element lenses constituting the imaging system lens array 130. By providing the light shielding member in this way, it is possible to prevent or reduce the leakage of the light beam that has passed through one element lens into the light beam that has passed through another adjacent element lens. Thereby, it is possible to prevent or suppress image quality degradation due to black float (flare phenomenon) in the element image group captured by the element image group image sensor 140.

  The stereoscopic image capturing apparatus 10 separates the light beam coming from the subject O into two light fluxes, records the first light flux as the parallax image group data of the integral photography system, and the second light flux. Record as planar image data. That is, the stereoscopic image capturing apparatus 10 records parallax image group data including depth information representing the stereoscopic effect of the subject O, and also records planar image data including a high-definition resolution of the subject O.

As described above, the stereoscopic image display device 20 reads the parallax image group data and the planar image data recorded by the stereoscopic image capturing device 10, converts the parallax image group data into the luminous flux of the parallax image group, and also converts the planar image. The data is converted into a light beam of a plane image, and these two light beams are output together. That is, the stereoscopic image display device 20 displays the re-development I obtained by combining the integral photography stereoscopic image and the high-definition planar image.
The redevelopment I displayed by the stereoscopic image display device 20 is a high-definition stereoscopic image having a sense of depth. However, since the redevelopment I is an image obtained by combining the stereoscopic image and the planar image, the stereoscopic effect obtained when the observer observes the redevelopment I from the front is the stereoscopic effect in the stereoscopic image and the stereoscopic effect in the planar image. It becomes an intermediate thing. Further, the high definition feeling obtained when the observer observes the redevelopment I is intermediate between the high definition feeling in the stereoscopic image and the high definition feeling in the planar image. In other words, the re-development I perceived by the observer can be a high-definition image as compared with the conventional integral photography system while maintaining the stereoscopic effect.

[Second Embodiment]
The second embodiment of the present invention is a stereoscopic image capturing and displaying apparatus that can adjust the depth of redevelopment.
FIG. 4 is a block diagram illustrating a functional configuration of the stereoscopic image pickup display device according to the present embodiment.
In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.
As shown in the figure, the stereoscopic image capturing and displaying apparatus 1a includes a stereoscopic image capturing apparatus 10 and a stereoscopic image displaying apparatus 20a.

The stereoscopic image display device 20a reads the parallax image group data and the planar image data recorded in the stereoscopic image imaging device 10, respectively, performs first weighting on the parallax image group data, and converts the parallax image group data into a luminous flux of the parallax image group. This is a device for converting the plane image data into the light flux of the plane image by applying the second weighting, and outputting the light flux of the parallax image group and the light flux of the plane image in accordance with the light flux of one system. The re-development I reproduced by the light beam output from the stereoscopic image display device 20a is a composite image of a stereoscopic image and a two-dimensional planar image by the integral photography method, and is perceived as a stereoscopic image by an observer. .
The stereoscopic image display device 20a adjusts the trade-off between the stereoscopic effect of the redevelopment I and the resolution by the first and second weights.

A functional configuration of the stereoscopic image display device 20a will be described.
In the stereoscopic image display device 20a, a signal level change control unit 310 is newly added to the stereoscopic image display device 20 in the first embodiment, and the parallax image group reproduction unit 220 is changed to a parallax image group reproduction unit 220a. Thus, the planar image generating unit 230 is changed to the planar image reproducing unit 230a.

The signal level change control unit 310 calculates a change weight coefficient W ₁ for changing the level value of the parallax image group data and a change weight coefficient W ₂ for changing the level value of the planar image data, respectively. It supplies the weighting coefficients _{W 1} to the parallax image group reproduction unit 220a, and supplies the changed weighting coefficient _{W 2} on the planar image reproduction unit 230a.
Change weight coefficient W ₁ is larger the value is a coefficient for increasing the level value of the parallax image group data. Change weighting coefficient W ₂ is larger the value is a coefficient for increasing the level value of the planar image data. For example, the signal level change control unit 310 calculates W ₁ = S, W ₂ = 1−S, 0 ≦ S ≦ 1, so that the smaller the S in the range of 0 ≦ S ≦ 1, the smaller the change weight coefficient W ₂ is relatively large, as changes the weighting factor W ₁ S is large becomes relatively large.

Signal level change control unit 310 changes the changing weight factors W ₁ and changes the weighting factor W ₂ in accordance with the scene to be displayed. For example, the signal level change control unit 310 generates a new change weight coefficient W ₁ and a change weight coefficient W ₂ suitable for the switched scene according to a signal indicating the scene change timing supplied from the outside of the stereoscopic image display device 20a. Is received from the outside, and the current change weight coefficient W ₁ and change weight coefficient W ₂ are updated with these new change weight coefficient W ₁ and change weight coefficient W ₂ . Detection of a scene change is performed by an external device using conventional techniques such as detection of a luminance change of the entire frame image and object detection.

Parallax image group reproduction unit 220a, based on the change weighting coefficients W ₁ supplied from the signal level change control unit 310 changes the level value of the parallax image group data supplied from the acquisition unit 210. Specifically, for example, the parallax image group reproduction unit 220a multiplies the luminance values of the parallax image group data supplied from the changed weighting coefficient W ₁ and obtaining unit 210. Then, the parallax image group reproduction unit 220a supplies the changed parallax image group data to the parallax image group display element 240 at a predetermined periodic timing as in the first embodiment.

Planar image reproduction unit 230a, based on the signal level change control unit 310 changes the weighting factor W ₂ supplied to change the level value of the planar image data supplied from the acquisition unit 210. Specifically, for example, planar image reproduction unit 230a multiplies the luminance value of the planar image data supplied from the acquisition unit 210 and changes the weighting factor W _2. Then, the planar image reproduction unit 230a supplies the modified planar image data to the planar image display element 260 at the same timing as the predetermined periodic timing.

  That is, the function combining the signal level change control unit 310, the parallax image group reproduction unit 220a, and the planar image reproduction unit 230a is a signal level change unit that changes the level value of the parallax image group data and the level value of the plane image data. It includes the function of.

The re-development I displayed by the stereoscopic image display device 20a is a high-definition stereoscopic image having a sense of depth as in the first embodiment. Since the redevelopment I is an image in which a stereoscopic image and a planar image are combined, the stereoscopic effect and the sense of resolution obtained when the observer observes from the front are intermediate. However, by adjusting the level value of the change weighting coefficients W ₁ and based on the change weighting coefficient W ₂ element image group data and planar image data, it changes the three-dimensional effect and resolution feeling perceived.

Specifically, according to the stereoscopic image display device 20a is larger than the change weighting coefficient W ₂ changes the weighting coefficients W _1, is larger than the level value of the planar image data level values of the elemental image group data, observations The stereoscopic effect perceived by the person becomes stronger. For example, according to the stereoscopic image display device 20a is larger than the change weighting coefficient W ₂ changes the weighting coefficients W _1, is larger than the luminance value of the planar image data luminance values of the elemental image group data, the luminance of the stereoscopic image Becomes higher than the brightness of the planar image, and the stereoscopic effect is enhanced.
On the other hand, according to the stereoscopic image display device 20a is larger than the change weighting coefficients W ₁ changes the weighting factor W _2, larger than the level value of the elemental image group data level values of the plane image data, the viewer perceive The resolution to do becomes high. For example, according to the stereoscopic image display device 20a is larger than the change weighting coefficients W ₁ changes the weighting factor W _2, larger than the luminance value of the elemental image group data the luminance value of the planar image data, the luminance of the planar image Becomes higher than the luminance of the stereoscopic image, and the sense of resolution is increased.

  As described above, the stereoscopic image display device 20a reads the parallax image group data and the planar image data recorded by the stereoscopic image capturing device 10, performs the first weighting on the parallax image group data, and the luminous flux of the parallax image group. In addition to the above, the second weight is applied to the plane image data to convert it into the light flux of the plane image, and these two light fluxes are combined and output. The stereoscopic image display device 20a determines the first and second weights according to the scene to be displayed. Thereby, the stereoscopic image display device 20a can arbitrarily adjust the trade-off between the stereoscopic effect and the resolution by the first and second weights according to the scene to be displayed.

In the second embodiment described above, the signal level change control unit 310 may change the change weight coefficient W ₁ and the change weight coefficient W ₂ according to the complexity of the image of the planar image data. For example, when the signal level change control unit 310 analyzes the planar image data and determines that the planar image data is a fine image, in other words, an image having a large change in luminance and / or color in the planar image. It is larger than the change weighting coefficients W ₁ changes the weighting factor W _2. Accordingly, the stereoscopic image display device 20a can make the luminance of the planar image higher than the luminance of the stereoscopic image, and can display the re-development I with a high sense of resolution.
Further, the signal level change control unit 310 may change the change the weighting coefficients W ₁ and changes the weighting factor W ₂ in accordance with the degree of the depth of the element image group data. For example, when the signal level change control unit 310 extracts depth information from the element image group data and determines that the image has a large depth, in other words, an image that has a prominent stereoscopic effect, the change weight coefficient W ₁ is used as the change weight coefficient W _1. to be larger than W _2. Accordingly, the stereoscopic image display device 20a can make the luminance of the stereoscopic image higher than the luminance of the planar image, and can display the redevelopment I having a strong stereoscopic effect.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to that embodiment, The design of the range which does not deviate from the summary of this invention, etc. are included.

DESCRIPTION OF SYMBOLS 1,1a Stereo image pick-up display apparatus 10 Stereo image pick-up apparatus 20, 20a Stereo image display apparatus 110 Imaging lens 120 Beam splitter 121 Joint surface 130 Imaging system lens array 140 Parallax image group image sensor 150 Planar image image sensor 160 Parallax image group recording Control unit 170 Plane image recording control unit 180 Recording unit 210 Acquisition unit 220, 220a Parallax image group reproduction unit 230, 230a Plane image reproduction unit 240 Parallax image group display element 250 Display system lens array 260 Plane image display element 270 Beam mixer 271 Joint surface 310 Signal level change control unit

Claims (4)

A beam splitter that separates the light flux coming from the subject into two systems;
A lens array for converting one light beam separated by the beam splitter into a light beam of a parallax image group having a parallax;
A parallax image group imaging device that obtains a parallax image group by imaging a light beam of the parallax image group obtained through the lens array;
A planar image pickup device that captures the other light flux separated by the beam splitter and obtains a planar image;
A stereoscopic image pickup apparatus comprising: An acquisition unit that acquires parallax image group data having parallax and plane image data that is a two-dimensional plane image, each of which is image data of the same subject,
A parallax image group display element that displays the parallax image group data acquired by the acquisition unit;
A lens array that converts a light beam of a parallax image group displayed by the parallax image group display element into a light beam of a stereoscopic image;
A planar image display element for displaying the planar image data acquired by the acquisition unit;
A beam mixer that emits the light beam of the stereoscopic image obtained through the lens array and the light beam of the planar image displayed by the planar image display element;
A stereoscopic image display device comprising: The stereoscopic image display device according to claim 2, further comprising a signal level changing unit that changes a weight between a level value of the parallax image group data and a level value of the planar image data. A beam splitter that separates the light flux coming from the subject into two systems;
An imaging system lens array for converting one light beam separated by the beam splitter into a light beam of a parallax image group having a parallax;
A parallax image group imaging device that obtains a parallax image group by imaging a light beam of the parallax image group obtained through the imaging system lens array;
A planar image pickup device that captures the other light flux separated by the beam splitter and obtains a planar image;
A parallax image group display element that displays the parallax image group obtained by the parallax image group imaging element;
A display system lens array that converts a light beam of a parallax image group displayed by the parallax image group display element into a light beam of a stereoscopic image;
A planar image display element for displaying the planar image obtained by the planar image imaging element;
A beam mixer that emits the light beam of the stereoscopic image obtained through the display system lens array and the light beam of the planar image displayed by the planar image display element;
A stereoscopic image capturing and displaying apparatus comprising:

Images