JP5704702B2 - Stereo image acquisition device - Google Patents

Description

  The present invention relates to a stereoscopic image acquisition apparatus.

  Stereoscopic images are attracting attention as future video media, and the development and practical application of video equipment for stereoscopic images is advancing rapidly. In particular, autostereoscopic image systems that do not require special glasses for observing stereoscopic images are expected to be applied to home stereoscopic television broadcasting. In many autostereoscopic image systems, a subject is photographed to obtain a parallax image only in the horizontal direction, and the obtained parallax image is displayed on a binocular or multi-view stereoscopic display using only horizontal parallax. Conventionally, as a technique for acquiring a horizontal parallax image, there is a method in which a plurality of cameras are arranged in a line in a horizontal direction and a parallax image of a subject is acquired by these cameras.

  For example, in the stereoscopic image display device described in Patent Document 1, a subject is photographed from different directions by a plurality of cameras, and images obtained by the plurality of cameras are synchronized with blinking of corresponding light sources. The images are sequentially displayed on a spatial light modulation element (for example, a liquid crystal display). In the imaging system described in Patent Document 2, subjects are photographed from different directions by a plurality of cameras, and camera calibration, image correction, image matching processing, and the like are performed.

  Also, a method for acquiring a plurality of parallax images with one camera has been proposed. For example, in the stereoscopic image capturing system described in Patent Document 3, an adapter lens is attached to the front surface of a general camera lens, and a parallax image is obtained by dividing a light beam from a subject into a plurality of parts by the adapter lens. Is done. In the stereoscopic image capturing apparatus described in Patent Literature 4, a plurality of circular shutters are sequentially opened and closed, and light passing through the opened circular shutter is imaged on an imaging plate, whereby a parallax image is acquired.

JP 9-329762 A JP 2005-142957 A JP 2003-5314 A Japanese Patent Laid-Open No. 10-271534

  The methods using a plurality of cameras as disclosed in Patent Documents 1 and 2 require highly accurate camera calibration. Therefore, various adjustments of each camera are difficult. In addition, it is difficult to change the shooting angle of view or to perform camera work such as panning. Furthermore, since there are variations in luminance, color, and the like between images acquired by different cameras, complicated and time-consuming work such as correction processing for aligning the characteristics of these images is required.

  Further, in the method using a single camera as shown in Patent Documents 3 and 4, parallax images having a small number of parallaxes from two parallaxes corresponding to the number of shutters or apertures in the imaging optical system are time-divisionally divided. You can generally only get. Therefore, it is desired to obtain a large number of high-density parallax images in real time in a wide shooting angle of view.

  An object of the present invention is to provide a stereoscopic image acquisition apparatus that can efficiently acquire an element image necessary for displaying a stereoscopic image with a simple configuration.

  (1) A stereoscopic image acquisition apparatus according to the present invention is a stereoscopic image acquisition apparatus that acquires a plurality of parallax images, and includes a plurality of element lenses that respectively image rays from a subject as a plurality of element images, and a subject. A light guide optical system that guides the light beam to a plurality of element lenses, and a first imaging device that acquires a plurality of element images respectively formed by the plurality of element lenses. A light passage region having an optical axis along the direction and extending in a long shape in a second direction substantially orthogonal to the first direction, and guiding light rays passing through the light passage region to a plurality of element lenses, The light beam that does not pass through the light passage region is configured not to be guided to the plurality of element lenses.

  In the stereoscopic image acquisition apparatus, light rays from a subject are guided to a plurality of element lenses by a light guide optical system, and formed as a plurality of element images by the plurality of element lenses. A plurality of element images thus formed are acquired by the first imaging device.

  In this case, since a plurality of element images can be simultaneously acquired by the common first image capturing device, unlike the case of using a plurality of image capturing devices, there is no variation in characteristics among the plurality of element images. Therefore, a complicated operation for aligning the characteristics between the plurality of element images becomes unnecessary. Also, camera work such as changing the angle of view and panning is possible. Furthermore, it becomes possible to acquire a plurality of high-density element images over a wide angle of view.

  In the light guide optical system, a light passage region is formed so as to extend in a long shape in the second direction. Light rays that pass through the light passage region are guided to the plurality of element lenses, and light rays that do not pass through the light passage region are not guided to the plurality of element lenses. Thus, by limiting the light beam guided to the plurality of element lenses, the direction in which parallax occurs between the plurality of element images is limited to the second direction. Normally, if there is parallax in one direction between a plurality of element images, a stereoscopic image can be observed with the left and right eyes. Therefore, element images necessary for displaying a stereoscopic image can be efficiently acquired with a simple configuration, compared to a case where a plurality of element images having parallax in multiple directions are acquired.

  (2) The plurality of element lenses are two-dimensionally arranged on a plane parallel to the second direction and a third direction substantially orthogonal to the first and second directions, and the light guide optical system transmits light. A diverging optical element that diffuses or refracts the light beam so that the light beam passing through the region spreads in a plane including the first and third directions may be included.

  In this case, the light beam passing through the light passage region can be guided to more element lenses. As a result, more element images having parallax in the second direction can be acquired. Therefore, it is possible to increase the density of parallax between a plurality of element images and display a high-quality stereoscopic image.

  (3) The plurality of element lenses may be arranged so as to be aligned on a plurality of lines that are inclined with respect to the third direction and are parallel to each other.

  In this case, the plurality of element lenses are densely arranged in the second direction. Therefore, the density of parallax between a plurality of element images can be further increased.

  (4) The light guide optical system includes a first region in which the first polarized light can be transmitted and a second polarized light different from the first polarized light cannot be transmitted, and the first polarized light cannot be transmitted. A first polarizing element having a second region that is capable of transmitting the second polarized light, and a second polarizing element that is capable of transmitting the first polarized light and cannot transmit the second polarized light. The light from the subject may be guided to the plurality of element lenses through the first and second polarizing elements in order, and the light passing area may be formed by the first area of the first polarizing element.

  In this case, the light beam transmitted through the first region of the first polarizing element is transmitted through the second polarizing device, and the light beam transmitted through the second region of the first polarizing device is not transmitted through the second polarizing device. . Thereby, the light beam guided to the plurality of element lenses can be limited with a simple configuration.

  (5) The stereoscopic image acquisition device may further include a distance image acquisition unit that acquires a distance image including distance information of the subject.

  In this case, an interpolated image between a plurality of element images, a three-dimensional model of a subject, and the like can be easily generated using the distance image acquired by the distance image acquisition unit.

  (6) Each of the plurality of element lenses may be a variable focus lens capable of adjusting a focal length.

  In this case, the angle of view of a plurality of element images can be adjusted. In addition, the plurality of element lenses can be in a non-lens state in which light rays do not converge and diverge. Thereby, it is possible to selectively acquire a plurality of element images with parallax and a normal image without parallax.

  (7) The three-dimensional image acquisition device may further include a second imaging device and a split optical element that guides a part of the light beam guided to the plurality of element lenses by the light guide optical system to the second imaging device. .

  In this case, it is possible to acquire a plurality of element images by the first imaging device and to acquire a high-definition image of the subject by the second imaging device. For this reason, it is possible to correct a plurality of element images using an image acquired by the second imaging device, and to generate an interpolated image between the plurality of element images.

  (8) The light guide optical system may include a light shielding member in which an opening as a light passage region is formed. In this case, the light beam guided to the plurality of element lenses can be limited with a simple configuration.

  (9) The light guide optical system may be a telecentric optical system having an aperture as a stop. In this case, the light beam that has passed through the light passage region can be accurately guided to the plurality of element lenses, and a plurality of element images can be easily formed at desired positions by the plurality of element lenses.

  (10) The first polarizing element may be a spatial modulation element capable of adjusting polarization characteristics. In this case, the shape and arrangement of the light passage region can be easily adjusted. Thereby, it is possible to easily adjust the direction in which the parallax occurs between the plurality of element images.

  (11) The light guide optical system may include a long optical element extending in the second direction, and the light passing region may be formed by the long optical element. In this case, the light beam guided to the plurality of element lenses can be limited with a simple configuration.

  According to the present invention, it is possible to efficiently acquire an element image necessary for displaying a stereoscopic image with a simple configuration.

It is a figure which shows the structure of the stereo image acquisition apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the arrangement | sequence of the some element lens of a lens array. It is a figure which shows the other arrangement | sequence example of the some element lens of a lens array. It is a figure for demonstrating the basic principle of the stereo image acquisition apparatus which concerns on 1st Embodiment. It is a figure for demonstrating the basic principle of the stereo image acquisition apparatus which concerns on 1st Embodiment. It is a figure which shows the structure of the stereo image acquisition apparatus which concerns on the 2nd Embodiment of this invention. It is a front view of the 1st polarizer. It is a figure which shows the structure of the stereo image acquisition apparatus which concerns on the 3rd Embodiment of this invention. It is a figure which shows the structure of the stereo image acquisition apparatus which concerns on the 4th Embodiment of this invention. It is a perspective view of the lens used in 4th Embodiment. It is a figure which shows the structure of the stereo image acquisition apparatus which concerns on the 5th Embodiment of this invention. It is a block diagram which shows the structure of a stereo image display apparatus. It is a flowchart which shows operation | movement of a control part.

  Hereinafter, a stereoscopic image acquisition apparatus according to an embodiment of the present invention will be described with reference to the drawings. The stereoscopic image acquisition apparatus according to the present embodiment acquires images of subjects (parallax images) observed from a plurality of different positions. In the following description, three axes orthogonal to each other are defined as an x-axis, a y-axis, and a z-axis, respectively, and a direction parallel to the x-axis, a direction parallel to the y-axis, and a direction parallel to the z-axis are respectively x-direction, y Called the direction and z direction. The x direction and the z direction correspond to, for example, the horizontal direction, and the y direction corresponds to, for example, the vertical direction.

(1) First Embodiment (1-1) Basic Configuration FIG. 1 is a diagram illustrating a configuration of a stereoscopic image acquisition apparatus according to a first embodiment of the present invention. As shown in FIG. 1, the stereoscopic image acquisition apparatus 100 includes an imaging optical system 1, a diffusion plate 2, a lens 3, a lens array 4, and a camera 5 in order from the subject W.

  The imaging optical system 1 includes an imaging lens 11, an aperture plate 12 and an imaging lens 13. The optical axes of the imaging lenses 11 and 13 are along the z direction and coincide with each other. The aperture plate 12 is disposed in parallel to the xy plane. The aperture plate 12 has a long slit 12a extending in the x direction.

  The diffusing plate 2 is arranged in parallel to the xy plane and diffuses the light beam from the imaging optical system 1 in the y direction and does not diffuse or slightly diffuses in the x direction. As the diffusion plate 2, a holographic optical element capable of controlling the direction of light diffusion by light diffraction or refraction can be used. For example, an optical element in which irregularities are formed on a transparent plate made of acrylic or glass, an optical element having different refractive indexes in the x direction and the y direction, or the like can be used. Further, a lenticular screen having a minute structure may be used as the diffusion plate 2. Also in this case, light can be selectively diffused only in a specific direction.

  The optical axis of the lens 3 coincides with the optical axes of the imaging lenses 11 and 13 of the imaging optical system 1. The lens 3 is a field lens that controls the size of the image formed by the lens array 4, the angle of view, and the traveling direction of light rays. Instead of the lens 3, another field lens may be provided between the lens array 4 and the camera 5, or another field lens may be provided between the lens array 4 and the camera 5 in addition to the lens 3. Also good.

  The lens array 4 includes a plurality of element lenses 4a regularly and two-dimensionally arranged on the xy plane. The optical axis of each element lens 4a is along the z-axis. The camera 5 includes an imaging lens 5a and an imaging element 5b made of a CMOS (complementary metal oxide semiconductor) or a CCD (charge coupled device). In the present embodiment, a plurality of element images are formed by the plurality of element lenses 4 a of the lens array 4, and the plurality of element images are captured by the camera 5. Details will be described later.

(1-2) Lens Array FIG. 2 is a diagram showing an arrangement of a plurality of element lenses 4 a of the lens array 4. As shown in FIG. 2, the plurality of element lenses 4a are arranged so as to be aligned along the x direction and the y ′ direction. The y ′ direction is inclined at an angle θa with respect to the y direction. Hereinafter, the arrangement of the element lenses 4a along the x direction is referred to as a row of the element lenses 4a, and the arrangement of the element lenses 4a along the y ′ direction is referred to as a column of the element lenses 4a. In the example of FIG. 2, H element lenses 4a form each row, and V element lenses 4a form each column. The number of element lenses 4a in each row and each column is set according to the shape of the image sensor 5b of the camera 5 (FIG. 1) and the number of pixels. The distance between the center points of the adjacent element lenses 4a in each row is px, and the distance between the center points in the y direction of the adjacent element lenses 4a in each column is py.

  In each row, the distance between the center points in the x direction of the adjacent element lenses 4a is ph. In this example, ph = px / V. In this case, in two adjacent rows, the distance between the center points in the x direction between the element lens 4a located at the lower end of one row and the element lens 4a located at the upper end of the other row is ph. Thereby, all the element lenses 4a are evenly arranged in the x direction. In this way, the plurality of element lenses 4a are densely arranged in the x direction.

  FIG. 3 is a diagram showing another arrangement example of the plurality of element lenses 4 a of the lens array 4. In the example of FIG. 3, the plurality of element lenses 4 a are arranged so as to be aligned along the x ′ direction and the y ′ direction. The x ′ direction is inclined by an angle θa with respect to the x direction, and the y ′ direction is inclined by an angle θa with respect to the y direction. Also in this case, the plurality of element lenses 4a are densely arranged in the x direction.

  In the example of FIG. 3, a lens array 4 in which a plurality of element lenses 4a are arranged in a matrix can be used. Such a lens array 4 is easy to manufacture and inexpensive. Therefore, in the example of FIG. 3, the plurality of element lenses 4a can be arranged densely in the x direction easily and at a lower cost than the example of FIG. A lens array 4 in which a plurality of element lenses 4a are arranged in a delta arrangement may be used instead of the lens array 4 in which the plurality of element lenses 4a are arranged in a matrix.

  2 and 3, the shape of each element lens 4a is not limited to a circle, but may be other shapes such as a square or a hexagon. If the center positions of the plurality of element lenses 4 are different from each other in one direction (x direction in this example), the lens characteristics such as the size, shape, and focal length of the plurality of element lenses 4 are the same. Or may be different.

  As each element lens 4a of the lens array 4, a variable focus lens whose focal length is changed by external electrical or physical control may be used. For example, it is possible to use various focus variable lenses such as a liquid crystal focus variable lens using a liquid crystal material, a liquid lens based on an electrowetting principle in which a liquid contact angle changes depending on an applied voltage, or a liquid lens whose curvature changes depending on pressure. .

  In this case, the angle of view of the element image can be adjusted by changing the focal length of the element lens 4a. In addition, by using such a variable focus function, a light is converged (hereinafter referred to as a lens state) and a light is transmitted without being converged and diverged (hereinafter referred to as a non-lens state). Each element lens 4a can be switched. Accordingly, it is possible to selectively perform parallax image shooting and normal two-dimensional image shooting in the common stereoscopic image acquisition apparatus 100. In this case, by switching between the lens state and the non-lens state at high speed, the parallax image and the normal image can be alternately captured in units of frames, for example. Therefore, a parallax image and a high-definition image can be acquired in parallel.

(1-3) Basic Principle FIGS. 4 and 5 are diagrams for explaining the basic principle of the stereoscopic image acquisition apparatus 100 according to the present embodiment. 4 is a diagram illustrating the propagation of light rays from the subject W in the yz plane, and FIG. 5 is a diagram illustrating the propagation of light rays from the subject W in the xz plane. 4 and 5, the center points of the imaging lenses 11 and 13 and the lens 3 are arranged on the z-axis. In the following description, the direction toward the subject W is referred to as the front, and the opposite direction is referred to as the rear.

As shown in FIGS. 4 and 5, the focal length of the imaging lens 11 is f _1, a focal length of the imaging lens 13 is f _2, a focal length of the lens 3 is f _3, each element lens the focal length of 4a is _{f 4.} The distance between the imaging lens 11 and the aperture plate 12 is f _1, the distance between the center plane of the aperture plate 12 and the imaging lens 13 is f _2. The slit 12 a of the aperture plate 12 is disposed on the common optical axis of the imaging lenses 11 and 13. In the yz plane, the imaging optical system 1 is a double-sided telecentric optical system.

The distance between the center plane of the imaging lens 13 and the diffusion plate 2 is f _2. The distance between the center plane of the diffusion plate 2 and the lens 3 is f _3. In this example, the lens array 4 of FIG. 2 is used. The center point of the element lens 4a located substantially at the center of the lens array 4 is arranged on the z axis. Hereinafter, the element lens 4a having the center point on the z-axis is referred to as a reference element lens 4a. The distance between the center plane of the lens 3 and the center plane of the reference element lens 4a is (f ₃ + f ₄ ).

The propagation of light rays in the yz plane will be described with reference to FIG. Here, propagation of the light ray V ₁ from the object point P ₀ on the subject W (FIG. 1) will be described. 4, the principal ray from the object point P _o is shown in dashed lines.

The y coordinate of the object point _{P 0} and _{y 0,} the angle (hereinafter, referred to as light beam angle) of light _{V 1} is formed between the z axis in the object point _{P 0} to the theta _0, object point center _{P 0} and the imaging lens 11 Let z ₀ be the distance to the surface. In this case, the light beam V ₁ transmitted through the imaging optical system 1 is expressed by the following equation (1).

In equation (1), y ₁ is the y coordinate of the point P ₁ through which the light beam V ₁ transmitted through the imaging optical system 1 passes, θ ₁ is the ray angle at the point P ₁ , and z ₁ is This is the distance between the point P ₁ and the center plane of the imaging lens 13.

When the light ray V ₁ from the object point P ₀ is imaged at the point P ₁ , y ₁ is constant regardless of the value of the light ray angle θ ₀ , and therefore the following equation (2) is satisfied.

  From the equation (2), the following equation (3) is obtained.

On the other hand, in the imaging optical system 1, the light beam angle θ ₀ of the light beam V ₁ transmitted through the imaging optical system 1 is limited by the aperture plate 12. When the width in the y direction of the slit 12a of the aperture plate 12 is a, the range of light angles theta ₀ of rays _{V 1} that passes through the slit 12a is represented by the following formula (4).

  From the equations (1) and (4), the following equation (5) is obtained.

The light beam V _{1 that} has passed through the slit 12 a enters the point P ₂ on the diffusion plate 2 through the point P ₁ . The spread Δy of the light beam V ₁ at the point P ₂ is expressed by the following equation (6) from the equations (1) and (5).

  Since the imaging optical system 1 is a double-sided telecentric optical system, the size of the image formed on the diffusion plate 2 even if the position of the diffusion plate 2 is shifted in the z direction from the imaging position by the imaging optical system 1. Is substantially equal to the size of the image formed at the imaging position by the imaging optical system 1. However, the image on the diffusion plate 2 is blurred due to the spread of light rays Δy.

As can be seen from Equation (6), the smaller the width a of the slit 12a of the aperture plate 12, the smaller the light spread Δy. Therefore, blurring of the image on the diffusing plate 2 is suppressed. On the other hand, the smaller the width a is, the smaller the area of the slit 12a is, and the proportion of light rays that pass through the slit 12a among the light rays from the subject W decreases. Therefore, it is preferable to adjust the width a of the slit 12a so that an image of the subject W can be satisfactorily obtained according to the ambient brightness and the like. Here, it is assumed that the width a of the slit 12a is sufficiently small and the light spread Δy is zero. In this case, the y coordinate of the point P _{1 and} the y coordinate of the point P ₂ on the diffusion plate 2 are equal to each other and y ₁ .

Rays V ₁ incident on the point P ₂ on the diffusion plate 2 is diffused in the y direction by the diffusion plate 2. Thereby, the light ray V ₁ spreads in a fan shape in the yz plane from the point P ₂ to the rear. In this case, since the distance between the diffusing plate 2 and the lens 3 is equal to the focal length f ₃ of the lens 3, the diffused light from the point P ₂ becomes parallel light having a constant ray angle through the lens 3. The parallel light from the lens 3 enters each element lens 4 a of the lens array 4. The incident parallel light is imaged at an image point Pi on the focal plane of each element lens 4a.

  Light rays from each object point on the subject W are imaged on the focal plane of each element lens 4a, whereby an image EM of the subject W is formed on the focal plane of each element lens 4a. Hereinafter, the image EM formed on the focal plane of each element lens 4a is referred to as an element image EM.

  As shown in FIG. 2, the distance between the center points of the element lenses 4a in the y direction is py. The y coordinate of the center point of the reference element lens 4a is zero. Therefore, the y coordinate of the center point of the arbitrary element lens 4a is represented by npy (n is an integer).

Further, the ray angle θ ₄ of the light beam from the center point of each element lens 4 a toward each image point Pi is equal to the ray angle θ ₃ of the parallel light beam from the lens 3 toward the lens array 4, and the light beam angle θ ₃ is equal to the point P. _It is equal to the ray angle θ ₂ of the light ray from ₂ toward the center point of the lens 3. Thereby, the y coordinate y _Ln of the image point Pi by the arbitrary element lens 4a is expressed by the following equation (7).

As can be seen from Expression (7), the y coordinate y _Ln of the image point Pi does not depend on the distance z ₀ between the object point P ₀ and the imaging optical system 1. Therefore, no parallax occurs in the y direction between the plurality of element images EM.

Next, propagation of rays in the xz plane will be described with reference to FIG. Here, the propagation of the light beam V ₂ from the object point Q ₀ on the subject W will be described.

The light beam V ₂ from the object point Q ₀ on the subject W enters the lens array 4 through the imaging lenses 11 and 13 and the lens 3 of the imaging optical system 1. In the xz plane, the light beam V ₂ is not affected by the aperture plate 12 and the diffusion plate 2. The x coordinate of the object point _{Q 0} and _{x 0,} light _{V 2} is set to ₀ angle (ray angle) phi formed by the z-axis in the object point _{Q 0,} between the center plane of the object point _{Q 0} and the imaging lens 11 the distance between the _{z 4.} In this case, the light ray V ₂ incident on the lens array 4 is expressed by the following equation (8). In the formula (8), F = (f ₂ f ₄ ) / (f ₁ f ₃ ).

As shown in FIG. 2, the distance between the center points of the element lenses 4a in the x direction is Ph. The x coordinate of the center point of the reference element lens 4a is zero. Therefore, the x coordinate of the arbitrary element lens 4a is represented by mPh (m is an integer). Rays V ₂ passing through the arbitrary element lens 4a is expressed by the following equation (9).

In Expression (9), x _Lm is the x coordinate of the image point Qi where the light ray V ₂ transmitted through the arbitrary element lens 4a is imaged, φ _Lm is the ray angle at the image point Qi, and z _m is the distance between the image point Qi and the center plane of the element lens 4a.

At the image point Qi, x _Lm is constant regardless of the value of the ray angle φ ₀ , so the following equation (10) is satisfied.

  From the equation (10), the following equation (11) is obtained.

  From the equations (9) and (11), the following equation (12) is obtained.

As can be seen from equation (12), x-coordinate of the image point Qi depends on the z-coordinate of the object point _{Q 0.} Therefore, parallax in the x direction occurs between the plurality of element images EM. In this way, a plurality of element images EM having no parallax in the y direction and parallax in the x direction are formed.

  The imaging device 5b of the camera 5 (FIG. 1) has a plurality of imaging areas respectively corresponding to a plurality of element images EM, and a plurality of pixels are assigned to each imaging area. Light from the plurality of element images EM is received in the corresponding imaging region of the imaging element 5b through the imaging lens 5a. Thereby, a plurality of element images EM are simultaneously acquired by one camera 5.

  The number of pixels constituting one element image EM is equal to or less than the value obtained by dividing the total number of pixels of the image sensor 5b by the number of element images EM (number of element lenses 4a). Therefore, it is preferable to use the image sensor 5b having a larger number of pixels. For example, when the imaging device 5b having 8000 × 4000 pixels of the super high-definition class is used, 16 high-definition class element images EM or 100 VGA (video graphics array) class element images EM are simultaneously used. Can be acquired.

  Further, by using a plurality of element images EM acquired by the camera 5, an interpolation image for interpolating between the plurality of element images EM can be generated. In this case, a higher quality stereoscopic image can be displayed. Furthermore, it is possible to improve the resolution of each element image EM by using information (for example, difference) between a plurality of element images EM by a method called a super-resolution technique.

  In the present embodiment, the imaging optical system 1 is configured to be a bilateral telecentric optical system in the y direction. However, the present invention is not limited thereto, and the imaging optical system 1 is a one-side telecentric optical system in the y direction. It may be configured. The imaging optical system 1 may have a configuration other than the telecentric optical system as long as the amount of change in the image size on the diffusion plate 2 can be adjusted within an allowable range. Further, the imaging optical system 1 may be a zoom lens optical system with variable magnification. In this case, the imaging magnification of each element image EM can be changed.

  The distance between the imaging lens 13 and the diffusing plate 2, the distance between the diffusing plate 2 and the lens 3, and the distance between the lens 3 and the lens array 4 are not limited to the above example, and a plurality of element images EM can be obtained appropriately. It may be appropriately changed within the allowable range.

  In the above embodiment, a plurality of element images having parallax in the x direction and no parallax in the y direction are acquired. However, by rotating the aperture plate 12 and the lens array 4 by 90 degrees around the z axis, A plurality of element images with parallax and no parallax in the x direction can be acquired. Thereby, a plurality of element images having parallax in all directions can be acquired. Therefore, it is possible to display a stereoscopic image that can be observed from all directions.

(1-4) Effect In the present embodiment, since a plurality of element images EM can be simultaneously acquired by one camera 5, the brightness between the plurality of element images EM is different from the case of using a plurality of cameras. In addition, variations in characteristics such as color do not occur. Therefore, a complicated operation for aligning the characteristics between the plurality of element images EM becomes unnecessary. Also, camera work such as changing the angle of view and panning is possible. Further, it is possible to obtain a plurality of high-density element images EM over a wide angle of view.

  In the present embodiment, since the light beam guided to the plurality of element lenses 4a is limited by the aperture plate 12 of the imaging optical system 1, the direction in which parallax occurs between the plurality of element images EM is limited to the x direction. Is done. Normally, if there is parallax in one direction between the plurality of element images EM, a stereoscopic image can be observed with the left and right eyes. Therefore, it is possible to efficiently acquire the element image EM necessary for displaying the stereoscopic image with a simple configuration, compared to the case of acquiring a plurality of element images EM having parallax in multiple directions.

  In the present embodiment, the plurality of element lenses 4a are two-dimensionally arranged so as to be arranged along the x direction (x ′ direction in the example of FIG. 3) and the y ′ direction. In each case, a light beam from the subject W is guided through the diffusion plate 2. In this case, the plurality of element lenses 4a are densely arranged in the x direction, and the light rays from the subject W are diffused by the diffusion plate 2, so that each of the plurality of element lenses 4a is reliably guided. Thereby, the density of parallax in the x direction can be further increased between the plurality of element images EM. As a result, a higher quality stereoscopic image can be displayed.

(2) Second Embodiment FIG. 6 is a diagram showing a configuration of a stereoscopic image acquisition apparatus 100 according to a second embodiment of the present invention. Differences between the stereoscopic image acquisition apparatus 100 of FIG. 6 and the stereoscopic image acquisition apparatus 100 according to the first embodiment will be described.

  In the stereoscopic image acquisition apparatus 100 of FIG. 6, a first polarizer 21 is provided instead of the aperture plate 12 of the imaging optical system 1. Further, a half mirror 22 and a second polarizer 23 are provided between the imaging optical system 1 and the diffuser plate 2, and a camera 24 is provided in the direction in which light rays are reflected by the half mirror 22. The camera 24 includes an imaging lens 24a and an imaging element 24b.

  The first polarizer 21 is disposed in parallel to the xy plane. FIG. 7 is a front view of the first polarizer 21. As shown in FIG. 7, the first polarizer 21 includes a first polarization region 21a and a second polarization region 21b. The first polarizing region 21a has the same shape as the slit 12a of the aperture plate 12 in the first embodiment, and is provided so as to extend in a long shape in the x direction. The second polarizing region 21b is provided so as to extend on both sides of the first polarizing region 21a in the y direction. The first polarizing region 21a and the second polarizing region 21b have different polarization characteristics. In this example, the first polarizing region 21a transmits only the polarized light in the x direction, and the second polarizing region 21b transmits only the polarized light in the y direction.

  As shown in FIG. 6, the half mirror 22 is disposed behind the imaging optical system 1 so as to be inclined with respect to the z direction. In the example of FIG. 6, the half mirror 22 is arranged so as to be parallel to the x direction and at an angle of 45 degrees with respect to the y direction and the z direction.

  A second polarizer 23 is disposed behind the half mirror 22 in parallel with the xy plane. The light beam transmitted through the half mirror 22 enters the second polarizer 23. The second polarizer 23 has the same polarization characteristics as the first polarization region 21a of the first polarizer 21 and transmits only polarized light in the x direction.

  In the stereoscopic image acquisition apparatus 100 of FIG. 6, a part of the light beam that has passed through the imaging optical system 1 is transmitted through the half mirror 22, and the other part is reflected by the half mirror 22.

The light beam transmitted through the half mirror 22 enters the second polarizer 23. In this case, the light beam transmitted through the first polarizing region 21a of the first polarizer 21 is transmitted through the second polarizer 23, and the light beam transmitted through the second polarizing region 21b of the first polarizer 21 is the first light beam. The second polarizer 23 is not transmitted. Therefore, only the light beam that has passed through the first polarizing region 21 a of the first polarizer 21 out of the light beam from the subject W propagates behind the second polarizer 23 and enters the camera 5. As a result, as in the first embodiment, a plurality of element images having no parallax in the y direction and parallax in the x direction are acquired.

On the other hand, the light beam reflected by the half mirror 22 enters the camera 24. Light rays incident on the camera 24 include light rays that have passed through the first polarization region 21a of the first polarizer 21 and light rays that have passed through the second polarization region 21b. Therefore, an image of the subject W imaged by the imaging lenses 11 and 13 can be obtained with almost no influence of the first polarizer 21. Since the image acquired by the camera 24 is composed of all the pixels of the image sensor 24b , it has high definition. Further, the S / N ratio is high, and the angle of view in the y direction is large. Thereby, it is possible to correct a plurality of element images obtained by the camera 5 using an image obtained by the camera 24, and to generate an interpolated image between the plurality of element images.

Instead of the half mirror 22 and the second polarizer 23, a polarizing beam splitter may be used. In this case, the light beam transmitted through the first polarization region 21a of the first polarizer 21 is transmitted through the polarization beam splitter, and the light beam transmitted through the second polarization region 21b is reflected by the polarization beam splitter. A beam splitter is arranged. When the width in the y direction of the first polarization region 21a is sufficiently small, an image of the subject W is formed only by the light beam that has passed through the second polarization region 21b. Therefore, the image of the subject W can be acquired by the camera 24.

  As the first polarizer 21, a spatial light modulation element that can arbitrarily change the direction of the light beam may be used. As the spatial light modulator, for example, a spatial modulator having a polarizing plate and a liquid crystal can be used. By using a spatial light modulator as the first polarizer 21, the size and shape of the first and second polarizing regions 21a and 21b can be arbitrarily changed. Thereby, it is possible to control the parallax direction, the luminance, and the like in the plurality of element images without using a mechanical drive mechanism or the like. In addition, it is possible to transmit the light incident on the first polarizer 21 as it is without being polarized.

  Furthermore, a plurality of element images having parallax in the x direction and a plurality of element images having parallax in the y direction can be alternately obtained at high speed. Thereby, a plurality of element images having parallax in all directions can be acquired. In this case, three-dimensional modeling is facilitated by using a plurality of acquired element images, and generation of a three-dimensional image of an integral photography system or a hologram system is facilitated.

(3) Third Embodiment FIG. 8 is a diagram showing a configuration of a stereoscopic image acquisition apparatus 100 according to a third embodiment of the present invention. Differences between the stereoscopic image acquisition apparatus 100 in FIG. 8 and the stereoscopic image acquisition apparatus 100 according to the first embodiment will be described.

  In the example of FIG. 8, the imaging optical system 1 is provided with a single focal lens 31 having a short focal length instead of the imaging lens 13, and the diffusion plate 2 is not provided behind the imaging optical system 1. In this case, the light beam that has passed through the slit 12a of the aperture plate 12 is imaged by the single focus lens 31, and greatly spreads backward from the imaging position. Therefore, the light beam that has passed through the slit 12a can be guided to the plurality of element lenses 4a arranged in the y direction.

  Thus, instead of the diffusing plate 2, another optical element that spreads light in the y direction may be used. In place of the single focus lens 31 of FIG. 8, a silicon lens having a short focal length and a large curvature may be used, or a prism optical element that divides and refracts light rays in the x direction may be used.

(4) Fourth Embodiment FIG. 9 is a diagram showing a configuration of a stereoscopic image acquisition apparatus 100 according to a fourth embodiment of the present invention. Differences between the stereoscopic image acquisition apparatus 100 of FIG. 9 and the stereoscopic image acquisition apparatus 100 according to the first embodiment will be described.

  In the example of FIG. 9, in the imaging optical system 1, a lens 51 and a light shielding member 51 a are provided instead of the imaging lens 11 and the aperture plate 12. FIG. 10 is a perspective view of the lens 51 used in the present embodiment. As shown in FIG. 10, the lens 51 is provided in a long shape so as to extend in the x direction. As the lens 51, a Fresnel lens made of acrylic or plastic can be used. 9 is arranged so as to sandwich the lens 51 in the y direction.

  In the stereoscopic image acquisition device 100 of FIG. 9, the dimension of the lens 51 in the y direction is significantly smaller than the dimension in the x direction. Therefore, similarly to the case where the aperture plate 12 is provided, the light beam transmitted through the imaging optical system 1 is limited in the y direction. Thereby, as in the first embodiment, a plurality of element images having no parallax in the y direction and parallax in the x direction are acquired.

  In order to obtain a wider range of parallax in the x direction, it is necessary to increase the dimension in the x direction of the lens located at the forefront of the imaging optical system 1. When a circular lens such as the imaging lens 11 of FIG. 1 is used, it is necessary to increase the diameter of the lens, which requires advanced lens manufacturing technology and great cost. On the other hand, when the lens 51 of FIG. 10 is used, it is relatively easy and inexpensive, and a wider range of parallax can be obtained in the x direction.

(5) Fifth Embodiment FIG. 11 is a diagram illustrating a configuration of a stereoscopic image acquisition apparatus 100 according to a fifth embodiment of the present invention. A difference between the stereoscopic image acquisition apparatus 100 of FIG. 11 and the stereoscopic image acquisition apparatus 100 according to the first embodiment will be described.

  In the example of FIG. 11, a light irradiation unit 61 that irradiates the subject W with near-infrared light whose intensity is periodically modulated is further provided. The camera 5 further includes a dichroic mirror 62 and a distance detection imaging unit 63.

In the three-dimensional image acquisition device 100 of FIG. 11, the light beam incident on the camera 5 from the subject W is separated into visible light and near infrared light by the dichroic mirror 62. Visible light passes through the dichroic mirror 62 and enters the image sensor 5b. Thereby, a plurality of element images are acquired as in the first embodiment.

On the other hand, the near infrared ray is reflected by the dichroic mirror 62 and enters the distance detection imaging unit 63. The distance detection imaging unit 63 has a shutter mechanism that can be opened and closed at high speed, and performs high-speed gate imaging according to near-infrared intensity modulation. Based on a general optical time-of-flight measurement method, distance information indicating the distance between the camera 5 and each object point on the subject W can be detected from the image acquired by the distance detection imaging unit 63. Hereinafter, an image acquired by the distance detection imaging unit 63 is referred to as a distance image.

  In this manner, a single camera 5 can acquire a plurality of element images that have no parallax in the y direction and parallax in the x direction, and acquire a distance image having distance information of the subject W. be able to. In this case, the interpolated image can be easily generated based on the distance information detected from the distance image, and a three-dimensional model of the subject W can be easily generated.

  Further, in the example of FIG. 6, the camera 5 of FIG. 11 may be used. In this case, a plurality of element images, distance images, and high-definition images can be acquired simultaneously. Thereby, correction of a plurality of element images, generation of an interpolation image, and generation of a three-dimensional model can be performed more easily.

In this example, the dichroic mirror 62 and the distance detection imaging unit 63 are provided in the camera 5, but the present invention is not limited to this. For example, the dichroic mirror 62 may be disposed between the lens array 4 and the camera 5, and the distance detection imaging unit 63 may be disposed in the reflection direction.

(6) Stereoscopic Image Generation Device Hereinafter, a stereoscopic image display device that displays a stereoscopic image using a plurality of element images acquired by the stereoscopic image acquisition device 100 will be described. FIG. 12 is a block diagram illustrating a configuration of the stereoscopic image display apparatus.

  As illustrated in FIG. 12, the stereoscopic image display apparatus 200 includes a control unit 201 and a display unit 202. The control unit 201 includes a CPU (Central Processing Unit), a memory, a hard disk, and the like. Image data corresponding to a plurality of element images is given from the stereoscopic image acquisition apparatus 100 to the control unit 201. The display unit 202 includes a three-dimensional display and displays a stereoscopic image by any method such as an integral photography method or a hologram method.

  FIG. 13 is a flowchart showing the operation of the control unit 201 of the stereoscopic image display apparatus 200. As shown in FIG. 13, the control unit 201 extracts a plurality of element image data respectively corresponding to a plurality of element images from the image data given from the stereoscopic image acquisition apparatus 100 (step S1). Next, the control unit 201 performs geometric distortion correction processing for removing distortion caused by the lens on each extracted element image data (step S2).

  Next, the control unit 201 performs a resolution correction process for improving the resolution on each element image data (step S3). As described above, the number of pixels of each element image is equal to or less than the value obtained by dividing the total number of pixels of the image sensor 5b of the camera 5 by the number of element lenses 4a. Therefore, the resolution of each element image can be improved by using information between a plurality of element image data. In the example of FIG. 6, a high-definition image of the subject W can be obtained by the camera 24. Therefore, the resolution of each element image can be improved using the image data obtained by the camera 24.

  Next, the control unit 201 acquires depth information of the subject W from the plurality of element image data (step S4). The depth information indicates the positions in the z direction of a plurality of object points of the subject W. In the example of FIG. 11, the depth information can be easily obtained from the distance image acquired by the distance detection imaging unit 63.

Next, the control unit 201 generates a three-dimensional model of the subject W by a stereo matching method or the like based on the acquired depth information (step S5). Next, based on the generated three-dimensional model, the control unit 201 generates image data in a format corresponding to the method of the stereoscopic image to be displayed (step S6). For example, when displaying a stereoscopic image by the integral photography method, the control unit 201 regenerates image data corresponding to a plurality of images having parallax based on the generated three-dimensional model. In this case, the control unit 201 can adjust the depth (the pop-out amount) of the displayed stereoscopic image according to the performance of the display unit 202 based on the generated three-dimensional model. The control unit 201 can also perform nonlinear conversion processing of the displayed stereoscopic image based on the generated three-dimensional model. Thereafter, the control unit 201 displays a stereoscopic image on the display unit 202 using the generated image data (step S7).

  When displaying a stereoscopic image by the integral photography method, the control unit 201 displays the stereoscopic image on the display unit 202 in step S7 using a plurality of element image data subjected to the resolution correction processing in step 3. May be. In this case, since it is not necessary to perform the processes of steps S5 and S6, a stereoscopic image can be displayed quickly. In this case, it is also possible to generate interpolation image data corresponding to an interpolation image between a plurality of element images based on the three-dimensional model generated in step S5.

(7) Correspondence between each constituent element of claims and each element of the embodiment Hereinafter, an example of correspondence between each constituent element of the claims and each element of the embodiment will be described. It is not limited to.

  In the above embodiment, the element lens 4a is an example of an element lens, the imaging optical system 1, the diffusion plate 2, and the lens 3 are examples of a light guide optical system, and the camera 5 is an example of a first photographing apparatus. Yes, the slit 12a, the first polarization region 21a, and the lens 51 are examples of light passing regions, the first direction is an example of the z direction, the second direction is an example of the x direction, and the third The direction is an example of the y direction.

  Further, the diffusing plate 2 or the single focus lens 31 is an example of a divergent optical element, the aperture plate 12 is an example of a light shielding member, the slit 12a is an example of an opening, and the first polarizer 21 is a first polarized light. It is an example of an element, the 1st polarization field 21a is an example of the 1st field, the 2nd polarization field 21b is an example of the 2nd field, and the 2nd polarizer 22 is the 2nd polarization element The lens 51 is an example of a long optical element, the distance detection imaging unit 63 is an example of a distance image acquisition unit, the camera 24 is an example of a second imaging device, and the half mirror 22 is It is an example of a division | segmentation optical element.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be effectively used for acquiring various stereoscopic images.

DESCRIPTION OF SYMBOLS 1 Imaging optical system 2 Diffusing plate 3,31 Lens 4 Lens array 4a Element lens 5,24 Camera 5a, 24a Imaging lens 5b, 24b Image pick-up element 11 Imaging lens 12 Aperture plate 13 Imaging lens 21 1st polarizer 22 Half mirror 23 Second polarizer 51 Single focus lens 51a Light shielding member 61 Light irradiation unit 62 Dichroic mirror 63 Distance detection imaging unit 100 Stereo image acquisition device 200 Stereo image display device 201 Control unit 202 Display unit

Claims (6)

A stereoscopic image acquisition device that acquires a plurality of parallax images,
A plurality of element lenses that respectively image light rays from the subject as a plurality of element images;
A light guiding optical system for guiding light rays from the subject to the plurality of element lenses;
A first imaging device that acquires the plurality of element images respectively formed by the plurality of element lenses;
The light guide optical system has an optical axis along a first direction, and forms a light passage region extending in a long direction in a second direction substantially orthogonal to the first direction, and the light passage region A light beam that passes through the plurality of element lenses, and a light beam that does not pass through the light passage region is not guided to the plurality of element lenses.
The plurality of element lenses are two-dimensionally arranged on a plane parallel to the second direction and a third direction substantially orthogonal to the first and second directions,
The light guide optical system includes a divergent optical element that diffuses or refracts a light beam so that the light beam passing through the light passage region spreads in a plane including the first and third directions. Acquisition device. Wherein the plurality of element lenses, said third slanted and three-dimensional image acquisition apparatus according to claim 1, characterized in that it is arranged so as to be aligned in a plurality of parallel lines to each other with respect to the direction of. A stereoscopic image acquisition device that acquires a plurality of parallax images,
A plurality of element lenses that respectively image light rays from the subject as a plurality of element images;
A light guiding optical system for guiding light rays from the subject to the plurality of element lenses;
A first imaging device that acquires the plurality of element images respectively formed by the plurality of element lenses;
The light guide optical system has an optical axis along a first direction, and forms a light passage region extending in a long direction in a second direction substantially orthogonal to the first direction, and the light passage region A light beam that passes through the plurality of element lenses, and a light beam that does not pass through the light passage region is not guided to the plurality of element lenses .
The light guide optical system is:
A first region in which the first polarization can be transmitted and a second polarization different from the first polarization cannot be transmitted; and the first polarization cannot be transmitted and the second polarization A first polarizing element having a second region through which can be transmitted;
A second polarizing element capable of transmitting the first polarized light and not transmitting the second polarized light;
Light from the subject is guided to the plurality of element lenses through the first and second polarizing elements in order,
The first by the first area of the polarization element you characterized in that the light passage region is formed standing body image acquisition device. Three-dimensional image acquisition apparatus according to any one of claims 1 to 3, further comprising wherein a distance image acquiring unit that acquires a distance image including the distance information of the object. Each of the plurality of element lenses, stereoscopic image obtaining apparatus according to any one of claims 1 to 4, characterized in that an adjustable variable focus lens focal length. Claim and further comprising a dividing optical element for guiding the part of light guided to the plurality of element lenses by the second imaging device and the light-guiding optical system to the second imaging device 1-5 The three-dimensional image acquisition apparatus in any one of.

Images