JP2013044893A - Compound-eye imaging device, and distance image acquisition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent crosstalk of light beams among a plurality of lenses that constitute a lens array without providing a light-shielding wall in a compound-eye imaging device using a lens array.SOLUTION: A first wavelength selection filter 2 is provided on a subject side of a lens array 1, and a second wavelength selection filter 7 is provided in front of an image pickup device 4 corresponding to the first wavelength selection filter 2. The first and second wavelength selection filters 2 and 7 transmit separate wavelength bands for respective lens pairs 111 and 112, 121 and 122, 131 and 132, and 141 and 142 of the lens array 1.

Description

  The present invention relates to a compound eye imaging apparatus including a lens array including a plurality of lenses and an imaging element that captures a compound eye image of the lens array, and a distance image acquisition apparatus using the compound eye imaging apparatus.

  Imaging devices are useful as image information acquisition means in portable devices, in-vehicle devices, medical devices, industrial devices, etc., but not only two-dimensional image information, but also the distance to the subject and the three-dimensional shape of the subject There is a high demand for acquiring information at the same time. In general, an imaging apparatus for such an application is increased in size, but in order to enable installation at any location, it is desired to reduce the size and thickness of the apparatus.

  Conventionally, as an imaging device that can obtain distance information, which is reduced in size and thickness, a lens array composed of a plurality of lenses and an imaging region are divided into a plurality of divided regions corresponding to the plurality of lenses of the lens array. A compound eye imaging device including an imaging element is already known.

  However, in a compound eye imaging device using a conventional lens array, a lens array and an imaging element are used in a compound eye imaging device using a lens array in order to prevent crosstalk of light beams between a plurality of lenses constituting the lens array. In general, a light shielding wall for isolating optical paths respectively formed by a plurality of lenses of the lens array is provided (for example, Patent Document 1, Patent Document 2, etc.). Since the light shielding wall is made of metal such as aluminum or stainless steel, and the width of the light shielding wall is thick, an image cannot be captured in an area under the light shielding wall of the image sensor, and becomes an unused area. There is a problem that the resolution of the obtained image becomes small. For this reason, for example, Patent Document 2 proposes providing a space between the light-shielding wall and the image sensor, but there is a limit in preventing crosstalk.

  An object of the present invention is to eliminate a light-shielding wall in order to reduce an unused area of an image sensor in a compound eye imaging device using a lens array, and to prevent crosstalk of light flux between a plurality of lenses constituting the lens array. It is to prevent.

  The compound-eye imaging device of the present invention includes a lens array in which a plurality of lenses are arranged in an array arranged at a position facing a subject, and a plurality of lenses constituting the lens array provided on the image plane side of the lens array. An imaging element that captures a compound eye image that is a set of images of the subject (single-eye images) that are substantially imaged by each of the lenses, a first wavelength selection filter that is disposed on the subject side of the lens array, and In correspondence with the first wavelength selection filter, there is a second wavelength selection filter disposed in front of the imaging device, and the plurality of lenses constituting the lens array is composed of two or more lens pairs. The focal lengths of the lens pairs are equal, and the first and second wavelength selection filters are divided corresponding to the plurality of lenses constituting the lens array, respectively. Characterized in that to transmit the wavelength band different for each.

  According to the compound eye imaging device of the present invention, it is possible to prevent crosstalk between a plurality of lenses constituting the lens array without providing a light shielding wall, and the unused area of the imaging element is reduced.

It is a block diagram of one Embodiment of the compound-eye imaging device of this invention. It is a figure which shows an example of the wavelength transmission characteristic of the wavelength selection filter of FIG. It is a figure which shows an example of the compound eye image acquired with the compound eye imaging device of FIG. 1 is an overall configuration diagram of an embodiment of a distance image acquisition device to which a compound eye imaging device of the present invention is applied. It is a process flowchart of the arithmetic unit of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows a configuration diagram of an embodiment of a compound eye imaging apparatus of the present invention. FIG. 1A is a schematic cross-sectional view of a configuration in which a subject is present in the direction of an arrow and the subject is imaged by the compound eye imaging device. FIG. 1B is a diagram of the compound eye imaging device viewed from the subject direction. It is a plane schematic diagram.

  In FIG. 1, reference numeral 1 represents a lens array. The lens array 1 comprises two surfaces, a subject side surface and an image surface side surface, and a plurality of lenses are arranged in an array in the surface. FIG. 1A shows that lens surfaces are provided on both the object side and the image side. Here, 1a is a lens provided on the subject side surface, 1b is a lens provided on the image side surface, and 1a and 1b are a set (hereinafter referred to as a lens set) to display the subject image on the image plane. To form an image.

  As shown in FIG. 1B, in the present embodiment, the lens array 1 includes eight lens sets 111, 112, 121, 122, 131, 132, 141, 142. In each lens set, a circle indicated by a solid line represents an aperture of an aperture array described later, and its diameter is constant. A circle indicated by a broken line represents the diameter of the lens 1a including the invalid area, and a circle indicated by a dotted line represents the diameter of the lens 1b. Here, the lens sets 111 and 112, 121 and 122, 131 and 132, and 141 and 142 each form a lens set pair (stereo pair).

  Reference numeral 2 denotes a first wavelength selection filter that is disposed on the object-side surface of the lens array 1 and transmits only a specific wavelength band. For example, a structure in which a dielectric film is laminated and laminated on the surface of a glass plate. I am doing. The wavelength selection filter 2 is divided corresponding to each lens set (lens) constituting the lens array 1 and has a structure that transmits different wavelength bands for each stereo pair (lens pair). In FIG. 1B, reference numerals 211 and 212 denote wavelength bands of light rays incident on the stereo pair consisting of the lens sets 111 and 112, and reference numerals 221 and 222 denote wavelength bands of light rays incident on the stereo pair consisting of the lens sets 121 and 122. Reference numerals 231 and 232 denote regions that limit the wavelength band of light incident on the stereo pair including the lens sets 131 and 132, and reference numerals 241 and 242 indicate regions that limit the wavelength band of light incident on the stereo pair including the lens sets 141 and 142, respectively. Show. Here, each area | region of the wavelength selection filter 2 is arrange | positioned so that the transmission wavelength band of an adjacent area | region may not overlap. In FIG. 2, an example of the wavelength transmission characteristic of each area | region of the wavelength selection filter 2 is shown. Reference numerals in parentheses indicate corresponding lens sets.

  Reference numeral 3 denotes an aperture array in which a circular hole (opening) 3a is provided on a plate-like member corresponding to each lens set of the lens array 1, and acts as a lens diaphragm. As described above, the diameter of each opening 3a is constant. The aperture array 3 is fixed to convex portions 1c provided at the four corners of the planar portion of the subject-side surface of the lens array 1. A wavelength selection filter 2 is bonded to the surface of the aperture array 3 on the subject side.

  Reference numeral 4 denotes an image pickup device such as a CMOS sensor that picks up an optical image of a subject formed by each lens set of the lens array 1, and is mounted on the substrate 5. Reference numeral 6 denotes a housing, which is fixed to the end of the lens array 1 on the subject side, and holds the lens array 1, the wavelength selection filter 2, the aperture array 3, and the like. A substrate 5 is fixed to the housing 6.

  Reference numeral 7 denotes a second wavelength selection filter that is disposed on the front surface of the image sensor 4 and transmits only a specific wavelength band. For example, the surface of a glass plate is similar to the first wavelength selection filter 2 described above. It has a structure in which dielectric films are multilayered and laminated. The second wavelength selection filter 7 is also divided corresponding to the lens set (lens) constituting the lens array 1 and has a structure that transmits different wavelength bands for each stereo pair (lens pair). The transmission wavelength bands of the respective regions of the first wavelength selection filter 2 and the second wavelength selection filter 7 correspond to each other.

  The lens array 1 will be described in more detail. Each lens set 111, 112, 121, 122, 131, 132, 141, 142 constituting the lens array 1 has, for example, lens surfaces having different radii of curvature and becomes a lens set pair (stereo pair) 2 The radius of curvature of the two lens sets is substantially the same. For example, the lens sets 111 and 112 have substantially the same radius of curvature. These two lens sets 111 and 112 constitute a lens set pair (stereo pair) that acts as a stereo camera lens, and together with the pixel area located at a position corresponding to both lens sets of the image sensor 4 constitutes a stereo camera. Yes. Similarly, the lens sets 121 and 122, the lens sets 131 and 132, and the lens sets 141 and 142 have substantially the same radius of curvature in each pair, and constitute a stereo camera together with the corresponding pixel region of the image sensor 4. ing.

  The design center wavelength is different for each lens set pair (stereo pair), and the focal length at the design center wavelength of each lens set pair is equal. In the case of the example of FIG. 2, the focal lengths at the c-line wavelengths (approximately 656 nm) of 111 and 112, the focal lengths at the d-line wavelengths of 121 and 122 (approximately 587 nm), and the e-line wavelengths of 131 and 132 (approximately) The focal length at 546 nm is designed to be equal to the focal length at the f-line wavelength (approximately 486 nm) of 141 and 142.

  The distance between the lens sets in a plane substantially perpendicular to the optical axis constituting the lens set pair (stereo pair) is the base line length of the stereo camera constituted by the lens set pair. In the present embodiment, as shown in FIG. 1B, the base line lengths of the stereo pairs 111 and 112, 121 and 122, 131 and 132, and 141 and 142 are B1 and are all substantially equal.

  Next, an image acquired by the compound eye imaging apparatus of FIG. 1 will be described. In the present embodiment, the image obtained by the image sensor 4 is a compound eye image 8 as shown in FIG. In FIG. 3, I 11, I 12, I 21, I 22, I 31, I 32, I 41, I 42 are individual eyes corresponding to the respective lens sets 111, 112, 121, 122, 131, 132, 141, 142 constituting the lens array 1. It is a statue. Here, I11 and I12, I21 and I22, I31 and I32, and I41 and I42 form a stereo image pair.

  First and second wavelength selection filters 2 and 7 are arranged in front of the subject side of the lens array 1 and the image pickup device 4, and the transmission wavelength band of these wavelength selection filters is different for each adjacent single-eye image. Thus, crosstalk of the light beam can be prevented without providing a light shielding wall. Here, an image is not obtained when the subject color and the transmission wavelength band of the wavelength selection filter are different, but as shown in FIG. 2, the transmission wavelength band is different for each lens set pair (stereo pair). Thus, a stereo image pair can be acquired in a wide wavelength band.

  Next, a distance image acquisition apparatus using the compound eye imaging apparatus of the present invention will be described. FIG. 4 shows a configuration diagram of an embodiment of the distance image acquisition apparatus. The distance image acquisition apparatus includes an imaging unit 10 that images a subject, and an arithmetic unit 20 that calculates a distance image based on an image captured by the imaging unit 10. The imaging unit 10 is a compound eye imaging device having the configuration shown in FIG. 1, and a description thereof is omitted here.

  The arithmetic unit 20 includes an image capture unit (single-eye image division unit) 21, an image correction unit 22, a parallax detection unit 23, a parallax synthesis unit 24, and a distance calculation unit 25. Although the arithmetic unit 20 has a control part, a memory, etc. in addition, it is abbreviate | omitted in FIG. For example, the arithmetic unit 20 is formed on the substrate 5 together with the imaging device 4.

  FIG. 5 is a process flowchart for explaining the operation of the arithmetic unit 20. Hereinafter, the operation of the arithmetic unit 20 will be described in detail with reference to FIG.

  The image capture unit 21 captures a compound eye image 8 as shown in FIG. 3 acquired by the imaging element 4 of the imaging unit (compound eye imaging device) 10, and I11, I12, I21, I22, I31, I32, I41, Separated into I42 individual images (step 101). These single-eye images are transferred to the image correction unit 22.

  The image correcting unit 22 performs image distortion correction on the individual images I11 to I42 using internal parameters and external parameters specific to the camera according to lens work accuracy and assembly accuracy (steps 102 and 103). Each single-eye image whose image has been corrected is transferred to the parallax detection unit 23.

  The camera-specific internal parameters and external parameters are obtained by using Zhang's method (“A flexible new technique for camera calibration” IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11): 1330-1334, 2000). be able to.

  The parallax detection unit 23 divides the image-corrected individual images of I11 to I42 into individual image pairs (stereo image pairs) of I11 and I12, I21 and I22, I31 and I32, and I41 and I42. Parallax (parallax image) is detected from (steps 104 to 106). The parallax detection unit 23 includes four parallax detection units (1) to (4), the parallax detection unit (1) detects the parallax image 1 from the single-eye image pairs I11 and I12, and the parallax detection unit (2). The parallax image 22 is detected from the single-eye image pairs I21 and I22, the parallax detection unit (3) detects the parallax image 3 from the single-eye image pair I31 and I32, and the parallax detection unit (4) detects the single-eye image pair I41 and The parallax image 4 is detected from I42. Note that it is also possible to detect parallax images 1 to 4 by sequentially selecting each single-eye image pair I11 and I21, I21 and I22, I31 and I32, and I41 and I42 with only one parallax detection unit.

  Here, with reference to FIG. 3, the parallax detection process will be described using the parallax detection unit (1) as an example. In the images I11 and I12 subjected to the image correction, the corresponding points of the subject in each image are shifted in the horizontal direction according to the distance of the subject. The shifted position is detected. The amount of deviation corresponds to parallax. For the parallax calculation, a known technique called block matching is used. In the parallax detection unit (1), in order to cut out the minute area 9 from the image I11 and search for the corresponding area, the minute area 9 'is similarly cut out from the image I12, and the evaluation value is calculated. As the evaluation value, for example, a sum of luminance differences (SAD) between two minute regions, a square sum of luminance differences (SSD), and the like are obtained. In this case, on the basis of the position of the minute area 9 of the image I11, the position of the minute area 9 ′ of the image I12 is moved within a predetermined range, and a cutout position that gives the minimum value of SAD or SSD is searched at each position. As a result, the positional deviation between images can be obtained in units of pixels. Since parallax in units of pixels often lacks accuracy, if necessary, parallax of less than a pixel (subpixel) is also estimated. It is common to use equiangular straight line fitting or parabolic fitting. In addition, the data used for calculating SAD and SSD can be a value obtained by differentiating luminance (for example, DOG) instead of luminance. In this way, the sub-pixel level parallax for all pixels is calculated and detected as the parallax image 1. Here, each pixel of the parallax image 1 corresponds to each pixel of the image I11. Also, the parallax is set to 0 for the pixels for which the parallax could not be detected.

  Similarly, the parallax detection units (2) to (4) also detect the parallax images 2 to 4 for the monocular image pair (stereo image pair) of I21 and I22, I31 and I32, and I41 and I42, respectively. The parallax images 1 to 4 obtained by the parallax detection unit 13 are transferred to the parallax synthesis unit 24.

  The parallax synthesizer 24 synthesizes the parallax images 1 to 4 to generate a parallax synthesized image (step 107). As described above, when the subject color and the transmission wavelength band of the wavelength selection filter are different from each other, an image cannot be obtained, and parallax cannot be detected. By synthesizing the parallax images 1 to 4, a portion that could not be detected is interpolated. The parallax composite image synthesized by the parallax synthesis unit 24 is transferred to the distance calculation unit 25.

  The distance calculation unit 25 calculates a subject distance for each pixel or minute area of the parallax composite image, and generates a distance image (step 108). The subject distance is calculated using the following formula based on the principle of triangulation from parallax.

  Here, L is the subject distance, B is the baseline length (B1 in FIG. 1), f is the focal length of the lens set, S is the parallax of the subject image detected from the pair of eye images, and p is the image sensor (CMOS sensor or the like). ) Pixel pitch (pixel size).

  As mentioned above, although one Embodiment of this invention was described based on drawing, this invention is not limited to embodiment shown in drawing. For example, it is not necessary that the number of lens sets constituting the lens array is eight and the number of lens set pairs is four. Accordingly, the wavelength selective filter region and the transmission wavelength band are also increased or decreased. The lens array may be configured by arranging a plurality of lenses on one surface on the subject side or the image side.

DESCRIPTION OF SYMBOLS 1 Lens array 2 1st wavelength selection filter 3 Aperture array 3a Aperture 4 Image sensor 5 Board | substrate 6 Case 7 2nd wavelength selection filter 10 Imaging unit (compound eye imaging device)
DESCRIPTION OF SYMBOLS 10 Arithmetic unit 21 Image capture part 22 Image correction part 23 Parallax detection part 24 Parallax synthesis part 25 Distance calculation part

Japanese Patent Laid-Open No. 2003-143459 JP 2009-201008 A

Claims (4)

A lens array provided with a plurality of lenses arranged in an array at a position facing the subject;
Imaging that captures a compound eye image that is a set of images of the subject (hereinafter referred to as a single-eye image) that are substantially imaged by each of a plurality of lenses that form the lens array provided on the image plane side of the lens array. Elements,
A first wavelength selection filter disposed on the subject side of the lens array;
In correspondence with the first wavelength selection filter, a second wavelength selection filter disposed in front of the image sensor,
The plurality of lenses constituting the lens array includes two or more lens pairs, and the focal lengths of the lens pairs are equal.
The first and second wavelength selection filters are respectively divided corresponding to a plurality of lenses constituting the lens array, and transmit different wavelength bands for each lens pair.
A compound eye imaging apparatus characterized by the above. The plurality of lenses constituting the lens array is composed of two or more lens pairs each having a different radius of curvature, and the two lenses constituting the lens pair have substantially the same radius of curvature, and the design center wavelength differs for each lens pair. The focal length at the design center wavelength of each lens pair is equal.
The compound eye imaging apparatus according to claim 1. An imaging unit that is a compound eye imaging device according to claim 1 or 2, and an arithmetic unit that calculates a distance image from a compound eye image captured by an imaging element of the imaging unit,
The arithmetic unit includes means for separating the compound eye image into a plurality of single-eye images, means for detecting a parallax image for each single-eye image pair corresponding to each lens pair, means for combining the parallax images, and combining Means for calculating a subject distance from the parallax image thus generated, and generating a distance image;
A distance image acquisition apparatus characterized by that.   The distance image acquisition apparatus according to claim 3, wherein the arithmetic unit further includes means for correcting image distortion for each single-eye image separated from the compound eye image.