JP2014003433A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of detecting substance information, angle information, and parallax information by a small module housing.SOLUTION: The imaging apparatus comprises: imaging means 13 which respectively capture images of a subject by receiving reflected light incident upon plural lenses 6a and 6b upon which reflected lights from the subject are respectively incident; and filter means 31 which is provided between the lenses 6a and 6b and the imaging means 13 and comprises plural polarization regions transmitting different polarization components therethrough. Thereby, the imaging apparatus captures images including image regions corresponding to plural types of polarization components, and calculates the parallax information, substance information, etc. using the images.

Description

  The present invention relates to an imaging apparatus capable of detecting material information and angle information together with parallax information.

A sensing camera that can detect the difference in the state of the substance and its angle information by performing image processing on the image captured by the imaging sensor through a polarizing filter that can control the direction of the polarization plane of the transmitted polarized light, A distance measuring sensor using a stereo camera system in which a compound eye lens is arranged on one image sensor and a distance to an object is calculated based on a pixel shift (parallax) between images formed by the respective lenses (Patent Document) 1) etc. are already known.
As an example of the above-described sensing camera, Patent Document 2 discloses a lens array 102 having two lenses 121a and 121b and a light beam transmitted through each lens 121a and 121b of the lens array 102 as shown in FIG. A polarizing filter 104 having at least two polarizer regions 141a and 141b that are region-separated and whose transmission axes are orthogonal to each other, and a plurality of images that receive light passing through the polarizer regions 141a and 141b of the polarizing filter 104 and shoot a subject image The solid-state imaging unit 106 having the solid-state imaging elements 162a and 162b and the imaging apparatus 100 including the vertical imaging image (S-polarization image) and the horizontal polarization image (P-polarization image) are captured. A technique for acquiring / discriminating invisible information such as road surface conditions based on the intensity (luminance) ratio of a polarized image is disclosed. .

However, the function of the distance measuring device as described in Patent Document 1 and the function of the sensing camera for detecting material and angle information as described in Cited Document 2 are realized by a single device. In the case of an imaging device that can simultaneously acquire distance information and substance information, it is necessary to separately provide a sensing module that detects substance and angle information and a ranging module that measures parallax. There was a problem of increasing.
The present invention has been made in view of the above-described situation, and an object thereof is to provide an imaging apparatus capable of detecting material information, angle information, and parallax information with a single small module housing. .

  In order to solve the above-described problem, the invention according to claim 1 is configured to capture a plurality of lenses each receiving reflected light from a subject and images of the subject by receiving reflected light entering each lens. An imaging unit; and a filter unit that is provided between the lens and the imaging unit and includes a plurality of polarization regions that transmit different polarization components, and each of the images captured by the imaging unit includes a plurality of types of images. It has an image area corresponding to the polarization component.

  Since it comprised as mentioned above, according to this invention, the imaging device which can image the image which can acquire and detect substance information, angle information, and parallax information simultaneously with the single module with the same housing volume as the past Can be realized.

FIG. 3 is a cross-sectional view illustrating a basic configuration of an imaging module of the imaging apparatus according to the present embodiment. The figure explaining the polarizing glass chip with which the imaging module shown in FIG. 1 is provided. The figure which shows the polarizing filter element installed on the glass wafer. The figure explaining the wire grid element as an example of the polarizing filter element applicable to this embodiment. The figure which shows the structure of the other polarizing filter element concerning this embodiment, and the relationship with the pixel of an image pick-up element. The figure which showed the structure of the imaging device which included the processing circuit in the imaging module (imaging device) shown in FIG. FIG. 3 is a diagram for explaining in detail the configuration of a signal processing circuit in the imaging apparatus according to the present embodiment. The figure which shows the structure of the conventional sensing camera.

Embodiments of the present invention will be described below in detail with reference to the drawings.
FIG. 1 is a cross-sectional view illustrating a basic configuration of an imaging module according to the imaging apparatus according to the present embodiment.
As shown in FIG. 1, an imaging module 1 according to an imaging apparatus according to the present embodiment is mounted on a circuit board 3 and holds a lens array 6 in an opening 5a provided on the upper surface (subject side). 5 and an imaging chip 10 mounted on the circuit board 3 housed in the lens holder 5 and a polarizing filter element 31 that selectively extracts specific polarized light from light incident from the lens array 6 on one surface. A polarizing glass chip 30 is provided.
In order to obtain parallax information between two or more images, the lens array 6 includes a plurality of lenses 6a, 6b,... That are arranged in the same plane so as to form two images. Yes.

Although two lenses 6a and 6b are shown in FIG. 1, the present invention is not limited to this, and three or more lenses may be provided.
In addition, on the upper surface of the lens holder 5 above the lens array 6, an aperture 7 including windows 7a, 7b,... That transmits light corresponding to the lenses 6a, 6b,.
Note that the number of windows provided in the aperture 7 increases or decreases according to the number of lenses included in the lens array 6.

[Imaging chip]
The imaging chip 10 includes a semiconductor substrate 11, an inorganic insulating film 12 that is an oxide film such as SiO _2, an imaging element (imaging sensor) 13 such as a CMOS sensor, an electrode pad 14, as an example of an imaging unit, A microlens 15 and a structural film 16 are provided.
The image sensor 13 is provided between the semiconductor substrate 11 and the inorganic insulating film 12 and converts light incident through the lens array 6 and the polarization filter element 31 into an electric signal.
In the example of FIG. 1, the electrode pad 14 is on the same surface of the substrate 11 with respect to the imaging sensor 13, and is provided near the outer periphery of the light receiving surface of the imaging sensor 13 that faces the microlens 15. Then, an electric signal from the image sensor 13 is supplied to an external circuit.
That is, the electrode pad 14 is electrically connected to the substrate-side electrode pad 4 provided on the circuit board 3 by wire bonding using the wire 5.
Of course, the position where the electrode pad 14 is provided is not limited to the same plane as the imaging sensor 13, and may be provided on the side surface or the bottom surface of the semiconductor substrate 11.

The entire outer periphery of the wire 5 is sealed with a thermosetting resin 17 that is a potting material.
Further, the microlens 15 is formed on the other surface side of the inorganic insulating film 12 so as to face the image sensor 13, for example, with an organic material, and the incident light is transmitted through the inorganic insulating film 12. This lens leads to the image sensor 13.
The structural film 16 is a thermoplastic resin film formed by spin coating for bonding to a spacer member 34 of the polarizing glass chip 30 described later, and is an organic material or microlens equivalent to the material of the microlens 15. It is made of an organic material different from the material 15 and has the same film thickness as the microlens 15.
Although the microlens 15 and the structural film 16 are provided on the inorganic insulating film 12, the microlens 15 is provided directly on the imaging sensor 13 without providing the inorganic insulating film 12, and the structural film 16 is directly provided on the semiconductor substrate 11. You may make it provide in.

[Polarized glass chip]
2A and 2B are diagrams for explaining a polarizing glass chip provided in the imaging module shown in FIG. 1, wherein FIG. 2A is a plan view seen from the subject side, and FIG. 2B is a cross-sectional view.
As shown in FIGS. 1 and 2, the polarizing glass chip 30 includes a glass wafer 32 and a polarizing filter element 31 having a predetermined thickness D that is formed on one surface of the glass wafer 32 and defines the polarization direction of incident light. And.
The polarization filter element 31 includes a P polarization component that is a vibration component (polarization component) whose polarization plane is parallel to the incident surface, an S polarization component that is a vibration component whose polarization plane is perpendicular, and other polarization components. This is a filter element that can selectively extract either a P-polarized component or an S-polarized component when random light including it is incident.
In addition, the polarizing filter element 31 is disposed immediately above the light receiving surface of the imaging element 13. In this embodiment, the polarizing filter element 31 controls the polarization direction that is transmitted in the area of the pixel level of the imaging element 13. It is composed.
That is, the polarization direction of the polarizing filter element 31 that is transmitted at the pixel level is different.
With this structure, the polarization direction of light that is predominantly received by each pixel of the image sensor 13 is defined.

There is a sensor that can acquire polarization information such as polarization intensity by image processing.
Note that the thickness D of the polarizing filter element 31 is smaller than the wavelength of incident light, and optical characteristics generated by a structure smaller than the wavelength of incident light can be used.
The respective images formed and acquired by the imaging sensor 13 through the lenses 6a and 6b and the polarization filter element 31 have their planes of polarization orthogonal to each other by image processing by a signal processing circuit 40 (FIG. 6) described later. It is output as a distribution image of the intensity ratio of two polarized lights (P-polarized light and S-polarized light).
Since the reflectance of polarized light varies depending on the incident substance and the incident angle, the difference in material and angle clearly appears in this polarization intensity ratio image. Therefore, it is possible to extract feature points and shapes that have been difficult to detect in the past.
In the configuration of the present invention, the intensity ratio distribution between the P-polarized light and the S-polarized light is included in each of the images picked up through the lenses 6a and 6b. Therefore, the intensity ratio is calculated among a plurality of polarized images. There is no need.

The light transmitted through the aperture 7 is imaged on the light receiving surface of the image sensor 13 by the respective lenses 6a, 6b,. Therefore, the image sensor 13 outputs an image array divided into the number of lenses 6 a and 6 b formed in the lens array 6.
When the curvatures of the respective lenses included in the lens array 6 are equivalent, images acquired by the respective lenses are substantially equivalent images, but the distance (base length) between the lenses included in the lens array 6 and the target Deviation (parallax) of the image position according to the distance of the object occurs.
The distance information to the object can be calculated from the parallax information and the baseline length information.
When the curvatures of the lenses included in the lens array 6 are different, images having different focal points are formed (for example, front side focus, intermediate position focus, and back side focus).
By extracting a region in which the focal points of each image are in focus and synthesizing a plurality of images, it is possible to output an image in which the focal points are matched with respect to the entire distance.

Further, a light shielding structure (light shielding wall 35) is provided between the polarizing filter element 31 and the lens array 6 to prevent light transmitted through each lens constituting the lens array 6 from being received by a light receiving region other than the light receiving region immediately below the lens. ).
By preventing light other than the pair of the lens array 6 and the light receiving surface, generation of a ghost image can be prevented, and parallax information can be obtained accurately.
As the polarizing filter element 31, for example, a wire grid element formed in a metal fine uneven shape, an auto-cloning photonic crystal method, a polarizing film using an organic material, or the like can be used.
As shown in FIGS. 1 and 2, the polarizing glass chip 30 is provided near the outer peripheral end of the glass wafer 32 or in the vicinity of the outer peripheral end, and for providing a predetermined gap between the microlens 15 and the glass wafer 32. A spacer member 34 is provided.
A gap 36 is provided between the spacer member 34 and the polarizing filter element 31 on the glass wafer surface.
The polarizing glass chip 30 is provided so as to cover all the microlenses 15 in the light receiving area of the imaging chip 10.

The spacer member 34 is heated (for example, at 200 ° C. for 3 minutes) and pressurized (for example, 3 kgf) in a state of being pressed against the structural film 16 of the imaging chip 10, and welded to the structural film 16.
Thereby, the microlens 15 in the light receiving region of the image sensor 13 is sealed by the polarizing glass chip 30.
In the imaging module 1 having such a configuration, the light receiving area where the microlens 15 of the imaging chip 10 is provided is covered and sealed by the polarizing glass chip 30.
With such a configuration, it is possible to prevent foreign matter from entering the light receiving region.
Further, the gap between the microlens 15 and the polarizing filter glass (polarizing glass chip 30) can be reduced by adjusting the height of the spacer member 34. Therefore, it is possible to prevent a crosstalk phenomenon in which a plurality of polarized lights are received.
The spacer member 34 may be directly welded to the inorganic insulating film 12 of the imaging chip 10 without providing the structural film 16 on the imaging chip 10.
According to the imaging module having the above-described configuration, it is possible to prevent foreign matter from entering the light receiving region without impairing the light receiving characteristics.

The polarizing filter element will be described in detail below.
FIG. 3 is a diagram showing a polarizing filter element installed on a glass wafer.
As shown in FIG. 3, the polarization filter element 31 is a region-dividing polarizer filter having two polarizer regions 31 a and 31 b having different polarization planes by 90 degrees.
In FIG. 1, the polarizing filter element 31 is displayed so as to be provided on the imaging element side of the glass wafer 32, but may be provided on the surface opposite to the imaging element as shown in FIG. 3. .
The polarizer regions 31a and 31b shown in FIG. 3 transmit non-polarized light (random polarized light) in which an electromagnetic field vibrates in an unspecified direction, and transmits only the vibration component (polarized light component) in the direction along the polarization plane. (S-polarized light / P-polarized light).

FIG. 4 is a diagram illustrating a wire grid element as an example of a polarizing filter element applicable to the present embodiment.
As an image pickup device, a region-dividing polarizer filter with a clear boundary is obtained by using a wire grid method formed with a metal fine concavo-convex shape as shown in FIG. 4 or an auto-cloning photonic crystal method. Can be obtained.
A unique optical system that can absorb orthogonal polarization components by regularly arranging metal structures such as Al (aluminum) and Cr (chromium) with a thickness and pitch of several hundreds of nanometers on a substrate such as glass. The phenomenon appears.
The wire grid element is a polarizer formed by periodically arranging thin metal wires.
The structure of the wire grid type polarizer has a structure in which fine metal wires that are sufficiently thin compared to the wavelength of the input light are arranged at intervals that are sufficiently short compared to the wavelength.
When light is incident on such a structure, polarized light parallel to the metal thin wire is reflected and polarized light orthogonal thereto is transmitted.
Since the direction of the metal thin line can be changed independently for each region in a single substrate, the characteristics of the wire grid element can be changed for each region and further for each pixel of the corresponding image sensor. Is possible.

The pattern of the polarization region in the polarization filter element 31 is not limited to that shown in FIG.
In FIG. 3, the polarization filter element 31 is configured such that pixels that receive S-polarized light and pixels that receive P-polarized light in the imaging sensor 13 appear alternately in a matrix, but S-polarized light or P The pattern of the polarizing filter element 31 may be configured so that pixels that receive the polarized light appear in strips and alternately.
In addition, each pixel on the image sensor does not receive either S-polarized light or P-polarized light, but a pixel that dominantly receives either S-polarized light or P-polarized light, and S-polarized light The polarization filter element 31 may be configured such that there is a pixel that receives light mixed with P-polarized light.
Further, by arranging the image sensor pattern 31c tilted with respect to the polarization filter element 31, the required mounting accuracy conditions for the polarization filter element and the image sensor can be relaxed.

FIG. 5 is a diagram showing the structure of another polarizing filter element according to an embodiment of the present invention and the relationship with the pixels of the image sensor, (a) is a schematic diagram showing the light receiving surface of the image sensor 13, In the region of the light receiving surface of the image sensor 13, a plurality of pixels 13a each having a size of several μm are arranged in a lattice pattern.
FIG. 5B is a diagram illustrating a state in which the polarizing filter element 31 having a plurality of patterns capable of extracting mutually orthogonal polarized light is superimposed on the imaging sensor 13 illustrated in FIG.
For example, the pattern 31c transmits P-polarized light, and the pattern 31d transmits S-polarized light.
FIG. 5C is a diagram illustrating a state in which a polarization filter element 31 that can extract only one type of polarized light (for example, P-polarized light) is superimposed on the imaging sensor 13 illustrated in FIG.

For example, in FIG. 5C, with respect to the image sensor 13, the pattern 31c of the polarization filter element 31 has the same size as the pixel 13a (for example, 6 μm) of the image sensor 13 and a specific inclination (for example, 2 μm in length, horizontal). 1 μm).
When the pattern of the polarizing filter element 31 having such an inclination is formed on the glass wafer 32 and the glass with a filter and the image sensor 13 are bonded and mounted, all pixels on the light receiving surface of the image sensor 13 are polarized filter elements. Either the region (image region) that receives P-polarized light corresponding to the 31 pattern 31c becomes dominant, or the region corresponding to the region (image region) other than the pattern 31c becomes dominant. The area of the filter is not halved in each pixel.
In the case of FIG. 5B as well, in all the pixels on the light receiving surface of the image sensor 13, the region that receives P-polarized light corresponding to the pattern 31c of the polarization filter element 31 is dominant, or the pattern 31d Correspondingly, the region for receiving S-polarized light becomes dominant.
Therefore, high mounting accuracy is not necessary in mounting the filter and the image sensor.

FIG. 6 is a diagram illustrating a configuration of an imaging apparatus including a processing circuit in the imaging module (imaging apparatus) illustrated in FIG.
The signal processing circuit 40 in the figure includes a polarization information (brightness information) processing unit and a parallax information processing unit, and a substance using polarization information using the imaging module shown in FIG. 1 in common. A sensing function of information and angle information and a distance measuring function using parallax information can be realized at the same time.
Therefore, the imaging apparatus shown in FIG. 6 can realize the above-described plurality of functions with the same size as a conventionally known sensing camera and ranging camera.
The signal processing circuit 40 may be mounted on the substrate of the image sensor.
The output after processing by the signal processing circuit 40 is general image information, a polarization intensity ratio image, a feature point coordinate, a distance information image that expresses the distance to the object in color, and an image that is in focus in the entire distance range. Etc., can be set as required. A plurality of information may be output.

FIG. 7 is a diagram illustrating in detail the configuration of the signal processing circuit in the imaging apparatus according to the present embodiment.
In FIG. 7, in the imaging module 1 connected to the signal processing circuit 40, the lenses included in the lens array have the same shape.
A combination of images (single eye image and display) formed on the light receiving surface of the image sensor 13 by each lens 6a (single eye and display) of the lens array 6 is extracted, and parallax between the single eye images is calculated.
The distance to the object is calculated from the calculated parallax and the extracted inter-lens distance.

The signal processing circuit (arithmetic unit) 40 receives an image signal captured by the image sensor 13, and generates a plurality (for example, six of I1 to I6) of single-eye images from the image signal. A single-eye image generation unit 42 (separated into eye images), and a single-eye image generated by the single-eye image generation unit 42 are input, and the individual focused from these (single-eye images I1 to I6) A focused individual image pair selection extraction unit 43 that selects and extracts a pair of eye images, and an object that detects the parallax of the subject between the individual images by, for example, cross-correlation calculation for each minute region in the individual image An eye image pair parallax detection unit 44 and a distance calculation unit 45 that calculates the distance to the subject based on the detected parallax are provided.
For the extraction of the in-focus pair by the in-focus individual image pair selection extraction unit 43, a method of selecting the most focused individual image from the individual images I1 to I6 to obtain a focused image, A method of extracting the most focused pixel for each corresponding pixel from the images I1 to I6, combining these pixels, and outputting the focused image can be considered.
In the distance calculation unit 45, the distance can be calculated by the following calculation.

The distance between the optical axes of the lenses 6a and 6b is called the base line length, which is D, where A is the distance between the lens and the subject, and f is the focal length of the lens. Holds.
[Formula 1]
A = Df / Δ
Since the baseline length D and the focal length f of the lens are known, the distance A to the subject can be calculated using [Equation 1] if the parallax Δ is detected.
The signal processing circuit (arithmetic unit) 40 calculates an intensity ratio between the S-polarized light and the P-polarized light and outputs a contrast of the polarization intensity, and a difference between substances from the calculated intensity ratio. A substance / angle information calculation unit 47 that outputs an image in which a boundary portion such as a difference in angle is emphasized.

In the case of the example shown in FIG. 5, the substance / angle information calculation unit 47 calculates the intensity of each polarization from the intensity ratio of neighboring patterns, and outputs it as the contrast of the polarization intensity. It is possible to output an image in which the boundary portion such as the difference is emphasized. Therefore, it is possible to recognize transparent substances that could not be recognized conventionally.
As described above, this utilizes the well-known phenomenon that the reflection intensity of polarized light generally differs depending on the material and the incident angle. For example, among two types of polarized light, P-polarized light and S-polarized light, the substance A reflects P-polarized light strongly, and the substance B reflects S-polarized light strongly.
Note that the signal processing circuit 40 determines internal parameters related to the center position, focal length, and lens distortion of each lens based on luminance information and the like incident on the image sensor 13, and the distortion amount due to the ambient temperature of the light shielding wall 35. May be calculated and corrected.
In addition, when the curvature of the microlens 15 is different, an in-focus area extraction process for each image and a circuit are added.

In the conventional imaging apparatus shown in FIG. 8, polarizing filters 141a and 141b having different polarization directions that are transmitted are formed on one surface below the two lenses 121a and 121b forming the stereo camera.
Therefore, two types of images (P-polarized image and S-polarized image) having different polarization directions are formed (captured) by the imaging sensors 162a and 162b corresponding to the individual polarizing filters 141a and 141b.
Although these images are substantially the same image, the luminances of the two images are large, so that the image processing load required for extracting the same portion of the two images increases.
Therefore, there arises a problem that the module cost increases by the processing circuit used for the image processing.

On the other hand, in the imaging apparatus according to the present embodiment, the polarization filter element 31 is formed so that the polarization direction of transmission is different for each pixel, and is mounted corresponding to each pixel of the imaging sensor 13.
Reflected light that is incident on and reflected from a substance has a different reflectance for polarized light depending on the material and angle. Therefore, the intensity of light transmitted through the polarizing filter in each pixel is the intensity of the corresponding pixel in the adjacent image. The feature points can be easily extracted by calculating the ratio (contrast).
Further, by averaging the intensities of adjacent pixels, an image equivalent to a general luminance image is formed, so that it is easy to measure the pixel shift of the stereo camera.
For this reason, the structure of the present invention facilitates signal processing and can provide an inexpensive module.

  DESCRIPTION OF SYMBOLS 1 Imaging module, 3 Circuit board, 4 Electrode pad, 5 Lens holder, 5 Wire, 5a Aperture, 6 Lens array, 6a Lens, 6a Each lens, 7 Aperture, 7a Window, 10 Imaging chip, 11 Semiconductor substrate, 12 Inorganic Insulating film, 13 Imaging sensor, 13a Pixel, 14 Electrode pad, 15 Micro lens, 16 Structure film, 17 Thermosetting resin, 30 Polarized glass chip, 31 Polarizing filter element, 31a Polarizer region, 31c pattern, 31d pattern, 32 Glass Wafer, 33 Polarizing filter glass, 34 Spacer member, 35 Shading wall, 36 Gap, 40 Signal processing circuit, 41 Image capture unit, 42 Eye image generation unit, 43 Focusing eye image pair selection extraction unit, 44 Eye image Pair parallax detector, 45 distance calculator, 46 intensity ratio calculator, 47 objects Quality / Angle Information Calculation Unit, 100 Imaging Device, 102 Lens Array, 104 Polarization Filter, 106 Solid-State Imaging Unit, 121a Lens, 141a Polarizer Region, 141a Polarization Filter, 141a Polarizer Region, 162a Solid-State Imaging Device, 162a Imaging Sensor

JP 2011-209269 A JP 2010-025915 A

Claims (7)

A plurality of lenses each receiving reflected light from a subject;
Imaging means for receiving reflected light incident on each of the lenses and capturing images of the subject;
A filter unit provided between the lens and the imaging unit and including a plurality of polarization regions having different polarization components to be transmitted;
Each image picked up by the image pickup means has an image region corresponding to a plurality of types of polarization components. The imaging device according to claim 1,
The image pickup apparatus, wherein the filter unit includes a polarization region for extracting a vertical polarization component and a polarization region for extracting a horizontal polarization component. The imaging device according to claim 1,
The filter means includes: a polarization region that extracts a vertical polarization component or a horizontal polarization component; and a polarization region that transmits a non-polarization component including both the horizontal polarization component and the vertical polarization component. apparatus. In the imaging device according to any one of claims 1 to 3,
Signal processing means for processing a plurality of the images picked up by the image pickup means and calculating parallax information relating to the subject;
The image processing apparatus, wherein the signal processing unit calculates a distance from the subject using disparity information based on a plurality of the images. In the imaging device according to any one of claims 1 to 3,
The image processing apparatus, wherein the signal processing unit calculates material information or angle information of the subject based on a luminance intensity ratio between pixels on which different polarization components are imaged in the imaging unit. In the imaging device according to any one of claims 1 to 5,
The image pickup apparatus, wherein the polarization region of the filter means is inclined with respect to the pixel arrangement of the image sensor. In the imaging device according to any one of claims 1 to 6,
An image pickup apparatus comprising: a light blocking structure that prevents light transmitted through each lens from being received by an image pickup unit other than immediately below the lens between the filter unit and the plurality of lenses.

Images