JP2008157851A - Camera module - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a compact, thin camera module that can measure distance to an object, aims at gaining both of daytime color reproducibility and nighttime sensitivity, and can be utilized day and night continuously without providing any mechanical mechanism for turning on/off an infrared cut filter. <P>SOLUTION: The camera module has a plurality of imaging units, having a lens 1, an optical filter 2, and an imaging region 4 having a number of pixels. The plurality of imaging units comprise: a red light imaging unit; a blue light imaging unit; a green light imaging unit; and an infrared light imaging unit. There are each two imaging units selected from the red light imaging unit, blue light imaging unit, and the green light imaging unit, and each two infrared light imaging units. The camera module has an arithmetic circuit 5 for comparing and computing image information outputted from each imaging region of the plurality of imaging units, and each lens of the plurality of imaging units is integrated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a camera module that can be used both day and night, is small and thin, and can measure the distance to an object.

  Conventionally, in a method of measuring the distance to an object by parallax using two cameras, that is, stereo cameras, there is a mechanical attachment error of each camera. For example, the distance between the optical axes of two cameras and the deviation of the parallelism between the optical axes appear as parallax errors in the stereo image, and the calculated distance is a value far from the accurate distance. End up. Therefore, in order to obtain highly reliable distance information, a highly accurate mounting mechanism and complicated image processing for correcting mounting errors are required. For example, in Patent Document 1, as a method for correcting a mounting error of a stereo camera, a first vanishing point is obtained from the intersection of a plurality of approximate straight lines extending in the distance direction on one captured image plane. Similarly, in the other imaging plane, the second vanishing point is calculated from the intersection of a plurality of approximate straight lines extending in the distance direction and the amount of deviation between the first vanishing point and the second vanishing point. The error is corrected based on the above. Specifically, the white line of the road which exists in both right and left sides is detected, and the vanishing point is calculated using the edge of the white line.

  On the other hand, in order to increase the nighttime recognition rate, there is an image processing method in which one visible light camera and one infrared camera are combined to perform image processing on each piece of imaging information and increase the nighttime recognition rate. Proposed. For example, Patent Document 2 discloses a binarized infrared image by determining a binarization threshold value of an infrared image based on temperature information corresponding to a high-luminance portion such as a tail lamp of a binarized visible image. The vehicle is recognized based on.

  In recent years, especially in a vehicle or a surveillance camera, there is an increasing need for distance measurement at night as well as distance measurement to an object in the daytime.

Therefore, a distance measurement system that combines two visible light cameras for daytime measurement and two near-infrared cameras for nighttime measurement has been proposed.
JP 2003-83742 A Japanese Patent Laid-Open No. 10-255019 JP 2004-12863 A

  However, when calculating the distance by stereo vision, it is necessary to increase the mounting accuracy of the two cameras for visible light and near infrared, or to perform complicated arithmetic processing for error correction, This is a factor of cost increase.

  In Patent Document 3, a lens module in which two near-infrared distance measuring lenses for calculating the distance to a subject and one visible light photographing lens are arranged on the same plane is used. Yes. However, in this configuration, the distance measuring lens group and the photographic lens are separate configurations, and the distance measuring lens only calculates the movement amount of the photographic lens for the automatic focus adjustment function (AF). The image is taken with a photographic lens. Therefore, separate optical systems are required for distance measurement and photographing, which requires a large number of parts and space, increasing costs and increasing the size of the camera.

  In order to solve the above problems, a camera module according to the present invention is a camera module including a plurality of imaging units each having a lens, an optical filter, and an imaging region having a large number of pixels. A red light imaging unit for imaging red light, a blue light imaging unit for imaging blue light, a green light imaging unit for imaging green light, and an infrared light imaging unit for imaging light in the infrared wavelength band. Each of the red light imaging unit, the blue light imaging unit, and the green light imaging unit, and the two infrared light imaging units. An arithmetic circuit for comparing and calculating image information output from each imaging region is provided, and each of the lenses of the plurality of imaging units is integrated. It is a camera module.

  The camera module of the present invention can measure the distance of an object not only for color images but also for daytime and nighttime. Also, sharing the arithmetic circuit for distance measurement during the daytime and nighttime is advantageous for downsizing of the system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is an exploded perspective view of the camera module of Embodiment 1, and FIG. 2 is a cross-sectional view of the camera module as seen from the side.

  In FIG. 2, 1 is a lens module, 2 is an optical filter module, 3 is a substrate, 4a to 4c are imaging regions, and 5 is an arithmetic circuit including a digital signal processor (DSP). The arrows in the diagram of FIG. 1 mean light rays from the subject. In FIG. 2, reference numeral 6 denotes a fixing base that supports and fixes the lens module 1 and the optical filter module 2 and fixes the lens module 1 and the optical filter module 2.

  In FIG. 1, the lens module 1 is formed by integrally forming six lenses 1a to 1f in the same plane. Each lens is individually designed to satisfy optical specifications such as required MTF for red, blue, green and infrared (IR) light, which are the three primary colors of light. The lenses 1a to 1f are made of glass, plastic, or the like, and serve to form an image of light from the object that has passed through the optical filter module 2 on the imaging regions 4a to 4f. The lens 1a is green, 1b is red, 1c is infrared, 1d is blue, 1e is green, and 1f is designed for each wavelength band of infrared.

  Similarly to the lens module 1, the optical filter module 2 is formed of optical filters 2a to 2f, and is designed to transmit only the red, blue, green, and infrared wavelength bands for each corresponding lens. Yes. That is, 2a is green, 2b is red, 2c is infrared, 2d is blue, 2e is green, and 2f transmits light in each wavelength band of infrared. The imaging areas 4a to 4f are imaging sensors such as CCDs. The light that has passed through the lenses 1a to 1f of the lens module 1 passes through the optical filters 2a to 2f, and each light in each of the red, blue, green, and infrared wavelength bands is coupled to each imaging region. Image. As a result, the imaging area 4a forms an image of light in the green wavelength band, the imaging area 4b forms an image of light in the red wavelength band, and the imaging area 4c forms an image of light in the infrared wavelength band. The region 4d forms an image of light in the blue wavelength band, the imaging region 4e forms an image of light in the green wavelength band, and 4f forms an image of light in the infrared wavelength band. That is, the light from the object is separated into the red, green, blue, and infrared wavelength bands to form images. Each of the imaging regions 4a to 4f forms an image of light separated for each color on each imaging region, and outputs image information as an electrical signal (not shown) photoelectrically converted according to the intensity of the received light. To do. Various arithmetic processing, that is, image processing, is performed on each photoelectrically converted electric signal. For example, image processing such as creating a color image by combining images captured in four imaging regions 4a, 4b, 4d, and 4e is performed. These arithmetic processes are performed using the DSP 5 or the like.

  A set of imaging units including one lens, an optical filter corresponding to the lens, and an imaging region is defined as an imaging unit.

  In the camera module configured as described above, in the daytime, green light from the subject passes through the lenses 1a and 1e and the optical filters 2a and 2e, respectively, and forms images on the imaging regions 4a and 4e, respectively. Since the images picked up in the image pickup areas 4a and 4e are the same green, the images are shifted by the amount of parallax. Accordingly, by calculating the electrical signal obtained as a result of the photoelectric conversion, that is, image information, the parallax between the imaging units can be calculated, and the distance from the parallax to the object can be calculated.

  Similarly, at night, when an object is illuminated with an infrared light source, infrared light from the object passes through the lenses 1c and 1f and the optical filters 2c and 2f, respectively, and forms images in the imaging regions 4c and 4f, respectively. Since the images captured in the imaging regions 4c and 4f are the same infrared light image information, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  The measurement principle will be described with reference to FIG. The image of the imaging region (imaging surface) 4a is set as a reference image. The imaging area 4a is divided into specific blocks, for example, divided into blocks each having 8 × 8 pixels. Then, an area having a correlation with a certain block 4a is searched and specified in the image of the other imaging area 4e. Then, the distance to the object is calculated based on the parallax of the pixel block, that is, the relative shift amount of the pixel block, that is, the parallax.

  FIG. 5 shows a layout of each unit of the camera module in the present embodiment.

  In the daytime, in FIG. 4, the distance to the object is D, the lenses 1a and 1e are the same lens, the focal distance is f, the distance between the lenses 1a and 1e is L, and the parallax is Assuming Δa, the distance to the object can be obtained using (Equation 1).

  In the above configuration, since the lenses 1a and 1e are integrally formed as a lens module, the distance between the lenses is much higher accuracy and more stable than when assembled using two cameras. The distance between the objects can be maintained, and the distance to the object can be obtained with high accuracy.

  Since light in the red, green, and blue wavelength bands is incident on the camera module from the object during the daytime, for example, video signals obtained from the imaging areas 4a, 4b, and 4d are combined in consideration of parallax. Thus, it is possible to obtain one color image at the same time. Based on the image information of the green imaging regions 4a and 4e, the amount of displacement of the red and blue images with respect to the reference image 4a is calculated from the amount of relative displacement of 4e with respect to the reference image 4a. By correcting the image and synthesizing it with the reference image 4a, it is possible to obtain a color image without color misregistration after performing parallax correction.

  In addition, when shape recognition is performed using color images, it is possible to extract three components of luminance, hue, and saturation from the red, green, and blue wavelength components, and one geometrically connected region Since the colors are similar, color information can be used as a feature amount, and the shape recognition accuracy can be remarkably improved as compared with a monochrome image.

  The distance measurement principle at night is basically the same as that at daytime. That is, when the lenses 1c and 1f are the same lens, the focal length is f, the distance between the lenses 1c and 1f is L / √2, and the object whose distance is D is viewed, the parallax Δb is expressed by ). At night, it is preferable to provide an infrared light source to illuminate the object.

  The object at the distance D is shifted by a parallax of Δa in the comparison of the lenses 1a and 1e, and is shifted by a parallax of Δa / √2 in the comparison of the lenses 1c and 1f.

  On the other hand, when the unit pixel in the imaging region is p, the pixel pitches in the 71a and 71b directions are √2p and p, respectively.

  That is, although the amount of parallax is large in the lenses 1a and 1e, the pixel pitch in the direction in which the parallax is generated (here, 71a) is also coarse.

  The parallax decreases as the distance D of the object increases from (Equation 1). Therefore, the distance measurement of a distant object becomes possible as fine parallax can be read.

  When the parallax is read by directly comparing each pixel, the pitches of the pixels in the 71a and 71b directions are √2p and p, respectively, and the object at the distance D is the amount of parallax of Δa in the comparison of the lenses 1a and 1e. In the comparison between the lenses 1c and 1f, the distances of the objects having the parallax corresponding to the pitch of the pixels in the directions of 71a and 71b are exactly the same because they are shifted by the amount of parallax of Δa / √2.

  In addition, since it has sensitivity in the infrared wavelength band at night, for example, it is possible to simultaneously obtain a single monochrome image using image information obtained from the imaging region 4c.

  Since the lenses 1a, 1e, or 1c, 1f are integrally formed as a lens module, the distance between the lenses is much higher than that when assembled using two cameras. The distance can be ensured, and the distance to the object can be obtained by calculation with high accuracy.

  Here, the three primary color filters may be used as they are, but preferably the three primary color filters do not necessarily have to be a single filter, and have a characteristic of blocking infrared light. It is desirable. For example, a color filter having three primary colors and an infrared cut filter may be arranged to overlap each other, or a color filter having characteristics obtained by multiplying each of the RGB spectral characteristics by the spectral characteristics of the infrared cut filter may be configured. May be arranged.

  This camera module changes the operation mode during the day and at night.

  In the first mode, a distance measurement calculation is performed mainly using information of an imaging region where a photoelectric conversion signal of visible light, that is, an RGB color filter (multiplied by spectral characteristics of an infrared cut filter) is arranged. I do. Here, “mainly used” means that the signal output from the imaging region in which the G filter is arranged is larger than the signal output from the imaging region in which the infrared filter is arranged. This first mode is suitable for color images and is a mode used at daytime.

  On the other hand, in the second mode, the distance measurement calculation is performed mainly using the photoelectric conversion signal of infrared light, that is, the information of the imaging region where the IR filter is arranged. Here, “mainly used” means that the signal output from the imaging region in which the IR filter is arranged is larger than the signal output from the imaging region in which the green filter is arranged. This second mode has a high sensitivity and is used at night.

  As described above in detail, according to the present embodiment, image generation and object distance measurement are performed in the first mode when it is relatively bright such as daytime, and in the second mode when it is relatively dark such as at night. To improve both daytime color image color reproducibility and nighttime sensitivity.

  Further, in the present embodiment, the filter arrangement of the optical filter 2 is devised so that the infrared cut filter is applied only to the four RGBG imaging regions, and the infrared cut filter is not applied to the two IR imaging regions. I have to. Since switching between the first mode and the second mode is determined by software based on the electrical signal from the imaging region, it is necessary to provide a mechanical mechanism for turning on / off the infrared cut filter. Therefore, it is possible to reduce the size and weight of the imaging device, reduce costs, and improve reliability.

  In the above embodiment, the mode is switched from the signal output of each imaging region. However, a dedicated light receiving element for detecting the illuminance of visible light may be provided.

  Then, in the second mode, by irradiating the subject with infrared light and taking a picture, it becomes possible to take a picture of a darker subject (for example, photography in the darkness) that could not be photographed until now. In the above embodiment, the lens module 1 is configured as shown in FIG. 1, but in addition to these components, a light emitting element for infrared illumination (such as an infrared light emitting LED) may be further provided.

  Moreover, in the said embodiment, the camera module of this Embodiment is not limited to a digital camera. For example, the present invention can also be applied to a digital video camera, a mobile phone with a camera, a surveillance camera, a PDA (Personal Digital Assistant) with a camera, and the like. These terminals are suitable for applying this embodiment because there are many opportunities to shoot dark objects at night from bright subjects in the daytime.

  Further, the imaging region 4 described above can be applied to both the progressive method (all-pixel reading method) and the frame reading method as the photoelectric conversion signal reading method.

(Embodiment 2)
FIG. 6 is a layout diagram of units of the camera module according to the second embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 6, 9a and 9e are infrared wavelength units, 9b and 9d are green wavelength imaging units, 9c is blue, and 9f is red wavelength imaging units.

  In the daytime, in FIG. 1, green light rays from the subject are transmitted through the lenses 1b and 1d and the optical filters 2b and 2d, respectively, and are imaged in the imaging regions 4b and 4d, respectively. Since the images picked up in the image pickup regions 4b and 4d are the same green, they are images that are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, in FIG. 1, when an object is separately illuminated with an infrared light source, infrared rays from the object pass through the lenses 1a and 1e and the optical filters 2a and 2e, respectively, and the imaging regions 4a and Each image is formed at 4e. Since the images picked up in the image pickup areas 4a and 4e are the same infrared image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  In addition, since the unit for capturing two infrared light and the unit for capturing two green light are arranged in an oblique direction, both of the vertical line and the horizontal line have a pattern or an outer shape. The object can be detected.

  Note that it is desirable that the imaging area of each unit be divided from one imaging area, rather than using a plurality of imaging areas, because the accuracy and measurement stability of the object distance measurement are improved.

(Embodiment 3)
FIG. 7 is a layout diagram of units of the camera module according to the third embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 7, 10a and 10f are infrared wavelength units, 10c and 10d are green wavelength imaging units, 10b is blue, and 10e is red wavelength imaging units.

  In the daytime, in FIG. 1, green light rays from the subject are transmitted through the lenses 1c and 1d and the optical filters 2c and 2d, respectively, and are imaged in the imaging regions 4c and 4d, respectively. Since the images picked up in the image pickup regions 4c and 4d are the same green, they are images that are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, when the subject is illuminated separately with an infrared light source in FIG. 1, infrared rays from the subject pass through the lenses 1a and 1f and the optical filters 2a and 2f, respectively, and respectively in the imaging regions 4a and 4f. Form an image. Since the images captured in the imaging regions 4a and 4f are the same infrared image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  In addition, since the unit for capturing two infrared light and the unit for capturing two green light are arranged in an oblique direction, both of the vertical line and the horizontal line have a pattern or an outer shape. The object can be detected.

(Embodiment 4)
FIG. 8 is a layout diagram of units of the camera module according to the fourth embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 8, 11a and 11c are infrared wavelength units, 11d and 11f are green wavelength imaging units, 11b is blue, and 11e is red wavelength imaging units.

  In the daytime, in FIG. 1, green light rays from the subject are transmitted through the lenses 1d and 1f and the optical filters 2d and 2f, respectively, and are imaged in the imaging regions 4d and 4f, respectively. Since the images picked up in the image pickup regions 4d and 4f are the same green, they are images that are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, when the subject is separately illuminated with an infrared light source in FIG. 1, infrared rays from the subject pass through the lenses 1a and 1c and the optical filters 2a and 2c, respectively, and in the imaging regions 4a and 4c. Each forms an image. Since the images picked up in the image pickup regions 4a and 4c are the same infrared image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  In addition, since the parallax detection is performed in the day and night wavelength bands, the measurement resolution is also improved.

  For example, when a three-field interlaced image sensor is used, the first scanning line, the second scanning line, the third scanning line, the first scanning line,... The pixel information on each first scanning line is read at a time in the first time field, the pixel information on each second scanning line is read at a time in the second time field, and the pixel information on each second scanning line is read in the third time field. The pixel information on each scanning line 3 is read at a time.

  In the present embodiment, 11a and 11c are infrared wavelength units, and 11d and 11f are green wavelength imaging units, which are arranged in parallel with the scanning line, respectively, so that the first to third total time The pixel information of the field is not necessary, and the parallax can be detected only by the first time field.

  For this reason, since the pixel information data required for the calculation processing of the distance measurement is reduced, the distance detection time can be shortened, the responsiveness can be improved, the arithmetic circuit scale can be reduced, and the pixel information of the same field can be compared. Therefore, even if the object moves rapidly, the subject image is not easily displaced due to the time difference, and the distance measurement accuracy can be prevented from being lowered.

(Embodiment 5)
FIG. 9 is a layout diagram of units of the camera module according to the fifth embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 9, 12d and 12e are infrared wavelength units, 12b and 12c are green wavelength imaging units, 12a is blue, and 12f is red wavelength imaging units.

  In the daytime, in FIG. 1, green light from the object passes through the lenses 1b and 1c and the optical filters 2b and 2c, respectively, and forms images in the imaging regions 4b and 4c, respectively. Since the images picked up in the image pickup regions 4b and 4c are the same green, they are images that are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, when the subject is separately illuminated with an infrared light source in FIG. 1, infrared rays from the subject are transmitted through the lenses 1d and 1e and the optical filters 2d and 2e, respectively, and in the imaging regions 4d and 4e. Each forms an image. Since the images captured in the imaging regions 4d and 4e are the same infrared light image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  For example, when a three-field interlaced image sensor is used, the first scanning line, the second scanning line, the third scanning line, the first scanning line,... The pixel information on each first scanning line is read at a time in the first time field, the pixel information on each second scanning line is read at a time in the second time field, and the pixel information on each second scanning line is read in the third time field. The pixel information on each scanning line 3 is read at a time.

  In the present embodiment, 12d and 12e are infrared wavelength units, and 12b and 12c are green wavelength imaging units, which are arranged in parallel with the scanning line. This pixel information is not necessary, and the parallax can be detected only in the first time field.

  For this reason, since the pixel information data required for the calculation processing of the distance measurement is reduced, the distance detection time can be shortened, the responsiveness can be improved, the arithmetic circuit scale can be reduced, and the pixel information of the same field can be compared. Therefore, even if the object moves rapidly, the subject image is not easily displaced due to the time difference, and the distance measurement accuracy can be prevented from being lowered.

(Embodiment 6)
The output from each pixel of an image obtained through an image sensor such as a CCD is determined by the illuminance incident on the light receiving surface, that is, the optical sensor. At this time, since the light receiving surface has an area, the pixel value of the light incident on one pixel of the CCD is a value obtained by integrating the light intensity at each point on the light receiving surface. In FIG. 10, the hatched area is an area of one pixel. The distance between the centers of the pixels is the pixel pitch.

  In the method of measuring the distance to the subject using the triangulation method described in the first embodiment, the relative shift amount obtained from the two images is output from one pixel of the CCD as described above. The calculation is performed using the luminance value. That is, an image having different distance information or the like is captured within one pixel, but in reality, processing is performed using information of one pixel obtained by averaging the information of each point.

  Therefore, it can be understood that if not only the information of each pixel but also the luminance information between the pixels can be obtained, the measurement accuracy of the distance to the object can be increased. In this case, it can be seen that when measuring the distance to the subject while keeping the distance between the lenses short, it is possible to accurately measure farther distances.

  Therefore, as shown in FIG. 11A, an area smaller than one pixel (referred to as a sub-pixel) is set. In this case, one pixel is composed of a set of 5 × 5 subpixels. In FIG. 11A, a region surrounded by a solid line is one pixel, and a region surrounded by a dotted line is a subpixel. One subpixel is a hatched area.

  When measuring the distance to the object, in the same manner as described in the first embodiment, when searching for and specifying a region having a correlation with a block with 4a serving as a reference image in the other 4e image, one pixel unit It is possible to improve the measurement accuracy of the distance to the target object by taking the correlation in units of the sub-pixels described above instead of taking the correlation in step. Further, when measuring the distance to the object while keeping the distance between the lenses short, it is possible to accurately measure from a near distance to a far distance. In the example described in the first embodiment, the distance is calculated by measuring the parallax at one pixel pitch. In that case, it becomes impossible to calculate the distance of an object that is more than about 20 cm, which is a displacement amount of one pixel. On the other hand, by making one pixel into a sub-pixel of 1/5 block, the distance to the subject can be measured up to about 10 m. As a method for obtaining the pixel value of the sub-pixel, there is a method of obtaining the value by interpolating by linear interpolation from the pixel value output from one pixel in the imaging region.

  An example in which the pixel is divided into 2 × 2 subpixels will be described with reference to FIG.

  The pixel value output from one pixel is assumed to be at the center position of the pixel, and the pixel value output from the sub-pixel is also expressed to be at the center position of the pixel, and each pixel value of one pixel is represented by P (i, j), P (i, j), P (i, j), P (i, j), the pixel value of the subpixel to be obtained is P, and the distance from each pixel value to the pixel value of the subpixel is a1 , A2, b1, and b2, the pixel value P of the subpixel to be calculated can be obtained by (Equation 3).

  When obtaining the pixel value of a subpixel, a method of detecting the presence or absence of an edge in the subpixel and interpolating using the pixel values of adjacent subpixels, interpolating by combining linear interpolation and edge detection, etc. Can be obtained. In this embodiment, an example in which the pixel is divided into 2 × 2, 5 × 5 subpixels has been described. However, the number of divisions may be determined based on necessary accuracy or the like.

(Embodiment 7)
FIG. 12 is a layout diagram of units of the camera module according to the seventh embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 12, 13b and 13e are infrared wavelength units, 13a and 13f are green wavelength imaging units, 13c is blue, and 13d is red wavelength imaging units.

  In the daytime, in FIG. 1, green light from the object passes through the lenses 1a and 1f and the optical filters 2a and 2f, respectively, and forms images in the imaging regions 4a and 4f, respectively. Since the images picked up in the image pickup areas 4a and 4f are the same green color, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, in FIG. 1, when an object is separately illuminated with an infrared light source, infrared rays from the object pass through the lenses 1b and 1e and the optical filters 2b and 2e, respectively, and the imaging region 4b, Each image is formed at 4e. Since the images picked up in the image pickup areas 4b and 4e are the same infrared image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Further, in FIG. 4, the object at the distance D is shifted by a parallax of Δc in the comparison of the lenses 1a and 1f, and is shifted by a parallax of Δc / √5 in the comparison of the lenses 1a and 1f.

  On the other hand, when the unit pixel of the imaging region is p, the pitches of the pixels in the directions 71b and 71c are √5p and p, respectively.

  That is, although the amount of parallax is large in the lenses 1a and 1f, the pixel pitch in the direction in which the parallax occurs (here 71c) also becomes coarse.

  The parallax decreases as the distance D of the object increases from (Equation 1). The distance to a distant object can be measured as fine parallax can be read.

  When the parallax is read by directly comparing each pixel, the pitches of the pixels in the directions 71b and 71c are p and √5p, respectively, and the object at the distance D is only the amount of parallax of Δa in the comparison of the lenses 1a and 1c. In the comparison between the lenses 1a and 1f, the distance is shifted by the amount of parallax of Δc / √5. Therefore, the distance between the objects having the parallax corresponding to the pitch of the pixels in the 71b and 61c directions is exactly the same, which is effective for detection. is there.

  When measuring the distance to the object, as in the description in the first embodiment, when searching for and specifying a region having a correlation with the block serving as the reference image in the other image, a correlation is obtained in units of one pixel. Instead of taking the correlation in units of sub-pixels, it is possible to increase the measurement accuracy of the distance to the object.

(Embodiment 8)
FIG. 13 is a layout diagram of units of the camera module according to the eighth embodiment. Note that description and description similar to those of the first embodiment are omitted. In FIG. 13, 14a and 14c are infrared wavelength units, 14e and 14f are green wavelength imaging units, 14b is blue, and 14d is red wavelength imaging units.

  In the daytime, as shown in FIG. 1, green light rays from the subject pass through the lenses 1e and 1f and the optical filters 2e and 2f, respectively, and form images in the imaging regions 4e and 4f, respectively. Since the images captured in the imaging regions 4e and 4f are the same green, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  Similarly, at night, in FIG. 1, when an object is separately illuminated with an infrared light source, infrared rays from the object pass through the lenses 1a and 1c and the optical filters 2a and 2c, respectively, and the imaging region 4a and Each image is formed at 4c. Since the images picked up in the image pickup regions 4a and 4c are the same infrared image, the images are shifted by the amount of parallax. Electric signals obtained as a result of photoelectric conversion are processed, parallax is calculated based on each captured image data, and a distance from the parallax to the object is calculated.

  When measuring the distance to the object, as in the description in the first embodiment, when searching for and specifying a region having a correlation with the block serving as the reference image in the other image, a correlation is obtained in units of one pixel. Instead of taking the correlation in units of sub-pixels, it is possible to increase the measurement accuracy of the distance to the object.

(Embodiment 9)
In FIG. 14, reference numeral 100 is a view of the automobile as viewed from above. Reference numerals 101 to 104 denote camera modules, and 101a to 104a denote camera angles of view. 105 is a preceding vehicle, 106 is an oncoming vehicle, 107 is a road white line, and 108 is a pedestrian. By arranging a plurality of camera modules so that the angles of view overlap as much as possible as shown in FIG. 14, the blind spots from the driver's seat are eliminated, and the pedestrian 108, the white line 107, Information on the oncoming vehicle 106 and the preceding vehicle 105 can be detected. In addition, it is also possible to display the imaging result obtained from each camera module using a display device (not shown). Considering the design of the vehicle, aerodynamic characteristics, etc., a thin and small shape is desired to mount a plurality of camera modules on the vehicle. Since the camera module of the present invention uses a lens, an optical filter, and an imaging region for each wavelength band, the camera module can be realized with a single lens configuration, and can be very thin. In addition, since it has at least two optical systems of the same wavelength band, it is possible to measure the distance to the object using the triangulation method, and further preventive safety by combining with the vehicle speed and steering wheel steering angle signal Can be achieved. In addition, since it has both imaging information in the visible light region and the infrared region, it recognizes with distance information from bright situations such as headlights for oncoming vehicles to dark situations such as pedestrians at night. Is possible. Furthermore, since it is possible to obtain a color image in the visible light region, it is possible to extract three components of luminance, hue, and saturation from the red, green, and blue wavelength components, and geometrically Since one connected area is similar in color, color information can be used as a feature amount, and the shape recognition accuracy can be significantly improved as compared with a monochrome image. In addition, although the vehicle exterior is assumed as surrounding situation information, it goes without saying that it is also effective in the vehicle interior.

(Embodiment 10)
Since the automobile is moving at a certain speed, it is necessary to obtain the situation information in the direction faster and more accurately in order to proceed. In FIG. 15, a camera module 103 is a camera module for obtaining situation information in front of the vehicle. When the camera is fixed and attached, it is possible to obtain the front situation information only when the traveling direction of the vehicle body changes. In order to obtain the situation information ahead more safely and faster, the camera module 103 is rotated by the rotation mechanism 200 about the z-axis 202 as shown in FIG. Here, the x, y, and z axes are the directions shown in FIG. The vehicle traveling direction is the x axis, the horizontal direction is the y axis, and the direction orthogonal to each is the z axis. The rotation angle and the time until positioning settling are determined by estimating the traveling direction of the vehicle using the vehicle speed signal and the steering angle signal. In this way, by predicting the traveling direction of the vehicle and rotating the camera module around the z axis, it becomes possible to obtain situation information on the traveling direction before the traveling direction of the vehicle body changes, It becomes possible to provide higher safety. In order to rotate the camera module around the z-axis, the rotary table 200 is used in FIG. 15, but the present invention is not limited to this example, and a torsion bar 301 is provided at the rotation center as in the example of FIG. , By applying force F1 (302) and F2 (303) optimally in the x-axis direction, the mechanism that rotates the lens module, such as rotating around the z-axis, adopts the optimal mechanism according to the space to be mounted on the vehicle do it. In this embodiment, the camera module having six lenses has been described. However, if there are two or more lenses, the above-described high-precision distance measurement is possible. Further, as for the arrangement of the lenses used for measuring the distance to the object, an appropriate arrangement may be selected based on parallax, ranging accuracy, and the like. In this embodiment, the optical path from the lens to the imaging region is linear, but the same effect can be obtained by a reflection type in which the optical path is bent using a prism or the like.

  The camera module of the present invention is useful for mobile phones, digital still cameras, surveillance cameras, vehicle-mounted cameras, etc. that are small and thin and have camera functions.

1 is an exploded perspective view of a camera module according to Embodiment 1 of the present invention. Sectional drawing of the lens module of Embodiment 1 of this invention Explanatory drawing for demonstrating the parallax calculation of this invention Explanatory drawing for measuring the subject distance of Embodiment 1 of the present invention Arrangement of units of the camera module according to Embodiment 1 of the present invention Arrangement of units of the camera module according to the second embodiment of the present invention Arrangement of units of the camera module according to Embodiment 3 of the present invention Arrangement of units of the camera module according to Embodiment 4 of the present invention Arrangement of units of camera module according to Embodiment 5 of the present invention Conceptual diagram of the imaging sensor according to the sixth embodiment of the present invention. Conceptual diagram of sub-pixel according to Embodiment 6 of the present invention Arrangement of units of camera module according to embodiment 7 of the present invention Arrangement of units of camera module according to embodiment 8 of the present invention Explanatory drawing of the vehicle-mounted camera of Embodiment 9 of the present invention Configuration diagram of a camera module according to the tenth embodiment of the present invention Explanatory drawing of the reference axis of Embodiment 10 of this invention Explanatory drawing of the other rotation mechanism of Embodiment 10 of this invention. Conventional vanishing point calculation chart

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens module 1a-1f Lens 2 Optical filter module 2a-2f Optical filter 3 Board | substrate 4a-4f Imaging area 5 Digital signal processor 6 Fixing stand 7 Each unit of camera module 8 Each unit of camera module 9 Each unit of camera module 10 Camera Each unit of module 11 Each unit of camera module 12 Each unit of camera module 13 Each unit of camera module 14 Each unit of camera module

Claims (8)

A camera module including a plurality of imaging units having a lens, an optical filter, and an imaging region having a large number of pixels,
The plurality of imaging units include a red light imaging unit that captures red light, a blue light imaging unit that captures blue light, a green light imaging unit that captures green light, and a red that captures light in the infrared wavelength band. An external light imaging unit,
There are two each of the red light imaging unit, the blue light imaging unit, the green light imaging unit, and the infrared light imaging unit.
An arithmetic circuit that compares and calculates image information output from each imaging region of the plurality of imaging units;
A camera module in which the lenses of the plurality of imaging units are integrated. The camera module according to claim 1, wherein the arithmetic circuit calculates a distance to the photographed object from image information output from the imaging regions of the two imaging units that capture light in the same wavelength band. The arithmetic circuit calculates a distance to the captured object from image information output from the imaging regions of the two imaging units that capture light in the same wavelength band, and based on the calculated distance, 2. The camera module according to claim 1, wherein image information of a color image obtained by combining image information output from respective imaging regions of the red light imaging unit, the blue light imaging unit, and the green light imaging unit is output. The camera module according to claim 1, wherein the two imaging units that image light in the same wavelength band are arranged obliquely with respect to a pixel array of the imaging region. The camera module according to claim 1, wherein the two imaging units that image light in the same wavelength band are arranged in parallel with a scanning line direction of the imaging region. The arithmetic circuit calculates image information between pixels by interpolation from image information output from the imaging region, and from image information output from the imaging region and image information between pixels calculated from the content, The camera module according to claim 2, wherein a distance to the photographed object is calculated. The camera module according to claim 1, further comprising a light emitting element for infrared illumination. The camera module according to claim 1, wherein the camera module is mounted on an automobile and obtains surrounding situation information.

Images