JP5590457B2 - IMAGING DEVICE, OBJECT DETECTION DEVICE HAVING THE SAME, AND ON-VEHICLE DEVICE CONTROL DEVICE - Google Patents

Description

  The present invention relates to an imaging device that images an imaging region using an image sensor configured by a pixel array in which light receiving elements are two-dimensionally arranged, an object detection device including the imaging device, and an in-vehicle device control device.

  Conventionally, at night, an in-vehicle camera (imaging device) captures a front area (imaging area) in the traveling direction of the host vehicle, and performs object detection processing based on the captured image to detect an object such as a preceding vehicle or an oncoming vehicle, A driver support system that reduces the driving load on the driver of the vehicle is known. Some driver support systems perform, for example, information provision control that notifies or warns the driver using detection results of preceding vehicles, oncoming vehicles, and the like. In addition, for example, by using the detection result of the preceding vehicle or the oncoming vehicle, control of switching the irradiation direction of the headlamp of the own vehicle (switching control between high beam and low beam, etc.), and control of ensuring visibility by the wiper of the own vehicle Some control the in-vehicle devices that perform the operation. Such a driver assistance system reduces the load on driving by the driver and contributes to traffic safety, and is a very useful system. However, the detection result of the preceding vehicle or the oncoming vehicle is used. In order to appropriately perform the various controls, it is important to improve the detection accuracy of the preceding vehicle, the oncoming vehicle, and the like.

  Patent Document 1 discloses a system that detects a tail lamp of a preceding vehicle and a headlamp of an oncoming vehicle based on an image captured by an image sensor as an object detection device that can detect the preceding vehicle and the oncoming vehicle at night. . This system extracts strong white light from the headlamp and strong red light from the taillamp, and detects an oncoming vehicle or a preceding vehicle from the extraction result. Specifically, a lens formed with a red filter dye and a lens formed with a greenish blue filter dye are arranged in the front stage of the image sensor, and the input light from the imaging region is respectively received by these lenses. Focusing is performed at two locations on the image sensor. As a result, the red component of the input light is extracted from the luminance value of the light focused by the former lens, and the blue component of the input light is extracted from the luminance value of the light focused by the latter lens. When the red component of the input light is equal to or higher than the predetermined brightness (luminance), it is assumed that the tail lamp of the preceding vehicle exists in the imaging region, and the tail lamp of the preceding vehicle is detected, and the blue component of the input light has the predetermined brightness. When the brightness is equal to or greater than (luminance), the headlamp of the oncoming vehicle is detected in the imaging region, and the headlamp of the oncoming vehicle is detected.

  However, in the system disclosed in Patent Document 1 described above, in order to extract red light from the tail lamp and white light (including a blue component) from the headlight, the input light is converted to red by two lenses having a filter function. Focus in blue. In such a configuration including a plurality of lenses, there is a problem that the optical system becomes complicated and the cost increases. In addition, the lens used in the system disclosed in Patent Document 1 uses a dye to realize a filter function. A filter using such a dye has a remarkable problem of deterioration of the filter function under an environment where the temperature change is severe, such as in the passenger compartment.

  As a method of extracting red light and white light that can solve such a problem, the semiconductor device is arranged on the image sensor using a semiconductor process similar to a known color sensor, instead of using the above-described two lenses. A method is conceivable in which a red component filter is stacked on a part of a pixel (unit region), and a pixel that receives only the red component and a pixel that receives white light are formed. However, such a semiconductor process requires an expensive investment, and an increase in cost is inevitable.

  In order to solve such a problem, an optical filter divided into a filter region (first region) that transmits only a red component and a non-filter region (second region) that does not limit the wavelength is used for a monochrome image sensor. A method of installing as a separate element in the previous stage is conceivable. This optical filter is formed by, for example, forming a wavelength selection filter (red component filter) that transmits only the red component uniformly on a glass plate by a method such as vapor deposition, and then lifting off the red component filter at a portion to be a non-filter region. It can be manufactured by a relatively simple and low-cost method of removing by such a method. Therefore, according to this method, it is possible to reduce the cost compared to the case where the red component filter is provided on the image sensor using the semiconductor process described above.

However, the present inventor has found that the following problems occur when this method is adopted.
In the case of an optical filter that partially forms a red component filter on a glass plate (filter substrate) and transmits red light and white light, a filter region in which the red component filter is formed, and such a red component filter A phase difference occurs in the transmitted light between the non-filter region where the wavelength is not selected and the wavelength is not selected. This is because a filter material that functions as a red component filter is present on the glass plate in the filter region, whereas an air layer is present instead of such a filter material in the non-filter region. Although the optical filter and the image sensor are arranged close to each other, it is impossible to press and bond them to each other. Therefore, there is a gap of about several microns between the optical filter and the image sensor. Therefore, a small part of the light transmitted through each filter area (or non-filter area) on the optical filter is received by surrounding pixels located around the corresponding pixel by diffraction phenomenon or refraction phenomenon when passing through the optical filter. Is done. At this time, if there is a phase difference between the transmitted light (red light) in the filter area and the transmitted light (white light) in the non-filter area, the surrounding pixels of the target pixel are compared to when there is no phase difference. The amount of light received by the light increases or decreases. This is because, for example, when a part of the transmitted light in the non-filter region is irradiated to the corresponding pixel corresponding to the filter region by diffraction or refraction, a part of the transmitted light in the non-filter region ( This is because the degree to which the diffracted light or refracted light) and the transmitted light through the filter region interfere and strengthen each other increases due to the phase difference. When the amount of light received by the surrounding pixels of the target pixel increases or decreases, the error between the amount of light received by the corresponding pixel and the amount of light transmitted through the filter region increases, and the detection accuracy of the amount of light transmitted through the filter region by the corresponding pixel is increased. Gets worse. The same applies to the corresponding pixels corresponding to the non-filter area. As a result, there arises a problem that, for example, the separation between the pixel corresponding to the filter region and the pixel corresponding to the non-filter region is not clear. This problem results in a decrease in the accuracy of the tail lamp detection based on the red light and the headlamp detection based on the white light in the above-described object detection apparatus using the imaging device including the optical filter.

  On the other hand, it is not impossible to solve the above-described problem by performing a process of correcting the diffraction phenomenon or the refraction phenomenon with respect to the amount of light received by each pixel. However, since the diffraction phenomenon and the refraction phenomenon have wavelength dependence, even if the diffraction phenomenon and the refraction phenomenon for the specific wavelength are corrected with respect to the received light amount of each pixel, other wavelengths are not corrected. Further, it is not practical to perform correction for a large number of wavelengths because it increases the processing load. Therefore, it is difficult to cope with the above problem by correcting the diffraction phenomenon and the refraction phenomenon with respect to the amount of light received by each pixel.

  In addition, although the said problem demonstrated the case where the 2nd area | region provided separately from the filter area | region which is a 1st area | region is a non-filter area | region which permeate | transmits light without performing wavelength selection, a 2nd area | region is the said area | region. The same problem may arise when the filter region is another filter region that selectively transmits only light in a predetermined wavelength range including the wavelength selected.

  The present invention has been made in view of the above problems, and an object of the present invention is to divide an optical filter, which is a separate element arranged in the previous stage of the image sensor, into a first area and a second area, thereby dividing the image sensor. An object of the present invention is to provide an imaging device in which the upper pixel is separated in accordance with the wavelength, and an object detection device and an in-vehicle device control device provided with the same.

In order to achieve the above object, according to the first aspect of the present invention, imaging is performed by receiving light emitted from an object existing in an imaging region by an image sensor including a pixel array in which light receiving elements are two-dimensionally arranged. an imaging device for imaging an area, and a second region where Ru transmits light without first region and a wavelength selection to transmit by selecting only light of a specific wavelength, one light receiving element on the image sensor Alternatively, an optical filter divided into unit areas composed of two or more light receiving element groups is arranged on the incident side of the image sensor, and the optical filter is a filter that becomes the first area on the filter substrate. After the film is uniformly formed, the filter film present at the position to be the second region is removed, and the portion from which the filter film has been removed is filled with a filler, so that the light transmitted through the first region is transmitted. Optical path length The optical path length of light passing through the second region is configured to be substantially the same, the optical filter, the image sensor side the filter film is formed in a state of being opposed to the image sensor And the filler is the adhesive .
Also, the invention of claim 2, the light emitted from the object present in the imaging area, and received by the image sensor constructed with a pixel array in which light receiving elements are arranged two-dimensionally, to image the imaging area In the imaging apparatus, the first region for selecting and transmitting only light of a specific wavelength and the second region for transmitting light without performing wavelength selection are one light receiving element or two or more light receiving elements on the image sensor. An optical filter divided into unit areas composed of groups is arranged on the incident side of the image sensor, and the optical filter is formed on the filter substrate having a convex portion at a location to be the second area. After the filter film to be the first region is uniformly formed, the filter film present at the location to be the second region is removed, so that the optical path length of the light transmitted through the first region and the second region are reduced. Optical path of light passing through Bets are characterized in that which has been configured to be substantially the same.
The invention of claim 3 is an object detection apparatus having an object detection processing means for performing detection processing of at least two types of detection objects that emit light in different wavelength bands based on a captured image captured by the imaging means. The imaging device according to claim 1 or 2 is used as the imaging means.
According to a fourth aspect of the present invention, there is provided the object detection device according to the third aspect , wherein the imaging device captures an area in front of the traveling direction of the host vehicle as an imaging region, and the at least two types of detection objects include The other vehicle that travels in the direction approaching the host vehicle and the other vehicle that travels in the direction away from the host vehicle are included, and the second region is a transmission region that transmits light without performing wavelength selection, The specific wavelength through which the first region transmits light is a wavelength of a tail lamp color of the vehicle, and the object detection processing means is based on an output signal of a light receiving element on the image sensor that has received the transmitted light of the first region. The other vehicle traveling in the direction away from the host vehicle is detected, and the other vehicle traveling in the direction approaching the host vehicle is detected based on the output signal of the light receiving element on the image sensor that has received the transmitted light of the second region. Specially It is an.
Further, the invention according to claim 5 is an object detection unit that captures an image of the periphery of a vehicle as an imaging region and detects at least two types of detection objects that emit light in different wavelength bands among objects existing in the imaging region. And a control means for controlling the operation of the in-vehicle equipment mounted on the vehicle based on the detection result of the object detection means. The in-vehicle equipment control device according to claim 3 or 4 as the object detection means. It is characterized by using an object detection device.
According to a sixth aspect of the present invention, in the in-vehicle device control apparatus according to the fifth aspect , the object detection device according to the fourth aspect is used as the object detection means, and the control means uses the detection result of the object detection means. Based on this, control is performed to change the irradiation direction of a headlamp which is an in-vehicle device mounted on the vehicle or to change the emission intensity of the headlamp.

The present invention comprises an optical path length of the light transmitted through the first region to transmit by selecting only light of a specific wavelength, the optical path length of the light transmitted through the second region Ru transmits light without wavelength selection The optical filter is configured to be substantially the same. Thereby, the phase difference produced between the transmitted light of the first region and the transmitted light of the second region can be reduced or eliminated. Such an optical filter can be produced, for example, by adjusting the thickness and the refractive index of the portion that becomes the first region and the portion that becomes the second region. If the phase difference between the transmitted light in the first region and the transmitted light in the second region is reduced or eliminated in this way, in the unit region corresponding to the first region, the diffracted light and the refracted light from the second region. It is possible to suppress an increase in the degree of strengthening or weakening due to interference between the light and the transmitted light in the first region. As a result, an increase in the error between the received light amount of the unit region corresponding to the first region and the transmitted light amount of the first region is suppressed, and the detection accuracy of the transmitted light amount of the first region by the unit region is deteriorated. It is suppressed. The same applies to the unit area corresponding to the second area. Therefore, it is possible to prevent the pixel corresponding to the first region and the pixel corresponding to the second region from being unclear.

  As described above, according to the present invention, when the optical filter of another element arranged in the previous stage of the image sensor is divided into the first region and the second region, and the pixels on the image sensor are separated according to the wavelength, the separation is performed. It is possible to obtain an excellent effect that does not become unclear.

It is a functional block diagram of the headlamp control system concerning an embodiment. It is explanatory drawing which shows schematic structure of the imaging device in the headlamp control system. It is an enlarged view which shows the optical filter and image sensor in the imaging device. It is explanatory drawing which shows the correspondence of the area | region division pattern of the same optical filter, and the pixel of the same image sensor. 6 is an explanatory diagram of an optical filter in Configuration Example 1. FIG. It is explanatory drawing which overlapped and showed the red filter area | region of the optical filter in the structural example 2 on an image sensor. It is a perspective view including the cross section of the optical filter in embodiment. It is a graph which shows an example of the cut filter characteristic which can be used as a wavelength filter film | membrane of a red filter area | region. It is a graph which shows an example of the band pass filter characteristic which can be used as a wavelength filter film | membrane of a red filter area | region. It is a schematic diagram which shows the optical filter and image sensor in a reference example. It is a schematic diagram which shows the optical filter and image sensor in embodiment. It is a schematic diagram which shows the optical filter and image sensor in the structural example 3. It is a schematic diagram which shows the optical filter and image sensor in the structural example 4. It is a flowchart which shows the flow of the vehicle detection process in embodiment.

Hereinafter, an embodiment in which an imaging device according to the present invention is used in a headlamp control system as an in-vehicle device control device will be described.
Note that the imaging device according to the present invention is not limited to the headlamp control system. For example, the object detection device detects an object to be detected such as a preceding vehicle or an oncoming vehicle at night and displays or warns the driver. The present invention can be similarly applied to a person support system, and can also be applied to other systems equipped with an object detection device that detects an object based on a captured image.

FIG. 1 is a functional block diagram of the headlamp control system according to the present embodiment.
This headlamp control system mainly includes an imaging device 11 as an imaging means, a vehicle behavior sensor 12, a vehicle detection control device 13 as an object detection device, and a headlamp control device 14 as a control means. ing.

  The imaging device 11 is mounted on the host vehicle so that the front area in the traveling direction of the host vehicle can be shot. Specifically, for example, it is attached to the vicinity of the windshield on the back side of the rearview mirror attached to the vehicle interior, built into the rearview mirror, or built into the headlamp. When the image pickup apparatus 11 is mounted, the image pickup direction of the image pickup apparatus 11 is adjusted and fixed so as to coincide with a predetermined reference direction (for example, the horizontal direction). The imaging device 11 is configured to be able to adjust the shutter speed, the frame rate, the gain of the digital signal output to the vehicle detection control device 13, and the like according to an instruction from a built-in control unit (not shown). Has been. And the imaging device 11 outputs the digital signal which shows the brightness (luminance) for every pixel of the image | photographed image as captured image data to the vehicle detection control apparatus 13 with the horizontal / vertical synchronizing signal of an image.

  The vehicle behavior sensor 12 is composed of, for example, a stroke sensor installed on a four-wheel suspension of the host vehicle, and detects behavior changes in the pitch direction and the roll direction of the host vehicle. When the host vehicle travels and acceleration occurs in the front, rear, left, and right directions, the body of the host vehicle swings in the pitch direction and the roll direction, and accordingly, the imaging direction of the imaging device 11 deviates from the reference direction. The vehicle behavior sensor 12 gives vehicle behavior information to the vehicle detection control device 13 for calculating how much the imaging direction of the imaging device 11 deviates from the reference direction.

  The vehicle detection control device 13 performs predetermined image processing using the captured image data input from the imaging device 11 and the above-described vehicle behavior information from the vehicle behavior sensor 12, so that the light source included in the image is detected. , Determine whether the tail lamp of the preceding vehicle (another vehicle traveling in a direction away from the host vehicle) or the headlamp of an oncoming vehicle (another vehicle traveling in a direction approaching the host vehicle) is applicable or not To do. When it is determined that the light source included in the image corresponds to the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle, detection information regarding the preceding vehicle or the oncoming vehicle is output to the headlamp control device 14.

  The headlamp control device 14 controls the direction of the headlamp (light irradiation direction) based on the detection information regarding the preceding vehicle and the oncoming vehicle input from the vehicle detection control device 13. For example, when the distance to the preceding vehicle or the oncoming vehicle specified based on the detection information is equal to or less than a predetermined distance, the headlamp direction is set to a low beam. This makes it difficult for the driver of the preceding vehicle or the oncoming vehicle to feel dazzled by the light of the headlamp of the host vehicle. On the other hand, when the distance to the preceding vehicle or the oncoming vehicle exceeds a predetermined distance, or when the preceding vehicle or the oncoming vehicle is not detected, the direction of the headlamp is set as a high beam. As a result, the field of view of the driver of the host vehicle can be secured even further at night. The headlamp control device 14 can switch the headlamp orientation OFF by the user, and can manually switch between the high beam and the low beam according to the user's recognition.

FIG. 2 is an explanatory diagram illustrating a schematic configuration of the imaging apparatus 11 according to the present embodiment.
The imaging device 11 mainly includes an imaging lens 1, an optical filter 2, a sensor substrate 3 including an image sensor 4 having a two-dimensionally arranged pixel array, and an analog electrical signal output from the sensor substrate 3 ( The signal processing unit 5 generates and outputs captured image data obtained by converting a received light amount received by each light receiving element on the image sensor 4 into a digital electric signal. Light from the imaging region including the subject (test object) passes through the imaging lens 1, passes through the optical filter 2, and is converted into an electrical signal corresponding to the light intensity by the image sensor 4. When the electrical signal (analog signal) output from the image sensor 4 is input to the signal processing unit 5, the brightness (luminance) of each pixel on the image sensor 4 is indicated as captured image data from the electrical signal. The digital signal is output to the subsequent device together with the horizontal / vertical synchronizing signal of the image.

FIG. 3 is an enlarged view showing the optical filter 2 and the image sensor 4.
The image sensor 4 is an image sensor using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and a photodiode 4a is used as a light receiving element thereof. The photodiodes 4a are two-dimensionally arranged for each pixel, and a microlens 4b is provided on the incident side of each photodiode 4a in order to increase the light collection efficiency of the photodiodes 4a. The image sensor 4 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 3. Since the wavelength range of the image to be captured is substantially visible light, an image sensor 4 having sensitivity in the visible light range may be selected.

  The optical filter 2 is disposed close to the surface of the image sensor 4 on the microlens 4b side. As shown in FIG. 4, the surface of the optical filter 2 on the side of the image sensor 4 selectively transmits only a transmissive region 2a as a second region that transmits light and a red wavelength component (specific wavelength). The red filter area 2b as one area has a stripe-shaped area division pattern alternately arranged in a specific direction (left and right direction in FIG. 4). The optical filter 2 having such a striped pattern has an advantage that the mounting accuracy is eased by adjusting the position only in the specific direction when adjusting the position with the image sensor 4.

  As a method for producing such an optical filter 2, a wavelength filter film that transmits only a uniform red component is formed on a glass plate as a filter substrate by a method such as vapor deposition, and then a wavelength filter is formed by a method such as lift-off. The film may be partially removed in a stripe pattern. In this case, the portion where the wavelength filter film is removed becomes the transmission region 2a, and the portion where the wavelength filter film remains is the red filter region 2b.

  In the present embodiment, the pixels (photodiode 4a) on the image sensor 4 are made into one group with two pixels, and each group has a wavelength band (red component) of the tail lamp color (red) of the vehicle as shown in FIG. And a pixel W that receives light in the wavelength band (white component) of the headlamp color (white) of the vehicle (that is, light that is not wavelength-selected). That is, each pixel on the image sensor 4 functions as one of the two types of pixels R and W. As a result, two types of images with two pixels R and W on the image sensor 4 as one pixel can be obtained. That is, a red image corresponding to the pixel R and a monochrome luminance image corresponding to the pixel W.

[Configuration example 1]
Here, a configuration example of the optical filter 2 in the present embodiment (hereinafter, this configuration example will be referred to as “configuration example 1”) will be described.
FIG. 5 is an explanatory diagram of the optical filter 2 in the first configuration example.
In the optical filter 2 according to Configuration Example 1, the transmission region 2a and the red filter region 2b are illustrated in FIG. 5 so that each pixel on the image sensor 4 functions as one of the two types of pixels R and W described above. As shown, they are arranged in a checkered pattern so as to be alternately adjacent in the two-dimensional direction. As a result, the pixels on the image sensor 4 are divided into regions such that the pixels R that receive the red component and the pixels W that receive the white component are arranged in a checkered pattern so that they are alternately adjacent in the two-dimensional direction. .

  Even when the optical filter 2 of the present configuration example 1 is used, a red image corresponding to the pixel R and a monochrome luminance image corresponding to the pixel W are obtained with the two pixels R and W on the image sensor 4 as one pixel. be able to.

[Configuration example 2]
Further, another configuration example of the optical filter 2 in the present embodiment (hereinafter referred to as “configuration example 2”) will be described.
FIG. 6 is an explanatory diagram showing the red filter region 2 b of the optical filter 2 in the second configuration example superimposed on the image sensor 4.
The optical filter 2 according to Configuration Example 2 is the same as the above embodiment in that the transmission regions 2a and the red filter regions 2b are arranged so as to be alternately adjacent in the one-dimensional direction and are divided into stripes. is there. However, in the above embodiment, the long direction of the red filter region 2b is parallel to the pixel column on the image sensor 4, whereas in the present configuration example 2, the long direction of the red filter region 2b is the image sensor 4. The difference is that it is oblique to the upper pixel row. In the optical filter 2 of Configuration Example 2, as shown in FIG. 6, the red filter region 2b has a width corresponding to one pixel of the image sensor 4 in the horizontal direction in the figure, and two of the image sensor 4 in the vertical direction in the figure. The pixel has a width of one pixel and is arranged obliquely with respect to the pixel column on the image sensor 4 so that one pixel in the horizontal direction in the figure is displaced by two pixels in the vertical direction in the figure.

  Even when the optical filter 2 of the present configuration example 2 is used, a red image corresponding to the pixel R and a monochrome luminance image corresponding to the pixel W are obtained with the two pixels R and W on the image sensor 4 as one pixel. be able to.

Next, the configuration of the optical filter 2 will be described in detail.
FIG. 7 is a perspective view including a cross section of the optical filter 2 in the present embodiment.
As shown in FIG. 7, in the optical filter 2, the portion where the red wavelength filter film 2d is formed on the glass flat plate 2c (that is, the red filter region 2b) becomes a convex portion, and the wavelength filter film is removed from the glass flat plate. The portion (that is, the transmission region 2a) has a concave and convex periodic structure. The wavelength filter film 2d in this embodiment has a multilayer structure in which high refractive index thin films and low refractive index thin films are alternately stacked. Using such a multi-layered wavelength filter film, the spectral transmittance can be set freely by utilizing the interference of light, and a reflectivity close to 100% with respect to a specific wavelength can be obtained by stacking many thin films. You can also. Here, light in a wavelength band other than the red component is reflected.

In the present embodiment, the transmission wavelength range of the wavelength filter film 2d of the optical filter 2 is a red wavelength component. Since the wavelength range of the image to be captured is substantially visible light range, for example, a wavelength filter of a cut filter having filter characteristics as shown in FIG. 8 that transmits light having a wavelength of 600 nm or more and cuts a wavelength of less than 600 nm. The film 2d may be formed on the glass flat plate 2c. Such a cut filter can be obtained by preparing a multilayer film having a structure of “substrate / (0.125L0.25H0.125L) p / medium A” in order from the glass flat plate side. Here, “0.125L” is a film thickness marking method of a low refractive index material (for example, SiO ₂ ) in which nd / λ is 1 L. Therefore, a film of “0.125L” has a 1/8 wavelength. It means a film of a low refractive index material having a film thickness that becomes an optical path length. “N” is a refractive index, “d” is a thickness, and λ is a cutoff wavelength. Similarly, “0.25H” is obtained by setting nd / λ to 1H in the film thickness marking method of a high refractive index material (for example, TiO ₂ ). Therefore, the film of “0.25H” has a quarter wavelength. It means a film of a high refractive index material having a film thickness that becomes an optical path length. “P” indicates the number of times the film combination shown in parentheses is repeated (laminated), and the more “p”, the more the influence of ripple and the like can be suppressed. The medium A is intended for air or a filler such as a resin or an adhesive described later.

Further, the wavelength filter film 2d of the optical filter 2 may be a band-pass filter having a filter characteristic as shown in FIG. 9 that transmits only the transmission wavelength range of 600 to 700 nm. With such a bandpass filter, it is possible to distinguish the near-infrared region and red region on the longer wavelength side than red. Such a band pass filter is, for example, “substrate / (0.125L0.5M0.125L) p (0.125L0.5H0.125L) q (0.125L0.5M0.125L) r / medium A”. It can be obtained by producing a multilayer film having the structure. As described above, if titanium dioxide (TiO ₂ ) is used as the high refractive index material and silicon dioxide (SiO ₂ ) is used as the low refractive index material, an optical filter with high weather resistance can be realized.

  An example of a method for producing the optical filter 2 of the present embodiment will be described. First, the above-described multi-layered wavelength filter film 2d is formed on the glass flat plate 2c. As a method for forming such a multilayer film, a well-known method such as vapor deposition may be used. Subsequently, the wavelength filter film is removed at a location corresponding to the transmission region 2a. As this removal method, a general lift-off processing method may be used. In the lift-off processing method, a pattern opposite to the target pattern is formed in advance on the glass flat plate 2c with a metal, a photoresist, etc., and a wavelength filter film is formed thereon, and then the transmission region 2a is supported. The portion of the wavelength filter film to be removed is removed together with the metal and photoresist.

  The optical filter 2 of this embodiment including these structural examples 1 and 2 is installed in the front stage of the image sensor 4. In the installation method, the concave-convex structure surface of the optical filter 2 is arranged in the proximity so as to face the light receiving surface of the image sensor 4 and then fixed. As a fixing method, a spacer is disposed around the light receiving surface of the image sensor 4 to hold the optical filter 2, or an adhesive is filled between the grating surface of the optical filter 2 and the light receiving element of the image sensor 4. The method of adhering is mentioned.

  As a method of attaching the optical filter 2 to the image sensor 4, a method of fixing the optical filter 2 to the sensor substrate 3 installed on the PWB by a method such as wire bonding (ASSY) later, or an optical filter 2. And a method of mounting the image sensor 4 fixed in advance on the PWB. More desirably, it can be manufactured at a low cost by dicing a wafer obtained by bonding the wafer of the image sensor 4 and the wafer on which the optical filter 2 is formed. In this embodiment, since the positional accuracy between each wafer is not required, it can be manufactured with a high yield.

FIG. 10 is a schematic diagram showing the optical filter 2 and the image sensor 4 in the reference example.
When the red filter region 2b of the optical filter 2 is opposed to only a part of the pixels on the image sensor 4 as in this embodiment, the transmission region 2a in which the multilayer wavelength filter film 2d is not formed is a simple air layer. As described above, a phase difference occurs in the transmitted light between the red filter region 2b in which the multilayer wavelength filter film 2d is formed and the transmission region 2a. Although the image sensor 4 and the optical filter 2 are arranged close to each other, they cannot be pressed and joined, so that they are arranged at a certain distance (several microns) apart. Therefore, a small part of the light transmitted through each area on the image sensor 4 is received by surrounding pixels located around the counter pixel due to diffraction phenomenon when passing through the optical filter. At this time, if there is a phase difference of the transmitted light between the red filter region 2b and the transmission region 2a, the amount of light received by the surrounding pixels of the counter pixel increases as compared with the case where no phase difference occurs. . As a result, for example, the separation between the pixel facing the red filter region 2b and the pixel facing the transmission region 2a is not clear.

FIG. 11 is a schematic diagram showing the optical filter 2 and the image sensor 4 in the present embodiment.
In this embodiment, as shown in FIG. 11, the recess (transmission region 2a) in the optical filter 2 of the reference example shown in FIG. 10 is filled with resin (filler) 2e, and the uneven surface thereof is made flat. An optical filter 2 is used. That is, in the present embodiment, instead of the air layer present in the concave portion of the transmission region 2a of the reference example, the concave portion is filled with a resin 2e having a refractive index different from that of the air layer, whereby the red filter region 2b. And a transmission region 2a are prevented from causing a phase difference of transmitted light.

In order to prevent this phase difference from occurring, there should be no optical path length difference in the transmitted light beam between the red filter region 2b in which the multilayer wavelength filter film 2d is interposed and the transmission region 2a in which the resin 2e is interposed. . Here, the average optical path length L1 of the light beam passing through the multilayer wavelength filter film 2d (red filter region 2b) is, for example, as follows when the wavelength filter film 2d is a cut filter having the characteristics shown in FIG. It can be obtained from equation (1).
L1 = (low refractive index material 1/8 wavelength + high refractive index material 1/4 wavelength + low refractive index material 1/8 wavelength) × p (1)

  In the present embodiment, the average optical path length L2 of the light beam passing through the resin 2e (transmission region 2a) is configured to match the average optical path length L1. The average optical path length L2 of the light beam passing through the resin 2e can be calculated by integrating the refractive index of the resin material and the thickness of the resin (that is, the thickness of the multilayer wavelength filter film 2d). As for the refractive index of the resin 2e, the refractive index of the resin itself can be selected. However, in the case of using a resin whose refractive index is controlled by mixing metal fine particles (mainly heavy metal fine particles) into the resin material. Then, the refractive index after the adjustment is selected.

[Configuration example 3]
Next, still another configuration example of the optical filter 2 in the present embodiment (hereinafter referred to as “configuration example 3”) will be described.
FIG. 12 is a schematic diagram showing the optical filter 2 and the image sensor 4 in the third configuration example.
In this configuration example 3, as a fixing method for fixing the optical filter 2 to the image sensor 4, a method of filling the adhesive 2f between the concave and convex surface of the optical filter 2 and the image sensor 4 and bonding the optical filter 2 is adopted. The role of the resin 2e in the above embodiment is given to the adhesive 2f filled in the two recesses. In the configuration example 3, since the adhesive used for fixing the optical filter 2 to the image sensor 4 is also used as a filler for adjusting the optical path length, the structure can be simplified and the manufacturing process can be simplified. Can do.

[Configuration Example 4]
Next, still another configuration example of the optical filter 2 in the present embodiment (hereinafter referred to as “configuration example 4”) will be described.
FIG. 13 is a schematic diagram illustrating the optical filter 2 and the image sensor 4 in the fourth configuration example.
In this configuration example 4, the optical path length of the transmitted light between the transmission region 2a and the red filter region 2b is matched without filling the concave portion of the optical filter 2 with a filler such as the resin 2e or the adhesive 2f. Make adjustments. Specifically, in the optical filter 2 of Configuration Example 4, the portion that becomes the red filter region 2b is a concave portion and the portion that becomes the transmission region 2a is a convex portion on the surface of the glass flat plate (filter substrate) 2c on the image sensor side. After forming the irregular periodic structure, a uniform multilayer wavelength filter film 2d is formed, and then the wavelength filter film formed on the convex portion of the glass flat plate 2c is removed.

  Also in the present configuration example 4, there is no phase difference between the red filter region 2b in which the multilayer wavelength filter film 2d is formed and the transmission region 2a in which the wavelength filter film 2d is not formed. I am doing so. Specifically, the refractive index of the glass flat plate 2c and the depth Z of the concave portion are selected so that the optical path lengths of both transmitted light coincide.

According to the fourth configuration example, the optical path length is adjusted without using a filler such as the resin 2e having relatively low weather resistance, so that the optical filter 2 having high weather resistance can be realized.
In the fourth configuration example, as a fixing method for fixing the optical filter 2 to the image sensor 4, a method in which an adhesive is filled between the concave and convex surface of the optical filter 2 and the image sensor 4 is used. Also good. In this case, it is necessary to adjust the optical path length in consideration of the refractive index of the adhesive and the thickness of the portion where the adhesive enters.

  Various information can be obtained from the captured image data (pixel data group indicating the amount of light received by each light receiving element on the image sensor 4) captured by the imaging device 11 as described above. Specifically, the intensity (brightness information) of light emitted from each point (light source) in the imaging region, the distance (distance information) between the light source (other vehicle) such as a headlamp and tail lamp and the own vehicle, Spectral information of the red component and white component (non-spectral) of the light emitted from the light source can be obtained.

  The brightness information will be described. When the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are photographed by the imaging device 11, the detected object is indicated on the photographed image data. One headlamp of the oncoming vehicle is projected brightest, and the tail lamp of the preceding vehicle, which is one of the detection objects, is projected darker than that. In addition, when the reflector is displayed in the captured image data, the reflector is not a light source that emits light by itself, but merely a bright image that is reflected by reflecting the headlamp of the host vehicle, so that it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the reflector is gradually darkened on the image sensor 4 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, primary identification of the two types of detection objects (head lamp and tail lamp) and the reflector is possible.

  Further, the distance information will be described. Most headlamps and tail lamps have a pair of left and right pair lamps. Therefore, the distance between the headlamps and tail lamps (that is, other vehicles) and the host vehicle is utilized using the characteristics of this configuration. Can be obtained. The pair of left and right lamps that are paired are displayed close to each other on the same image in the height direction on the captured image data captured by the imaging device 11, and the widths of the lamp image areas that project the lamps are substantially the same. The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, lamp image areas satisfying the conditions can be identified as pair lamps. At a long distance, the left and right lamps constituting the pair lamp cannot be distinguished and recognized as a single lamp.

  When a pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp and tail lamp which are the pair lamp structure. That is, the distance between the left and right headlamps and the distance between the left and right tail lamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the imaging lens 1 in the imaging device 11 is known, the distance w1 between the two lamp image areas respectively corresponding to the left and right lamps on the image sensor 4 of the imaging device 11 is calculated from the captured image data. Accordingly, the distance x between the light source of the headlamp or taillamp which is the pair lamp configuration and the host vehicle can be obtained by simple proportional calculation (x = f × w0 / w1). If the distance x calculated in this way is within an appropriate range, the two lamp image areas used for the calculation can be identified as headlamps and taillamps of other vehicles. Therefore, by using this distance information, the identification accuracy of the head lamp and tail lamp, which are detection objects, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, by extracting only pixel data corresponding to the pixel R on the image sensor 4 from the captured image data captured by the imaging device 11, It is possible to generate a red image in which only the red component is projected. Similarly, by extracting only pixel data corresponding to the pixel W on the image sensor 4 from the captured image data captured by the imaging device 11, a monochrome luminance image in the imaging region can be generated. Therefore, when there is an image area having a luminance equal to or higher than a predetermined luminance in the red image, the image area can be identified as a tail lamp image area in which a tail lamp is projected. Furthermore, it is possible to calculate the luminance ratio (red luminance ratio) between the image area on the red image and the image area on the monochrome luminance image corresponding to the image area. By using this red luminance ratio, it is possible to grasp the ratio of the relative red component contained in the light emitted from the object (light source) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the identification accuracy of the tail lamp is improved by using this red luminance ratio.

Next, a flow of detection processing for the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 14 is a flowchart showing a flow of vehicle detection processing in the present embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the imaging device 11, and an image region that is considered to be a detection target is extracted. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of detection objects is the type of light source displayed in the image area.

  First, in step S1, image data in front of the host vehicle imaged by the image sensor 4 of the imaging device 11 is taken into a memory. As described above, this image data includes a signal indicating the luminance in each pixel of the image sensor 4. Next, in step S <b> 2, information regarding the behavior of the host vehicle is acquired from the vehicle behavior sensor 12.

  In step S3, an image area with high brightness (high brightness image area) that is considered to be a detection target (tail lamp of the preceding vehicle and bed lamp of the oncoming vehicle) is extracted from the image data captured in the memory. This high luminance image region is a bright region having a luminance higher than a predetermined threshold luminance in the image data, and there are many cases where a plurality of high luminance image regions are extracted, but all of them are extracted.

  In the high-luminance image region extraction processing, first, in step S31, binarization processing is performed by comparing the luminance value of each pixel on the image sensor 4 with a predetermined threshold luminance. Specifically, a binary image is created by assigning “1” to pixels having a luminance equal to or higher than a predetermined threshold luminance and assigning “0” to other pixels. Next, in step S32, when pixels assigned with “1” are close to each other in the binarized image, a labeling process for recognizing them as one high luminance image region is performed. Thereby, a set of a plurality of adjacent pixels having a high luminance value is extracted as one high luminance image region.

  In step S4, which is executed after the above-described high-intensity image region extraction process, the distance between the object in the imaging region corresponding to each extracted high-intensity image region and the host vehicle is calculated. In this distance calculation process, the pair lamp distance calculation process that detects the distance by using the pair of left and right lamps of the vehicle and the left and right lamps constituting the pair lamp cannot be distinguished and recognized at a long distance. A single lamp distance calculation process is executed when the pair lamp is recognized as a single lamp.

  First, for a pair lamp distance calculation process, in step S41, a pair lamp creation process, which is a process of creating a lamp pair, is performed. The pair of left and right lamps that are paired are in the position where they are close to each other and have substantially the same height in the image data captured by the imaging device 11, the area of the high-luminance image region is substantially the same, and the shape of the high-luminance image region is the same. Satisfy the condition of being the same. Therefore, the high-intensity image areas that satisfy such conditions are used as a pair lamp. High brightness image areas that cannot be paired are considered a single lamp. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation process in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f in the imaging device 11 is known, the actual distance x to the pair lamps is calculated by a simple proportional calculation (x = f · w0 / w1). It should be noted that a dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio between the red image and the monochrome luminance image is used as spectral information. From this spectral information, whether the two high-intensity image regions that are paired lamps are due to light from the headlamp or light from the tail lamp. Lamp type identification processing is performed to identify whether or not In this lamp type identification process, first, in step S51, the ratio of the pixel data corresponding to the pixel R on the image sensor 4 to the pixel data corresponding to the pixel W on the image sensor 4 for the high-intensity image area that is a pair lamp. A red ratio image with the pixel value as is created. In step S52, the pixel value of the red ratio image is compared with a predetermined threshold, and a high-luminance image region that is equal to or higher than the predetermined threshold is determined to be a tail lamp image region by light from the tail lamp. For a certain high-intensity image area, a lamp type process is performed to determine that the area is a headlamp image area by light from the headlamp.

In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection process as described above are used for controlling the light irradiation direction of the headlamp that is an in-vehicle device of the host vehicle.
Specifically, when the tail lamp is detected by the vehicle detection process and approaches the distance range A where the headlight illumination light of the host vehicle is incident on the rearview mirror of the preceding vehicle, the headlight of the host vehicle is approached to the preceding vehicle. Control is performed to block a part of the headlight of the host vehicle or to shift the light irradiation direction of the headlight of the host vehicle in the vertical direction or the horizontal direction so that the illumination light does not strike.
Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range B where the headlight illumination light of the own vehicle hits, the headlight illumination light of the own vehicle is approached to the oncoming vehicle. In order not to hit the vehicle, a part of the headlight of the host vehicle is blocked, or the light irradiation direction of the headlight of the host vehicle is shifted in the vertical direction or the horizontal direction.

In the above description, the imaging device 11 according to the present invention is applied to an object detection device used in a headlamp control system as an in-vehicle device control device. However, the present invention is not limited to this. For example, the imaging device 11 according to the present invention controls the display monitor which is an in-vehicle device to warn of the approach of another vehicle, avoids the own vehicle colliding with another vehicle, or reduces the impact at the time of the collision The present invention includes an example in which a brake that is an in-vehicle device is controlled in order to automatically adjust the speed of the host vehicle by controlling an accelerator adjusting device that is an in-vehicle device in order to maintain the inter-vehicle distance from the preceding vehicle. Can be applied to various in-vehicle device control devices.
In addition, the present invention is not limited to such an in-vehicle device control apparatus, and can be widely used as long as it is an apparatus that uses a plurality of types of image data having different wavelengths by separating pixels on the image sensor according to the wavelength.
In the above description, the case where the second region provided separately from the red filter region 2b that is the first region is the transmission region 2a that transmits light without performing wavelength selection has been described. As the filter region, the red filter region 2b may selectively transmit only a predetermined range of light including the red wavelength.

As described above, the headlamp control system according to the present embodiment captures the surroundings of the host vehicle as an imaging region, and includes at least two types of detection objects that emit light in different wavelength bands among objects existing in the imaging region. Controls the operation of an object detection device that is an object detection means for detecting a headlamp of an oncoming vehicle and a tail lamp of a preceding vehicle, and a headlamp that is an in-vehicle device mounted on the host vehicle based on the detection result of the object detection device. It is an in-vehicle device control device having a headlamp control device 14 as control means. The object detection device used in the headlamp control system includes an imaging device 11 as an imaging unit that receives light emitted from an object existing in the imaging region by the image sensor 4 and images the imaging region. It has a vehicle detection control device 13 as an object detection processing means for detecting a headlamp of an oncoming vehicle and a tail lamp of a preceding vehicle based on a captured image captured by the imaging device 11. The imaging device 11 used in the vehicle detection control device 13 receives light emitted from an object existing in the imaging region by the image sensor 4 including a pixel array in which the light receiving elements 4a are two-dimensionally arranged, and the imaging region. A red filter region 2b as a first region for selecting and transmitting only light having a red wavelength that is a specific wavelength, and a transmission region as a second region for transmitting light without performing wavelength selection. The optical filter 2 is divided into unit areas composed of one light receiving element or two or more light receiving element groups on the image sensor 4 on the incident side of the image sensor. The red filter region 2b and the transmission region 2a in the optical filter 2 are formed so that the optical path length of the light transmitted through the red filter region 2b and the optical path length of the light transmitted through the transmission region 2a are substantially the same. . Thereby, it is possible to prevent the phase difference of the transmitted light between the red filter region 2b and the transmission region 2a from occurring, and the pixel facing the red filter region 2b and the pixel facing the transmission region 2a It can be clearly separated, and the identification accuracy of the tail lamp and the head lamp is improved.
In the present embodiment (excluding the above configuration example 4), the optical filter is formed by uniformly forming the wavelength filter film 2d to be the red filter region 2b on the glass plate 2c that is a filter substrate, and then transmitting the transmission region 2a. By removing the wavelength filter film existing at the location and filling the location from which the wavelength filter film is removed with resin 2e or adhesive 2f as a filler, the optical path length and transmission of the light transmitted through the red filter region 2b The optical path length of the light transmitted through the region 2a can be configured to be substantially the same.
In particular, in the configuration example 3 described above, the optical filter 2 with respect to the image sensor 4 is bonded to the image sensor 4 with the adhesive 2f with the side on which the wavelength filter film 2d is formed facing the image sensor 4. Since the adhesive 2f used for fixing can also be used as a filler for adjusting the optical path length, the structure can be simplified and the manufacturing process can be simplified.
In the above configuration example 4, after the wavelength filter film to be the red filter region 2b is uniformly formed on the filter substrate having the convex portion at the portion to be the transmission region 2a, the wavelength existing at the location to be the transmission region 2a By removing the filter film, the optical path length of the light transmitted through the red filter region 2b and the optical path length of the light transmitted through the transmission region 2a are made substantially the same. In this configuration, it is possible to adjust the optical path length without using a filler such as the resin 2e having relatively low weather resistance, and it is possible to realize the optical filter 2 having high weather resistance.

DESCRIPTION OF SYMBOLS 1 Imaging lens 2 Optical filter 2a Transmission area 2b Red filter area 2c Glass plate 2d Wavelength filter film 2e Resin 2f Adhesive 3 Sensor substrate 4 Image sensor 4a Photodiode (light receiving element)
DESCRIPTION OF SYMBOLS 5 Signal processing part 11 Imaging apparatus 12 Vehicle behavior sensor 13 Vehicle detection control apparatus 14 Headlamp control apparatus

JP 2004-189229 A

Claims (6)

In an imaging device that receives light emitted from an object existing in an imaging region by an image sensor configured by a pixel array in which light receiving elements are two-dimensionally arranged, and images the imaging region.
And a second region where Ru transmits light without first region and the wavelength selection to transmit by selecting only light of a specific wavelength, consists of one light receiving element or two or more light receiving element group on the image sensor An optical filter divided into unit areas is disposed on the incident side of the image sensor,
The optical filter is formed by uniformly forming the filter film to be the first region on the filter substrate, and then removing the filter film present at the location to be the second region, and filling the location from which the filter film has been removed. By filling the agent, the optical path length of the light transmitted through the first region and the optical path length of the light transmitted through the second region are configured to be substantially the same ,
The optical filter is adhered to the image sensor with an adhesive in a state where the side on which the filter film is formed faces the image sensor,
The imaging apparatus, wherein the filler is the adhesive .   In an imaging device that receives light emitted from an object existing in an imaging region by an image sensor configured by a pixel array in which light receiving elements are two-dimensionally arranged, and images the imaging region.
  The first region for selecting and transmitting only light of a specific wavelength and the second region for transmitting light without performing wavelength selection are configured by one light receiving element or two or more light receiving element groups on the image sensor. An optical filter divided into unit areas is disposed on the incident side of the image sensor,
  The optical filter is formed by uniformly forming a filter film serving as the first region on a filter substrate having a convex portion at the site serving as the second region, and then presenting the filter film present at the site serving as the second region. The image pickup apparatus is configured so that the optical path length of the light transmitted through the first region and the optical path length of the light transmitted through the second region are substantially the same by removing . In an object detection apparatus having an object detection processing unit that performs detection processing of at least two types of detection objects that emit light in different wavelength bands based on a captured image captured by an imaging unit,
An object detection apparatus using the imaging apparatus according to claim 1 or 2 as the imaging means. The object detection device according to claim 3 .
The imaging device captures an area in the traveling direction of the host vehicle as an imaging area,
The at least two types of detection objects include an other vehicle that travels in a direction approaching the host vehicle and another vehicle that travels in a direction away from the host vehicle,
The second region is a transmission region that transmits light without performing wavelength selection,
The specific wavelength through which the first region transmits light is a wavelength of a tail lamp color of a vehicle,
The object detection processing means detects another vehicle traveling in a direction away from the own vehicle based on an output signal of a light receiving element on an image sensor that has received the transmitted light of the first region, and transmits the transmitted light of the second region. An object detection apparatus for detecting another vehicle traveling in a direction approaching the host vehicle based on an output signal of a light receiving element on an image sensor that receives the light. An object detection means for imaging the periphery of the vehicle as an imaging region, and detecting at least two types of detection objects that emit light in different wavelength bands among the objects present in the imaging region;
In an in-vehicle device control device having a control means for controlling the operation of the in-vehicle device mounted on the vehicle based on the detection result of the object detection means,
An in-vehicle device control device using the object detection device according to claim 3 or 4 as the object detection means. In the in-vehicle device control device according to claim 5 ,
The object detection device according to claim 4 is used as the object detection means.
The control means performs control to change the irradiation direction of a headlamp which is an in-vehicle device mounted on the vehicle or to change the emission intensity of the headlamp based on the detection result of the object detection means. In-vehicle device control device.

Images